2024-04-25T16:36:09.633 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T16:36:09.634 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --owner scheduled_pdonnell@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} --verbose --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280 -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280/orig.config.yaml 2024-04-25T16:36:09.678 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T16:36:09.775 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} email: pdonnell@redhat.com first_in_suite: false job_id: '7673280' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGu8NGEuGQkb1jV9Xb27i1LpCP0Jj07s6pidfWmZr3LyeUMosqY0oO8/gJWyAfPqsll1QwH4HI+dPvOXZsNhRbE= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBFEDbrXlp5aEZYUnjRaYzIiE2ZC7qbzwLmxJvka+WOTYpvlaj5Y/OswrFC2ZJZx2t7z6yFZqE062kaVp4nguaU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T16:36:09.775 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T16:36:09.776 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T16:36:09.776 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T16:36:09.778 INFO:teuthology.task.internal:Checking packages... 2024-04-25T16:36:09.801 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T16:36:09.801 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T16:36:09.801 INFO:teuthology.packaging:ref: None 2024-04-25T16:36:09.801 INFO:teuthology.packaging:tag: None 2024-04-25T16:36:09.801 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T16:36:09.801 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:36:09.802 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T16:36:09.950 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291-g994e2f20-1jammy 2024-04-25T16:36:09.952 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T16:36:09.958 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T16:36:09.959 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T16:36:09.997 INFO:teuthology.task.internal:Saving configuration 2024-04-25T16:36:10.013 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T16:36:10.018 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T16:36:10.042 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:30:25.219560', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPVzVeKWRFqcujM19JCZYyIfvxioxkeulf/UMMdkn6NQ'} 2024-04-25T16:36:10.066 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:30:25.220807', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIfV+F9cJom12piw8jk/3w7zNHXWDG7VlOfFUHFN7xph'} 2024-04-25T16:36:10.066 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T16:36:10.072 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T16:36:10.072 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T16:36:10.073 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T16:36:10.155 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f255e8b33a0>, signals=[15]) 2024-04-25T16:36:10.155 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T16:36:10.161 INFO:teuthology.task.internal:Opening connections... 2024-04-25T16:36:10.161 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:36:10.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:36:10.244 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:36:10.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:36:10.322 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T16:36:10.329 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-04-25T16:36:10.335 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-04-25T16:36:10.335 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-04-25T16:36:10.382 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T16:36:10.382 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="Ubuntu" 2024-04-25T16:36:10.382 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="22.04" 2024-04-25T16:36:10.382 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T16:36:10.382 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_CODENAME=jammy 2024-04-25T16:36:10.382 INFO:teuthology.orchestra.run.smithi094.stdout:ID=ubuntu 2024-04-25T16:36:10.382 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE=debian 2024-04-25T16:36:10.382 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T16:36:10.382 INFO:teuthology.orchestra.run.smithi094.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T16:36:10.382 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T16:36:10.383 INFO:teuthology.orchestra.run.smithi094.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T16:36:10.383 INFO:teuthology.orchestra.run.smithi094.stdout:UBUNTU_CODENAME=jammy 2024-04-25T16:36:10.383 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-04-25T16:36:10.414 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-04-25T16:36:10.420 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-04-25T16:36:10.420 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-04-25T16:36:10.466 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T16:36:10.466 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="Ubuntu" 2024-04-25T16:36:10.466 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="22.04" 2024-04-25T16:36:10.466 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T16:36:10.466 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_CODENAME=jammy 2024-04-25T16:36:10.466 INFO:teuthology.orchestra.run.smithi138.stdout:ID=ubuntu 2024-04-25T16:36:10.466 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE=debian 2024-04-25T16:36:10.466 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T16:36:10.467 INFO:teuthology.orchestra.run.smithi138.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T16:36:10.467 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T16:36:10.467 INFO:teuthology.orchestra.run.smithi138.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T16:36:10.467 INFO:teuthology.orchestra.run.smithi138.stdout:UBUNTU_CODENAME=jammy 2024-04-25T16:36:10.467 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-04-25T16:36:10.489 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T16:36:10.498 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T16:36:10.539 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T16:36:10.540 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-04-25T16:36:10.542 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-04-25T16:36:10.547 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T16:36:10.554 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T16:36:10.554 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-04-25T16:36:10.587 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-04-25T16:36:10.671 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T16:36:10.729 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T16:36:10.742 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T16:36:10.742 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T16:36:10.743 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T16:36:10.743 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T16:36:10.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:36:10.747 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-04-25T16:36:10.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:36:10.748 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-04-25T16:36:10.794 INFO:teuthology.orchestra.run.smithi094.stdout:5.15.0-56-generic 2024-04-25T16:36:10.794 INFO:teuthology.task.kernel:Running kernel on smithi094: 5.15.0-56-generic 2024-04-25T16:36:10.795 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get clean 2024-04-25T16:36:10.796 INFO:teuthology.orchestra.run.smithi138.stdout:5.15.0-56-generic 2024-04-25T16:36:10.796 INFO:teuthology.task.kernel:Running kernel on smithi138: 5.15.0-56-generic 2024-04-25T16:36:10.796 DEBUG:teuthology.orchestra.run.smithi138:> sudo apt-get clean 2024-04-25T16:36:10.923 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get update 2024-04-25T16:36:10.932 DEBUG:teuthology.orchestra.run.smithi138:> sudo apt-get update 2024-04-25T16:36:11.078 INFO:teuthology.orchestra.run.smithi138.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T16:36:11.078 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T16:36:11.099 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T16:36:11.181 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T16:36:11.182 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T16:36:11.182 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T16:36:11.263 INFO:teuthology.orchestra.run.smithi138.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T16:36:11.308 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T16:36:11.321 INFO:teuthology.orchestra.run.smithi138.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T16:36:11.407 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T16:36:11.462 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T16:36:11.467 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T16:36:11.468 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T16:36:11.469 INFO:teuthology.orchestra.run.smithi094.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T16:36:11.470 INFO:teuthology.orchestra.run.smithi138.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T16:36:11.484 INFO:teuthology.orchestra.run.smithi138.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T16:36:11.491 INFO:teuthology.orchestra.run.smithi138.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T16:36:11.492 INFO:teuthology.orchestra.run.smithi138.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T16:36:11.505 INFO:teuthology.orchestra.run.smithi094.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T16:36:11.509 INFO:teuthology.orchestra.run.smithi094.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T16:36:11.510 INFO:teuthology.orchestra.run.smithi094.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T16:36:11.527 INFO:teuthology.orchestra.run.smithi094.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T16:36:11.549 INFO:teuthology.orchestra.run.smithi094.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T16:36:11.554 INFO:teuthology.orchestra.run.smithi094.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T16:36:11.556 INFO:teuthology.orchestra.run.smithi138.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T16:36:11.557 INFO:teuthology.orchestra.run.smithi138.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T16:36:11.558 INFO:teuthology.orchestra.run.smithi094.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T16:36:11.558 INFO:teuthology.orchestra.run.smithi094.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T16:36:11.559 INFO:teuthology.orchestra.run.smithi094.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T16:36:11.564 INFO:teuthology.orchestra.run.smithi138.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T16:36:11.564 INFO:teuthology.orchestra.run.smithi138.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T16:36:11.575 INFO:teuthology.orchestra.run.smithi138.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T16:36:11.587 INFO:teuthology.orchestra.run.smithi138.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T16:36:11.601 INFO:teuthology.orchestra.run.smithi094.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T16:36:11.601 INFO:teuthology.orchestra.run.smithi094.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-25T16:36:11.602 INFO:teuthology.orchestra.run.smithi094.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-25T16:36:11.604 INFO:teuthology.orchestra.run.smithi094.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T16:36:11.604 INFO:teuthology.orchestra.run.smithi094.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T16:36:11.604 INFO:teuthology.orchestra.run.smithi094.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-25T16:36:11.604 INFO:teuthology.orchestra.run.smithi094.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-25T16:36:11.609 INFO:teuthology.orchestra.run.smithi138.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T16:36:11.609 INFO:teuthology.orchestra.run.smithi138.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T16:36:11.610 INFO:teuthology.orchestra.run.smithi138.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T16:36:11.610 INFO:teuthology.orchestra.run.smithi138.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T16:36:11.610 INFO:teuthology.orchestra.run.smithi138.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T16:36:11.621 INFO:teuthology.orchestra.run.smithi094.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T16:36:11.627 INFO:teuthology.orchestra.run.smithi138.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T16:36:11.641 INFO:teuthology.orchestra.run.smithi094.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T16:36:11.642 INFO:teuthology.orchestra.run.smithi094.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T16:36:11.787 INFO:teuthology.orchestra.run.smithi138.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T16:36:12.033 INFO:teuthology.orchestra.run.smithi094.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T16:36:12.050 INFO:teuthology.orchestra.run.smithi094.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T16:36:12.059 INFO:teuthology.orchestra.run.smithi094.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T16:36:12.061 INFO:teuthology.orchestra.run.smithi094.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T16:36:12.063 INFO:teuthology.orchestra.run.smithi094.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T16:36:12.155 INFO:teuthology.orchestra.run.smithi094.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T16:36:12.166 INFO:teuthology.orchestra.run.smithi094.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T16:36:12.166 INFO:teuthology.orchestra.run.smithi094.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T16:36:12.198 INFO:teuthology.orchestra.run.smithi094.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T16:36:12.244 INFO:teuthology.orchestra.run.smithi138.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T16:36:12.312 INFO:teuthology.orchestra.run.smithi138.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T16:36:12.349 INFO:teuthology.orchestra.run.smithi138.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T16:36:12.350 INFO:teuthology.orchestra.run.smithi138.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T16:36:12.355 INFO:teuthology.orchestra.run.smithi138.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T16:36:12.379 INFO:teuthology.orchestra.run.smithi094.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T16:36:12.470 INFO:teuthology.orchestra.run.smithi138.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T16:36:12.489 INFO:teuthology.orchestra.run.smithi138.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T16:36:12.715 INFO:teuthology.orchestra.run.smithi138.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T16:36:12.788 INFO:teuthology.orchestra.run.smithi094.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T16:36:12.798 INFO:teuthology.orchestra.run.smithi094.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T16:36:12.814 INFO:teuthology.orchestra.run.smithi094.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T16:36:12.814 INFO:teuthology.orchestra.run.smithi094.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T16:36:12.817 INFO:teuthology.orchestra.run.smithi094.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T16:36:13.325 INFO:teuthology.orchestra.run.smithi138.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T16:36:13.498 INFO:teuthology.orchestra.run.smithi138.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T16:36:13.522 INFO:teuthology.orchestra.run.smithi138.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T16:36:13.524 INFO:teuthology.orchestra.run.smithi138.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T16:36:13.524 INFO:teuthology.orchestra.run.smithi138.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T16:36:13.532 INFO:teuthology.orchestra.run.smithi138.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T16:36:13.533 INFO:teuthology.orchestra.run.smithi138.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T16:36:13.533 INFO:teuthology.orchestra.run.smithi138.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-25T16:36:13.536 INFO:teuthology.orchestra.run.smithi138.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-25T16:36:13.541 INFO:teuthology.orchestra.run.smithi138.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T16:36:16.123 INFO:teuthology.orchestra.run.smithi138.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T16:36:16.206 INFO:teuthology.orchestra.run.smithi138.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-25T16:36:16.311 INFO:teuthology.orchestra.run.smithi138.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-25T16:36:16.400 INFO:teuthology.orchestra.run.smithi138.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T16:36:16.438 INFO:teuthology.orchestra.run.smithi138.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T16:36:17.322 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 13.2 MB in 2s (7,081 kB/s) 2024-04-25T16:36:18.699 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-04-25T16:36:18.716 DEBUG:teuthology.orchestra.run.smithi094:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T16:36:18.775 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-04-25T16:36:18.975 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2024-04-25T16:36:18.975 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2024-04-25T16:36:19.139 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T16:36:19.139 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T16:36:19.139 INFO:teuthology.orchestra.run.smithi094.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T16:36:19.139 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T16:36:19.139 INFO:teuthology.orchestra.run.smithi094.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T16:36:19.139 INFO:teuthology.orchestra.run.smithi094.stdout: python2.7-minimal 2024-04-25T16:36:19.139 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T16:36:19.140 INFO:teuthology.orchestra.run.smithi094.stdout:The following additional packages will be installed: 2024-04-25T16:36:19.141 INFO:teuthology.orchestra.run.smithi094.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T16:36:19.141 INFO:teuthology.orchestra.run.smithi094.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T16:36:19.141 INFO:teuthology.orchestra.run.smithi094.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T16:36:19.142 INFO:teuthology.orchestra.run.smithi094.stdout:Suggested packages: 2024-04-25T16:36:19.142 INFO:teuthology.orchestra.run.smithi094.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T16:36:19.142 INFO:teuthology.orchestra.run.smithi094.stdout:Recommended packages: 2024-04-25T16:36:19.142 INFO:teuthology.orchestra.run.smithi094.stdout: thermald 2024-04-25T16:36:19.183 INFO:teuthology.orchestra.run.smithi094.stdout:The following NEW packages will be installed: 2024-04-25T16:36:19.183 INFO:teuthology.orchestra.run.smithi094.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T16:36:19.183 INFO:teuthology.orchestra.run.smithi094.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T16:36:19.183 INFO:teuthology.orchestra.run.smithi094.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T16:36:19.184 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages will be upgraded: 2024-04-25T16:36:19.185 INFO:teuthology.orchestra.run.smithi094.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T16:36:19.241 INFO:teuthology.orchestra.run.smithi094.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T16:36:19.241 INFO:teuthology.orchestra.run.smithi094.stdout:Need to get 113 MB of archives. 2024-04-25T16:36:19.241 INFO:teuthology.orchestra.run.smithi094.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T16:36:19.242 INFO:teuthology.orchestra.run.smithi094.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T16:36:19.840 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T16:36:19.996 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T16:36:20.881 INFO:teuthology.orchestra.run.smithi138.stdout:Fetched 13.2 MB in 5s (2,432 kB/s) 2024-04-25T16:36:21.179 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T16:36:21.180 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T16:36:21.180 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T16:36:21.324 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T16:36:21.361 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T16:36:21.757 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 113 MB in 2s (52.2 MB/s) 2024-04-25T16:36:21.924 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T16:36:22.183 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T16:36:22.200 DEBUG:teuthology.orchestra.run.smithi138:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T16:36:22.258 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T16:36:22.467 INFO:teuthology.orchestra.run.smithi138.stdout:Building dependency tree... 2024-04-25T16:36:22.468 INFO:teuthology.orchestra.run.smithi138.stdout:Reading state information... 2024-04-25T16:36:22.648 INFO:teuthology.orchestra.run.smithi138.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T16:36:22.648 INFO:teuthology.orchestra.run.smithi138.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T16:36:22.648 INFO:teuthology.orchestra.run.smithi138.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T16:36:22.649 INFO:teuthology.orchestra.run.smithi138.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T16:36:22.649 INFO:teuthology.orchestra.run.smithi138.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T16:36:22.649 INFO:teuthology.orchestra.run.smithi138.stdout: python2.7-minimal 2024-04-25T16:36:22.649 INFO:teuthology.orchestra.run.smithi138.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T16:36:22.651 INFO:teuthology.orchestra.run.smithi138.stdout:The following additional packages will be installed: 2024-04-25T16:36:22.651 INFO:teuthology.orchestra.run.smithi138.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T16:36:22.651 INFO:teuthology.orchestra.run.smithi138.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T16:36:22.651 INFO:teuthology.orchestra.run.smithi138.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T16:36:22.652 INFO:teuthology.orchestra.run.smithi138.stdout:Suggested packages: 2024-04-25T16:36:22.653 INFO:teuthology.orchestra.run.smithi138.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T16:36:22.653 INFO:teuthology.orchestra.run.smithi138.stdout:Recommended packages: 2024-04-25T16:36:22.653 INFO:teuthology.orchestra.run.smithi138.stdout: thermald 2024-04-25T16:36:22.699 INFO:teuthology.orchestra.run.smithi138.stdout:The following NEW packages will be installed: 2024-04-25T16:36:22.699 INFO:teuthology.orchestra.run.smithi138.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T16:36:22.700 INFO:teuthology.orchestra.run.smithi138.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T16:36:22.700 INFO:teuthology.orchestra.run.smithi138.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T16:36:22.701 INFO:teuthology.orchestra.run.smithi138.stdout:The following packages will be upgraded: 2024-04-25T16:36:22.702 INFO:teuthology.orchestra.run.smithi138.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T16:36:22.759 INFO:teuthology.orchestra.run.smithi138.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T16:36:22.759 INFO:teuthology.orchestra.run.smithi138.stdout:Need to get 113 MB of archives. 2024-04-25T16:36:22.760 INFO:teuthology.orchestra.run.smithi138.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T16:36:22.760 INFO:teuthology.orchestra.run.smithi138.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T16:36:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T16:36:23.727 INFO:teuthology.orchestra.run.smithi138.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T16:36:24.002 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T16:36:24.006 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T16:36:24.154 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:24.984 INFO:teuthology.orchestra.run.smithi138.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T16:36:24.984 INFO:teuthology.orchestra.run.smithi138.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T16:36:24.984 INFO:teuthology.orchestra.run.smithi138.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T16:36:25.179 INFO:teuthology.orchestra.run.smithi138.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T16:36:25.216 INFO:teuthology.orchestra.run.smithi138.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T16:36:25.635 INFO:teuthology.orchestra.run.smithi138.stdout:Fetched 113 MB in 3s (45.2 MB/s) 2024-04-25T16:36:25.802 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T16:36:26.281 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T16:36:26.302 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T16:36:26.382 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:26.826 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T16:36:26.847 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T16:36:26.884 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:27.840 INFO:teuthology.orchestra.run.smithi138.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T16:36:27.844 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T16:36:27.908 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:30.042 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T16:36:30.064 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T16:36:30.157 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:30.612 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T16:36:30.633 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T16:36:30.662 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:33.386 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T16:36:33.514 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T16:36:33.847 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T16:36:33.943 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T16:36:34.247 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T16:36:34.269 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T16:36:34.313 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T16:36:37.378 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T16:36:37.506 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T16:36:37.796 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T16:36:37.892 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T16:36:38.212 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T16:36:38.234 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T16:36:38.278 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T16:36:40.335 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T16:36:40.373 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T16:36:40.418 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:42.989 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T16:36:43.111 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T16:36:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T16:36:43.525 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:43.635 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T16:36:43.763 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:44.227 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T16:36:44.264 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T16:36:44.301 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:45.100 INFO:teuthology.orchestra.run.smithi094.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T16:36:45.100 INFO:teuthology.orchestra.run.smithi094.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T16:36:45.100 INFO:teuthology.orchestra.run.smithi094.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T16:36:45.100 INFO:teuthology.orchestra.run.smithi094.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T16:36:45.234 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:47.171 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:47.641 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T16:36:47.766 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T16:36:48.134 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T16:36:48.263 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:48.364 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T16:36:48.468 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:48.588 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T16:36:48.715 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T16:36:48.841 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:48.903 INFO:teuthology.orchestra.run.smithi094.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T16:36:48.903 INFO:teuthology.orchestra.run.smithi094.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T16:36:49.778 INFO:teuthology.orchestra.run.smithi138.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T16:36:49.778 INFO:teuthology.orchestra.run.smithi138.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T16:36:49.778 INFO:teuthology.orchestra.run.smithi138.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T16:36:49.778 INFO:teuthology.orchestra.run.smithi138.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T16:36:49.946 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:53.383 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T16:36:53.508 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T16:36:53.635 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T16:36:53.701 INFO:teuthology.orchestra.run.smithi138.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T16:36:53.701 INFO:teuthology.orchestra.run.smithi138.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T16:37:03.750 INFO:teuthology.orchestra.run.smithi094.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T16:37:03.751 INFO:teuthology.orchestra.run.smithi094.stdout:Sourcing file `/etc/default/grub' 2024-04-25T16:37:03.762 INFO:teuthology.orchestra.run.smithi094.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T16:37:03.790 INFO:teuthology.orchestra.run.smithi094.stdout:Generating grub configuration file ... 2024-04-25T16:37:04.070 INFO:teuthology.orchestra.run.smithi094.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T16:37:04.095 INFO:teuthology.orchestra.run.smithi094.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T16:37:04.450 INFO:teuthology.orchestra.run.smithi094.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T16:37:04.456 INFO:teuthology.orchestra.run.smithi094.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T16:37:04.521 INFO:teuthology.orchestra.run.smithi094.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T16:37:04.527 INFO:teuthology.orchestra.run.smithi094.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T16:37:04.685 INFO:teuthology.orchestra.run.smithi094.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T16:37:04.685 INFO:teuthology.orchestra.run.smithi094.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T16:37:04.685 INFO:teuthology.orchestra.run.smithi094.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T16:37:04.729 INFO:teuthology.orchestra.run.smithi094.stdout:done 2024-04-25T16:37:05.194 DEBUG:teuthology.orchestra.run.smithi094:> dpkg -s linux-image-generic 2024-04-25T16:37:05.214 INFO:teuthology.orchestra.run.smithi094.stdout:Package: linux-image-generic 2024-04-25T16:37:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Status: install ok installed 2024-04-25T16:37:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Priority: optional 2024-04-25T16:37:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Section: kernel 2024-04-25T16:37:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Installed-Size: 21 2024-04-25T16:37:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T16:37:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: amd64 2024-04-25T16:37:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Source: linux-meta 2024-04-25T16:37:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Version: 5.15.0.105.102 2024-04-25T16:37:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T16:37:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T16:37:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Recommends: thermald 2024-04-25T16:37:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Description: Generic Linux kernel image 2024-04-25T16:37:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: This package will always depend on the latest generic kernel image 2024-04-25T16:37:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: available. 2024-04-25T16:37:05.216 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T16:37:05.216 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T16:37:05.216 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T16:37:05.216 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T16:37:05.216 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi094.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T16:37:05.216 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get clean 2024-04-25T16:37:05.377 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get update 2024-04-25T16:37:05.499 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T16:37:05.511 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T16:37:05.540 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T16:37:05.680 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T16:37:06.792 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-04-25T16:37:06.809 DEBUG:teuthology.orchestra.run.smithi094:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T16:37:06.868 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-04-25T16:37:07.066 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2024-04-25T16:37:07.066 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2024-04-25T16:37:07.244 INFO:teuthology.orchestra.run.smithi094.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T16:37:07.245 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T16:37:07.245 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T16:37:07.245 INFO:teuthology.orchestra.run.smithi094.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T16:37:07.245 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T16:37:07.245 INFO:teuthology.orchestra.run.smithi094.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T16:37:07.245 INFO:teuthology.orchestra.run.smithi094.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T16:37:07.245 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T16:37:07.287 INFO:teuthology.orchestra.run.smithi094.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T16:37:07.289 DEBUG:teuthology.orchestra.run.smithi094:> dpkg -s linux-image-generic 2024-04-25T16:37:07.344 INFO:teuthology.orchestra.run.smithi094.stdout:Package: linux-image-generic 2024-04-25T16:37:07.344 INFO:teuthology.orchestra.run.smithi094.stdout:Status: install ok installed 2024-04-25T16:37:07.345 INFO:teuthology.orchestra.run.smithi094.stdout:Priority: optional 2024-04-25T16:37:07.345 INFO:teuthology.orchestra.run.smithi094.stdout:Section: kernel 2024-04-25T16:37:07.345 INFO:teuthology.orchestra.run.smithi094.stdout:Installed-Size: 21 2024-04-25T16:37:07.345 INFO:teuthology.orchestra.run.smithi094.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T16:37:07.345 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: amd64 2024-04-25T16:37:07.345 INFO:teuthology.orchestra.run.smithi094.stdout:Source: linux-meta 2024-04-25T16:37:07.345 INFO:teuthology.orchestra.run.smithi094.stdout:Version: 5.15.0.105.102 2024-04-25T16:37:07.345 INFO:teuthology.orchestra.run.smithi094.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T16:37:07.345 INFO:teuthology.orchestra.run.smithi094.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T16:37:07.346 INFO:teuthology.orchestra.run.smithi094.stdout:Recommends: thermald 2024-04-25T16:37:07.346 INFO:teuthology.orchestra.run.smithi094.stdout:Description: Generic Linux kernel image 2024-04-25T16:37:07.346 INFO:teuthology.orchestra.run.smithi094.stdout: This package will always depend on the latest generic kernel image 2024-04-25T16:37:07.346 INFO:teuthology.orchestra.run.smithi094.stdout: available. 2024-04-25T16:37:07.346 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-04-25T16:37:07.389 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.WGKSwCC5Dt 2024-04-25T16:37:07.389 DEBUG:teuthology.orchestra.run.smithi094:> sudo cp /boot/grub/grub.cfg /tmp/tmp.WGKSwCC5Dt 2024-04-25T16:37:07.444 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.WGKSwCC5Dt 2024-04-25T16:37:07.555 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.WGKSwCC5Dt is 10KB 2024-04-25T16:37:07.567 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.WGKSwCC5Dt 2024-04-25T16:37:07.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T16:37:07.624 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:37:07.625 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T16:37:07.625 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T16:37:07.698 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T16:37:07.698 DEBUG:teuthology.orchestra.run.smithi094:> sudo update-grub 2024-04-25T16:37:07.775 INFO:teuthology.orchestra.run.smithi138.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T16:37:07.775 INFO:teuthology.orchestra.run.smithi138.stdout:Sourcing file `/etc/default/grub' 2024-04-25T16:37:07.803 INFO:teuthology.orchestra.run.smithi138.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T16:37:07.813 INFO:teuthology.orchestra.run.smithi138.stdout:Generating grub configuration file ... 2024-04-25T16:37:08.168 INFO:teuthology.orchestra.run.smithi138.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T16:37:08.203 INFO:teuthology.orchestra.run.smithi138.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T16:37:08.560 INFO:teuthology.orchestra.run.smithi138.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T16:37:08.566 INFO:teuthology.orchestra.run.smithi138.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T16:37:08.629 INFO:teuthology.orchestra.run.smithi138.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T16:37:08.634 INFO:teuthology.orchestra.run.smithi138.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T16:37:08.793 INFO:teuthology.orchestra.run.smithi138.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T16:37:08.794 INFO:teuthology.orchestra.run.smithi138.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T16:37:08.794 INFO:teuthology.orchestra.run.smithi138.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T16:37:08.842 INFO:teuthology.orchestra.run.smithi138.stdout:done 2024-04-25T16:37:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub' 2024-04-25T16:37:08.871 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T16:37:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-04-25T16:37:09.079 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T16:37:09.085 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T16:37:09.292 DEBUG:teuthology.orchestra.run.smithi138:> dpkg -s linux-image-generic 2024-04-25T16:37:09.312 INFO:teuthology.orchestra.run.smithi138.stdout:Package: linux-image-generic 2024-04-25T16:37:09.312 INFO:teuthology.orchestra.run.smithi138.stdout:Status: install ok installed 2024-04-25T16:37:09.312 INFO:teuthology.orchestra.run.smithi138.stdout:Priority: optional 2024-04-25T16:37:09.312 INFO:teuthology.orchestra.run.smithi138.stdout:Section: kernel 2024-04-25T16:37:09.312 INFO:teuthology.orchestra.run.smithi138.stdout:Installed-Size: 21 2024-04-25T16:37:09.313 INFO:teuthology.orchestra.run.smithi138.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T16:37:09.313 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: amd64 2024-04-25T16:37:09.313 INFO:teuthology.orchestra.run.smithi138.stdout:Source: linux-meta 2024-04-25T16:37:09.313 INFO:teuthology.orchestra.run.smithi138.stdout:Version: 5.15.0.105.102 2024-04-25T16:37:09.313 INFO:teuthology.orchestra.run.smithi138.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T16:37:09.313 INFO:teuthology.orchestra.run.smithi138.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T16:37:09.313 INFO:teuthology.orchestra.run.smithi138.stdout:Recommends: thermald 2024-04-25T16:37:09.313 INFO:teuthology.orchestra.run.smithi138.stdout:Description: Generic Linux kernel image 2024-04-25T16:37:09.313 INFO:teuthology.orchestra.run.smithi138.stdout: This package will always depend on the latest generic kernel image 2024-04-25T16:37:09.313 INFO:teuthology.orchestra.run.smithi138.stdout: available. 2024-04-25T16:37:09.313 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T16:37:09.314 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T16:37:09.314 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T16:37:09.314 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-25T16:37:09.314 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T16:37:09.314 DEBUG:teuthology.orchestra.run.smithi138:> sudo apt-get clean 2024-04-25T16:37:09.329 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T16:37:09.335 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T16:37:09.380 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T16:37:09.385 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T16:37:09.489 DEBUG:teuthology.orchestra.run.smithi138:> sudo apt-get update 2024-04-25T16:37:09.539 INFO:teuthology.orchestra.run.smithi094.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T16:37:09.540 INFO:teuthology.orchestra.run.smithi094.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T16:37:09.540 INFO:teuthology.orchestra.run.smithi094.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T16:37:09.555 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-04-25T16:37:09.571 DEBUG:teuthology.orchestra.run.smithi094:> sudo shutdown -r now 2024-04-25T16:37:09.735 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T16:37:09.735 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T16:37:09.811 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T16:37:09.896 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T16:37:11.088 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T16:37:11.107 DEBUG:teuthology.orchestra.run.smithi138:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T16:37:11.167 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T16:37:11.368 INFO:teuthology.orchestra.run.smithi138.stdout:Building dependency tree... 2024-04-25T16:37:11.368 INFO:teuthology.orchestra.run.smithi138.stdout:Reading state information... 2024-04-25T16:37:11.542 INFO:teuthology.orchestra.run.smithi138.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T16:37:11.542 INFO:teuthology.orchestra.run.smithi138.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T16:37:11.542 INFO:teuthology.orchestra.run.smithi138.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T16:37:11.543 INFO:teuthology.orchestra.run.smithi138.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T16:37:11.543 INFO:teuthology.orchestra.run.smithi138.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T16:37:11.543 INFO:teuthology.orchestra.run.smithi138.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T16:37:11.543 INFO:teuthology.orchestra.run.smithi138.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T16:37:11.543 INFO:teuthology.orchestra.run.smithi138.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T16:37:11.584 INFO:teuthology.orchestra.run.smithi138.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T16:37:11.586 DEBUG:teuthology.orchestra.run.smithi138:> dpkg -s linux-image-generic 2024-04-25T16:37:11.601 INFO:teuthology.orchestra.run.smithi138.stdout:Package: linux-image-generic 2024-04-25T16:37:11.601 INFO:teuthology.orchestra.run.smithi138.stdout:Status: install ok installed 2024-04-25T16:37:11.601 INFO:teuthology.orchestra.run.smithi138.stdout:Priority: optional 2024-04-25T16:37:11.601 INFO:teuthology.orchestra.run.smithi138.stdout:Section: kernel 2024-04-25T16:37:11.601 INFO:teuthology.orchestra.run.smithi138.stdout:Installed-Size: 21 2024-04-25T16:37:11.601 INFO:teuthology.orchestra.run.smithi138.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T16:37:11.601 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: amd64 2024-04-25T16:37:11.601 INFO:teuthology.orchestra.run.smithi138.stdout:Source: linux-meta 2024-04-25T16:37:11.602 INFO:teuthology.orchestra.run.smithi138.stdout:Version: 5.15.0.105.102 2024-04-25T16:37:11.602 INFO:teuthology.orchestra.run.smithi138.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T16:37:11.602 INFO:teuthology.orchestra.run.smithi138.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T16:37:11.602 INFO:teuthology.orchestra.run.smithi138.stdout:Recommends: thermald 2024-04-25T16:37:11.602 INFO:teuthology.orchestra.run.smithi138.stdout:Description: Generic Linux kernel image 2024-04-25T16:37:11.602 INFO:teuthology.orchestra.run.smithi138.stdout: This package will always depend on the latest generic kernel image 2024-04-25T16:37:11.602 INFO:teuthology.orchestra.run.smithi138.stdout: available. 2024-04-25T16:37:11.603 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-04-25T16:37:11.648 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.CaLDGkNbhz 2024-04-25T16:37:11.648 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub/grub.cfg /tmp/tmp.CaLDGkNbhz 2024-04-25T16:37:11.703 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.CaLDGkNbhz 2024-04-25T16:37:11.824 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.CaLDGkNbhz is 10KB 2024-04-25T16:37:11.837 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.CaLDGkNbhz 2024-04-25T16:37:11.843 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T16:37:11.896 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:37:11.896 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T16:37:11.896 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T16:37:11.964 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T16:37:11.965 DEBUG:teuthology.orchestra.run.smithi138:> sudo update-grub 2024-04-25T16:37:13.178 INFO:teuthology.orchestra.run.smithi138.stderr:Sourcing file `/etc/default/grub' 2024-04-25T16:37:13.178 INFO:teuthology.orchestra.run.smithi138.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T16:37:13.180 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-04-25T16:37:13.424 INFO:teuthology.orchestra.run.smithi138.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T16:37:13.430 INFO:teuthology.orchestra.run.smithi138.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T16:37:13.729 INFO:teuthology.orchestra.run.smithi138.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T16:37:13.734 INFO:teuthology.orchestra.run.smithi138.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T16:37:13.774 INFO:teuthology.orchestra.run.smithi138.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T16:37:13.779 INFO:teuthology.orchestra.run.smithi138.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T16:37:13.934 INFO:teuthology.orchestra.run.smithi138.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T16:37:13.934 INFO:teuthology.orchestra.run.smithi138.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T16:37:13.934 INFO:teuthology.orchestra.run.smithi138.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T16:37:13.947 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-04-25T16:37:13.956 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-04-25T16:37:39.581 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T16:37:39.582 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T16:37:39.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:37:43.961 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-25T16:37:43.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T16:37:43.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:37:57.919 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-04-25T16:38:02.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-04-25T16:38:06.922 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T16:38:06.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:38:11.532 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T16:38:11.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:38:14.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-04-25T16:38:26.602 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T16:38:26.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:38:45.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-04-25T16:39:00.038 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T16:39:00.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:39:00.434 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-25T16:39:01.000 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T16:39:01.000 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-25T16:39:01.000 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-04-25T16:39:01.046 INFO:teuthology.orchestra.run.smithi138.stdout:5.15.0-105-generic 2024-04-25T16:39:01.047 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T16:39:01.047 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T16:39:01.047 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T16:39:02.048 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T16:39:02.050 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T16:39:02.112 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-04-25T16:39:02.152 DEBUG:teuthology.parallel:result is None 2024-04-25T16:39:06.925 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T16:39:18.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T16:39:18.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:39:19.147 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-04-25T16:39:19.765 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T16:39:19.765 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-25T16:39:19.765 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-04-25T16:39:19.813 INFO:teuthology.orchestra.run.smithi094.stdout:5.15.0-105-generic 2024-04-25T16:39:19.813 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T16:39:19.814 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T16:39:19.814 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T16:39:20.814 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T16:39:20.815 DEBUG:teuthology.orchestra.run.smithi094:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T16:39:20.895 INFO:teuthology.orchestra.run.smithi094.stdout:ttyS1 2024-04-25T16:39:20.914 DEBUG:teuthology.parallel:result is None 2024-04-25T16:39:20.915 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T16:39:20.922 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T16:39:20.922 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T16:39:20.925 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T16:39:20.931 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T16:39:20.959 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T16:39:20.993 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T16:39:20.993 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T16:39:20.996 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T16:39:21.272 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T16:39:21.279 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T16:39:21.279 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T16:39:21.282 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T16:39:21.296 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:39:21.301 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:39:21.305 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:39:21.308 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:39:21.309 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T16:39:21.340 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T16:39:21.341 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T16:39:21.350 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T16:39:21.367 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T16:39:21.375 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T16:39:21.376 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T16:39:21.406 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T16:39:21.418 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:39:21.457 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:39:21.505 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:39:21.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T16:39:21.566 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:39:21.572 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:39:21.619 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:39:21.619 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T16:39:21.689 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-04-25T16:39:21.691 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-04-25T16:39:21.761 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T16:39:21.769 INFO:teuthology.task.internal:Starting timer... 2024-04-25T16:39:21.769 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T16:39:21.818 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T16:39:21.851 DEBUG:teuthology.task.selinux:Excluding smithi094: OS 'ubuntu' does not support SELinux 2024-04-25T16:39:21.851 DEBUG:teuthology.task.selinux:Excluding smithi138: OS 'ubuntu' does not support SELinux 2024-04-25T16:39:21.851 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T16:39:21.852 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T16:39:21.852 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T16:39:21.852 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T16:39:21.884 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T16:39:21.890 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T16:39:22.016 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T16:39:22.033 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T16:39:22.034 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T16:46:55.322 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-04-25T16:46:55.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T16:46:55.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:46:55.407 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-04-25T16:46:55.482 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T16:46:55.483 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T16:46:55.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:46:55.562 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-25T16:46:55.638 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T16:46:55.638 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T16:46:55.654 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T16:46:55.654 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T16:46:55.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:46:55.657 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T16:46:55.657 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:46:55.685 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T16:46:55.685 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: Command line: ntpd -gq 2024-04-25T16:46:55.685 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: ---------------------------------------------------- 2024-04-25T16:46:55.685 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T16:46:55.685 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T16:46:55.685 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: corporation. Support and training for ntp-4 are 2024-04-25T16:46:55.685 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: available at https://www.nwtime.org/support 2024-04-25T16:46:55.685 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: ---------------------------------------------------- 2024-04-25T16:46:55.685 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: proto: precision = 0.043 usec (-24) 2024-04-25T16:46:55.685 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: basedate set to 2022-02-04 2024-04-25T16:46:55.685 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: gps base set to 2022-02-06 (week 2196) 2024-04-25T16:46:55.686 INFO:teuthology.orchestra.run.smithi094.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T16:46:55.686 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T16:46:55.686 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T16:46:55.687 INFO:teuthology.orchestra.run.smithi094.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T16:46:55.687 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T16:46:55.687 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T16:46:55.688 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T16:46:55.688 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: Listen normally on 3 enp3s0f1 172.21.15.94:123 2024-04-25T16:46:55.688 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: Listen normally on 4 lo [::1]:123 2024-04-25T16:46:55.688 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d763%5]:123 2024-04-25T16:46:55.688 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:46:55 ntpd[17199]: Listening on routing socket on fd #22 for interface updates 2024-04-25T16:46:55.707 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T16:46:55.707 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: Command line: ntpd -gq 2024-04-25T16:46:55.707 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: ---------------------------------------------------- 2024-04-25T16:46:55.707 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T16:46:55.707 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T16:46:55.707 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: corporation. Support and training for ntp-4 are 2024-04-25T16:46:55.707 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: available at https://www.nwtime.org/support 2024-04-25T16:46:55.707 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: ---------------------------------------------------- 2024-04-25T16:46:55.708 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: proto: precision = 0.046 usec (-24) 2024-04-25T16:46:55.708 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: basedate set to 2022-02-04 2024-04-25T16:46:55.708 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: gps base set to 2022-02-06 (week 2196) 2024-04-25T16:46:55.708 INFO:teuthology.orchestra.run.smithi138.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T16:46:55.709 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T16:46:55.709 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T16:46:55.709 INFO:teuthology.orchestra.run.smithi138.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T16:46:55.710 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T16:46:55.710 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T16:46:55.710 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T16:46:55.710 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: Listen normally on 3 enp3s0f1 172.21.15.138:123 2024-04-25T16:46:55.710 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: Listen normally on 4 lo [::1]:123 2024-04-25T16:46:55.710 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6f71%5]:123 2024-04-25T16:46:55.711 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:46:55 ntpd[17199]: Listening on routing socket on fd #22 for interface updates 2024-04-25T16:46:56.685 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:56 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:56.685 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:56 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:56.685 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:56 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:56.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:56 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:56.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:56 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:56.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:56 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:56.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:56 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:46:56.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:56 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:46:56.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:56 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:56.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:56 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:56.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:56 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:46:56.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:56 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:46:56.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:56 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:56.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:56 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:56.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:56 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:56.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:56 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:56.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:56 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:56.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:56 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:56.710 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:56 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:46:56.710 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:56 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:46:58.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:58.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:58.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:58.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:58.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:58.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:58.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:58.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:46:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:46:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:46:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:46:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:46:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:46:58.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:58.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:58.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:58.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:58.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:58.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:58.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:46:58.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:46:58.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:58.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:58.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:58 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:46:58.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:58 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:46:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:59 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:46:59 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:59.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:59 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:59.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:59 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:59.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:59 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:46:59.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:59 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:46:59.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:59 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:46:59.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:46:59 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:47:00.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:00.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:00.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:00.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:00.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:00.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:00.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:00.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:00.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:47:00.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:47:00.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:00.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:00.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:47:00.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:47:00.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:00.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:00.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:47:00.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:47:00.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:00.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:00.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:00.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:00.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:00.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:00.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:47:00.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:47:00.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:00.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:00.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:00 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:47:00.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:00 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:47:01.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:01 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:01.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:01 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:01.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:01 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:01.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:01 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:01.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:01 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:47:01.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:01 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:47:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:02 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:02 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:02 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:02 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:02 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:02 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:02 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:02 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:02 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:47:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:02 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:47:02.686 INFO:teuthology.orchestra.run.smithi094.stdout:25 Apr 16:47:02 ntpd[17199]: ntpd: time slew -0.000717 s 2024-04-25T16:47:02.686 INFO:teuthology.orchestra.run.smithi094.stdout:ntpd: time slew -0.000717s 2024-04-25T16:47:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:02 ntpd[17199]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T16:47:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:25 Apr 16:47:02 ntpd[17199]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T16:47:02.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:02 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:02.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:02 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:02.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:02 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:02.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:02 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:02.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:02 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T16:47:02.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:02 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T16:47:02.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:02 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T16:47:02.708 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:02 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T16:47:02.709 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 16:47:02 ntpd[17199]: ntpd: time slew +0.000642 s 2024-04-25T16:47:02.709 INFO:teuthology.orchestra.run.smithi138.stdout:ntpd: time slew +0.000642s 2024-04-25T16:47:02.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:02 ntpd[17199]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T16:47:02.709 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 16:47:02 ntpd[17199]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T16:47:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T16:47:02.737 INFO:teuthology.orchestra.run.smithi094.stdout:============================================================================== 2024-04-25T16:47:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T16:47:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T16:47:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T16:47:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T16:47:02.753 INFO:teuthology.orchestra.run.smithi138.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T16:47:02.753 INFO:teuthology.orchestra.run.smithi138.stdout:============================================================================== 2024-04-25T16:47:02.753 INFO:teuthology.orchestra.run.smithi138.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T16:47:02.753 INFO:teuthology.orchestra.run.smithi138.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T16:47:02.753 INFO:teuthology.orchestra.run.smithi138.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T16:47:02.753 INFO:teuthology.orchestra.run.smithi138.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T16:47:02.754 INFO:teuthology.run_tasks:Running task install... 2024-04-25T16:47:02.763 DEBUG:teuthology.task.install:project ceph 2024-04-25T16:47:02.763 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T16:47:02.763 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T16:47:02.763 INFO:teuthology.task.install:Using flavor: default 2024-04-25T16:47:02.770 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T16:47:02.771 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T16:47:02.771 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-key list | grep Ceph 2024-04-25T16:47:02.772 DEBUG:teuthology.orchestra.run.smithi138:> sudo apt-key list | grep Ceph 2024-04-25T16:47:02.833 INFO:teuthology.orchestra.run.smithi094.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T16:47:02.853 INFO:teuthology.orchestra.run.smithi138.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T16:47:02.860 INFO:teuthology.orchestra.run.smithi094.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T16:47:02.861 INFO:teuthology.orchestra.run.smithi094.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T16:47:02.861 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T16:47:02.861 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T16:47:02.862 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:47:02.881 INFO:teuthology.orchestra.run.smithi138.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T16:47:02.881 INFO:teuthology.orchestra.run.smithi138.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T16:47:02.882 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T16:47:02.882 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T16:47:02.882 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:47:03.105 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default/ 2024-04-25T16:47:03.105 INFO:teuthology.task.install.deb:Package version is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T16:47:03.108 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default/ 2024-04-25T16:47:03.109 INFO:teuthology.task.install.deb:Package version is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T16:47:03.218 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:47:03.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T16:47:03.228 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:47:03.228 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T16:47:03.231 DEBUG:teuthology.orchestra.run.smithi138:> sudo apt-get update 2024-04-25T16:47:03.242 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get update 2024-04-25T16:47:03.465 INFO:teuthology.orchestra.run.smithi094.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T16:47:03.488 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T16:47:03.488 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T16:47:03.493 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T16:47:03.493 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T16:47:03.506 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T16:47:03.510 INFO:teuthology.orchestra.run.smithi094.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T16:47:03.521 INFO:teuthology.orchestra.run.smithi138.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T16:47:03.527 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T16:47:03.532 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-25T16:47:03.559 INFO:teuthology.orchestra.run.smithi138.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T16:47:03.559 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-04-25T16:47:03.596 INFO:teuthology.orchestra.run.smithi138.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T16:47:03.600 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:7 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T16:47:03.634 INFO:teuthology.orchestra.run.smithi138.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-04-25T16:47:03.681 INFO:teuthology.orchestra.run.smithi138.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-25T16:47:03.682 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T16:47:03.764 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T16:47:03.865 INFO:teuthology.orchestra.run.smithi138.stdout:Fetched 29.2 kB in 0s (63.8 kB/s) 2024-04-25T16:47:03.906 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 29.2 kB in 0s (60.5 kB/s) 2024-04-25T16:47:04.824 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T16:47:04.840 DEBUG:teuthology.orchestra.run.smithi138:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3291-g994e2f20-1jammy cephadm=19.0.0-3291-g994e2f20-1jammy ceph-mds=19.0.0-3291-g994e2f20-1jammy ceph-mgr=19.0.0-3291-g994e2f20-1jammy ceph-common=19.0.0-3291-g994e2f20-1jammy ceph-fuse=19.0.0-3291-g994e2f20-1jammy ceph-test=19.0.0-3291-g994e2f20-1jammy ceph-volume=19.0.0-3291-g994e2f20-1jammy radosgw=19.0.0-3291-g994e2f20-1jammy python3-rados=19.0.0-3291-g994e2f20-1jammy python3-rgw=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy python3-rbd=19.0.0-3291-g994e2f20-1jammy libcephfs2=19.0.0-3291-g994e2f20-1jammy libcephfs-dev=19.0.0-3291-g994e2f20-1jammy librados2=19.0.0-3291-g994e2f20-1jammy librbd1=19.0.0-3291-g994e2f20-1jammy rbd-fuse=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy cephfs-shell=19.0.0-3291-g994e2f20-1jammy cephfs-top=19.0.0-3291-g994e2f20-1jammy cephfs-mirror=19.0.0-3291-g994e2f20-1jammy 2024-04-25T16:47:04.887 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-04-25T16:47:04.890 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T16:47:04.902 DEBUG:teuthology.orchestra.run.smithi094:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3291-g994e2f20-1jammy cephadm=19.0.0-3291-g994e2f20-1jammy ceph-mds=19.0.0-3291-g994e2f20-1jammy ceph-mgr=19.0.0-3291-g994e2f20-1jammy ceph-common=19.0.0-3291-g994e2f20-1jammy ceph-fuse=19.0.0-3291-g994e2f20-1jammy ceph-test=19.0.0-3291-g994e2f20-1jammy ceph-volume=19.0.0-3291-g994e2f20-1jammy radosgw=19.0.0-3291-g994e2f20-1jammy python3-rados=19.0.0-3291-g994e2f20-1jammy python3-rgw=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy python3-rbd=19.0.0-3291-g994e2f20-1jammy libcephfs2=19.0.0-3291-g994e2f20-1jammy libcephfs-dev=19.0.0-3291-g994e2f20-1jammy librados2=19.0.0-3291-g994e2f20-1jammy librbd1=19.0.0-3291-g994e2f20-1jammy rbd-fuse=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy cephfs-shell=19.0.0-3291-g994e2f20-1jammy cephfs-top=19.0.0-3291-g994e2f20-1jammy cephfs-mirror=19.0.0-3291-g994e2f20-1jammy 2024-04-25T16:47:04.956 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-04-25T16:47:05.075 INFO:teuthology.orchestra.run.smithi138.stdout:Building dependency tree... 2024-04-25T16:47:05.075 INFO:teuthology.orchestra.run.smithi138.stdout:Reading state information... 2024-04-25T16:47:05.131 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2024-04-25T16:47:05.132 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2024-04-25T16:47:05.251 INFO:teuthology.orchestra.run.smithi138.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T16:47:05.251 INFO:teuthology.orchestra.run.smithi138.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T16:47:05.251 INFO:teuthology.orchestra.run.smithi138.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T16:47:05.251 INFO:teuthology.orchestra.run.smithi138.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T16:47:05.251 INFO:teuthology.orchestra.run.smithi138.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T16:47:05.251 INFO:teuthology.orchestra.run.smithi138.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T16:47:05.251 INFO:teuthology.orchestra.run.smithi138.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T16:47:05.253 INFO:teuthology.orchestra.run.smithi138.stdout:The following additional packages will be installed: 2024-04-25T16:47:05.253 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T16:47:05.253 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T16:47:05.253 INFO:teuthology.orchestra.run.smithi138.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T16:47:05.253 INFO:teuthology.orchestra.run.smithi138.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T16:47:05.253 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T16:47:05.253 INFO:teuthology.orchestra.run.smithi138.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T16:47:05.253 INFO:teuthology.orchestra.run.smithi138.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T16:47:05.253 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T16:47:05.253 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T16:47:05.253 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T16:47:05.254 INFO:teuthology.orchestra.run.smithi138.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T16:47:05.255 INFO:teuthology.orchestra.run.smithi138.stdout:Suggested packages: 2024-04-25T16:47:05.255 INFO:teuthology.orchestra.run.smithi138.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T16:47:05.255 INFO:teuthology.orchestra.run.smithi138.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T16:47:05.255 INFO:teuthology.orchestra.run.smithi138.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T16:47:05.255 INFO:teuthology.orchestra.run.smithi138.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T16:47:05.255 INFO:teuthology.orchestra.run.smithi138.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T16:47:05.255 INFO:teuthology.orchestra.run.smithi138.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T16:47:05.255 INFO:teuthology.orchestra.run.smithi138.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T16:47:05.256 INFO:teuthology.orchestra.run.smithi138.stdout:Recommended packages: 2024-04-25T16:47:05.256 INFO:teuthology.orchestra.run.smithi138.stdout: btrfs-tools 2024-04-25T16:47:05.300 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T16:47:05.300 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T16:47:05.300 INFO:teuthology.orchestra.run.smithi094.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T16:47:05.300 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T16:47:05.300 INFO:teuthology.orchestra.run.smithi094.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T16:47:05.301 INFO:teuthology.orchestra.run.smithi094.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T16:47:05.301 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T16:47:05.301 INFO:teuthology.orchestra.run.smithi094.stdout:The following additional packages will be installed: 2024-04-25T16:47:05.301 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T16:47:05.303 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T16:47:05.303 INFO:teuthology.orchestra.run.smithi094.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T16:47:05.303 INFO:teuthology.orchestra.run.smithi094.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T16:47:05.303 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T16:47:05.303 INFO:teuthology.orchestra.run.smithi094.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T16:47:05.303 INFO:teuthology.orchestra.run.smithi094.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T16:47:05.303 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T16:47:05.303 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T16:47:05.303 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T16:47:05.303 INFO:teuthology.orchestra.run.smithi094.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T16:47:05.303 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T16:47:05.304 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T16:47:05.304 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T16:47:05.304 INFO:teuthology.orchestra.run.smithi094.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T16:47:05.304 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T16:47:05.304 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T16:47:05.304 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T16:47:05.304 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T16:47:05.304 INFO:teuthology.orchestra.run.smithi094.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T16:47:05.304 INFO:teuthology.orchestra.run.smithi094.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T16:47:05.304 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T16:47:05.304 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T16:47:05.304 INFO:teuthology.orchestra.run.smithi094.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T16:47:05.305 INFO:teuthology.orchestra.run.smithi094.stdout:Suggested packages: 2024-04-25T16:47:05.305 INFO:teuthology.orchestra.run.smithi094.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T16:47:05.305 INFO:teuthology.orchestra.run.smithi094.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T16:47:05.305 INFO:teuthology.orchestra.run.smithi094.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T16:47:05.305 INFO:teuthology.orchestra.run.smithi094.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T16:47:05.305 INFO:teuthology.orchestra.run.smithi094.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T16:47:05.305 INFO:teuthology.orchestra.run.smithi094.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T16:47:05.305 INFO:teuthology.orchestra.run.smithi094.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T16:47:05.305 INFO:teuthology.orchestra.run.smithi094.stdout:Recommended packages: 2024-04-25T16:47:05.305 INFO:teuthology.orchestra.run.smithi094.stdout: btrfs-tools 2024-04-25T16:47:05.359 INFO:teuthology.orchestra.run.smithi138.stdout:The following NEW packages will be installed: 2024-04-25T16:47:05.359 INFO:teuthology.orchestra.run.smithi138.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T16:47:05.359 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T16:47:05.359 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T16:47:05.359 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T16:47:05.359 INFO:teuthology.orchestra.run.smithi138.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T16:47:05.359 INFO:teuthology.orchestra.run.smithi138.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T16:47:05.359 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T16:47:05.360 INFO:teuthology.orchestra.run.smithi138.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T16:47:05.360 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T16:47:05.361 INFO:teuthology.orchestra.run.smithi138.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T16:47:05.362 INFO:teuthology.orchestra.run.smithi138.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T16:47:05.362 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T16:47:05.362 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T16:47:05.362 INFO:teuthology.orchestra.run.smithi138.stdout: socat unzip xmlstarlet zip 2024-04-25T16:47:05.408 INFO:teuthology.orchestra.run.smithi094.stdout:The following NEW packages will be installed: 2024-04-25T16:47:05.408 INFO:teuthology.orchestra.run.smithi094.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T16:47:05.408 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T16:47:05.408 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T16:47:05.408 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T16:47:05.409 INFO:teuthology.orchestra.run.smithi094.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T16:47:05.409 INFO:teuthology.orchestra.run.smithi094.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T16:47:05.410 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T16:47:05.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T16:47:05.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T16:47:05.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T16:47:05.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T16:47:05.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T16:47:05.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T16:47:05.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T16:47:05.411 INFO:teuthology.orchestra.run.smithi094.stdout: socat unzip xmlstarlet zip 2024-04-25T16:47:05.547 INFO:teuthology.orchestra.run.smithi138.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T16:47:05.547 INFO:teuthology.orchestra.run.smithi138.stdout:Need to get 175 MB of archives. 2024-04-25T16:47:05.547 INFO:teuthology.orchestra.run.smithi138.stdout:After this operation, 887 MB of additional disk space will be used. 2024-04-25T16:47:05.547 INFO:teuthology.orchestra.run.smithi138.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T16:47:05.574 INFO:teuthology.orchestra.run.smithi138.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3291-g994e2f20-1jammy [3,525 kB] 2024-04-25T16:47:05.600 INFO:teuthology.orchestra.run.smithi094.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T16:47:05.601 INFO:teuthology.orchestra.run.smithi094.stdout:Need to get 175 MB of archives. 2024-04-25T16:47:05.601 INFO:teuthology.orchestra.run.smithi094.stdout:After this operation, 887 MB of additional disk space will be used. 2024-04-25T16:47:05.601 INFO:teuthology.orchestra.run.smithi094.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T16:47:05.625 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3291-g994e2f20-1jammy [3,525 kB] 2024-04-25T16:47:05.631 INFO:teuthology.orchestra.run.smithi138.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T16:47:05.679 INFO:teuthology.orchestra.run.smithi138.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T16:47:05.688 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T16:47:05.735 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T16:47:05.890 INFO:teuthology.orchestra.run.smithi138.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T16:47:05.935 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T16:47:06.043 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T16:47:06.061 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T16:47:06.126 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T16:47:06.166 INFO:teuthology.orchestra.run.smithi138.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T16:47:06.249 INFO:teuthology.orchestra.run.smithi138.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T16:47:06.375 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T16:47:06.392 INFO:teuthology.orchestra.run.smithi094.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T16:47:06.424 INFO:teuthology.orchestra.run.smithi094.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T16:47:06.433 INFO:teuthology.orchestra.run.smithi094.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T16:47:06.471 INFO:teuthology.orchestra.run.smithi138.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3291-g994e2f20-1jammy [3,287 kB] 2024-04-25T16:47:06.525 INFO:teuthology.orchestra.run.smithi094.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3291-g994e2f20-1jammy [3,287 kB] 2024-04-25T16:47:06.638 INFO:teuthology.orchestra.run.smithi094.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T16:47:06.725 INFO:teuthology.orchestra.run.smithi138.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T16:47:06.825 INFO:teuthology.orchestra.run.smithi138.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3291-g994e2f20-1jammy [726 kB] 2024-04-25T16:47:06.876 INFO:teuthology.orchestra.run.smithi094.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3291-g994e2f20-1jammy [726 kB] 2024-04-25T16:47:06.895 INFO:teuthology.orchestra.run.smithi138.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3291-g994e2f20-1jammy [356 kB] 2024-04-25T16:47:06.931 INFO:teuthology.orchestra.run.smithi138.stdout:Get:12 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3291-g994e2f20-1jammy [32.8 kB] 2024-04-25T16:47:06.933 INFO:teuthology.orchestra.run.smithi138.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3291-g994e2f20-1jammy [180 kB] 2024-04-25T16:47:06.942 INFO:teuthology.orchestra.run.smithi138.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3291-g994e2f20-1jammy [67.1 kB] 2024-04-25T16:47:06.944 INFO:teuthology.orchestra.run.smithi138.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3291-g994e2f20-1jammy [330 kB] 2024-04-25T16:47:06.951 INFO:teuthology.orchestra.run.smithi094.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3291-g994e2f20-1jammy [356 kB] 2024-04-25T16:47:06.961 INFO:teuthology.orchestra.run.smithi094.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3291-g994e2f20-1jammy [32.8 kB] 2024-04-25T16:47:06.977 INFO:teuthology.orchestra.run.smithi138.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3291-g994e2f20-1jammy [7,189 kB] 2024-04-25T16:47:06.984 INFO:teuthology.orchestra.run.smithi094.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3291-g994e2f20-1jammy [180 kB] 2024-04-25T16:47:06.989 INFO:teuthology.orchestra.run.smithi094.stdout:Get:19 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3291-g994e2f20-1jammy [67.1 kB] 2024-04-25T16:47:06.996 INFO:teuthology.orchestra.run.smithi094.stdout:Get:20 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3291-g994e2f20-1jammy [330 kB] 2024-04-25T16:47:07.028 INFO:teuthology.orchestra.run.smithi094.stdout:Get:21 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3291-g994e2f20-1jammy [7,189 kB] 2024-04-25T16:47:07.411 INFO:teuthology.orchestra.run.smithi094.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T16:47:07.464 INFO:teuthology.orchestra.run.smithi094.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T16:47:07.470 INFO:teuthology.orchestra.run.smithi138.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3291-g994e2f20-1jammy [112 kB] 2024-04-25T16:47:07.473 INFO:teuthology.orchestra.run.smithi138.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3291-g994e2f20-1jammy [471 kB] 2024-04-25T16:47:07.508 INFO:teuthology.orchestra.run.smithi094.stdout:Get:24 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3291-g994e2f20-1jammy [112 kB] 2024-04-25T16:47:07.511 INFO:teuthology.orchestra.run.smithi094.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T16:47:07.517 INFO:teuthology.orchestra.run.smithi138.stdout:Get:19 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3291-g994e2f20-1jammy [25.6 MB] 2024-04-25T16:47:07.539 INFO:teuthology.orchestra.run.smithi094.stdout:Get:26 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3291-g994e2f20-1jammy [471 kB] 2024-04-25T16:47:07.549 INFO:teuthology.orchestra.run.smithi094.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T16:47:07.552 INFO:teuthology.orchestra.run.smithi094.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T16:47:07.555 INFO:teuthology.orchestra.run.smithi094.stdout:Get:29 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3291-g994e2f20-1jammy [25.6 MB] 2024-04-25T16:47:07.598 INFO:teuthology.orchestra.run.smithi094.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T16:47:07.636 INFO:teuthology.orchestra.run.smithi094.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T16:47:07.669 INFO:teuthology.orchestra.run.smithi094.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T16:47:07.677 INFO:teuthology.orchestra.run.smithi094.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T16:47:07.679 INFO:teuthology.orchestra.run.smithi094.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T16:47:07.860 INFO:teuthology.orchestra.run.smithi094.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T16:47:07.877 INFO:teuthology.orchestra.run.smithi138.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T16:47:07.946 INFO:teuthology.orchestra.run.smithi094.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T16:47:07.957 INFO:teuthology.orchestra.run.smithi094.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T16:47:07.968 INFO:teuthology.orchestra.run.smithi138.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T16:47:08.026 INFO:teuthology.orchestra.run.smithi094.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T16:47:08.055 INFO:teuthology.orchestra.run.smithi094.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T16:47:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T16:47:08.094 INFO:teuthology.orchestra.run.smithi094.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T16:47:08.282 INFO:teuthology.orchestra.run.smithi094.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T16:47:08.298 INFO:teuthology.orchestra.run.smithi094.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T16:47:08.304 INFO:teuthology.orchestra.run.smithi094.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T16:47:08.310 INFO:teuthology.orchestra.run.smithi094.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T16:47:08.340 INFO:teuthology.orchestra.run.smithi138.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T16:47:08.482 INFO:teuthology.orchestra.run.smithi138.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T16:47:08.497 INFO:teuthology.orchestra.run.smithi094.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T16:47:08.586 INFO:teuthology.orchestra.run.smithi094.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T16:47:08.602 INFO:teuthology.orchestra.run.smithi094.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T16:47:08.787 INFO:teuthology.orchestra.run.smithi094.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T16:47:08.838 INFO:teuthology.orchestra.run.smithi094.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T16:47:08.857 INFO:teuthology.orchestra.run.smithi094.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T16:47:09.097 INFO:teuthology.orchestra.run.smithi138.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T16:47:09.135 INFO:teuthology.orchestra.run.smithi094.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T16:47:09.143 INFO:teuthology.orchestra.run.smithi094.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T16:47:09.163 INFO:teuthology.orchestra.run.smithi094.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T16:47:09.171 INFO:teuthology.orchestra.run.smithi094.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T16:47:09.204 INFO:teuthology.orchestra.run.smithi094.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T16:47:09.209 INFO:teuthology.orchestra.run.smithi138.stdout:Get:25 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3291-g994e2f20-1jammy [4,917 kB] 2024-04-25T16:47:09.349 INFO:teuthology.orchestra.run.smithi094.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3291-g994e2f20-1jammy [4,917 kB] 2024-04-25T16:47:09.424 INFO:teuthology.orchestra.run.smithi094.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T16:47:09.427 INFO:teuthology.orchestra.run.smithi138.stdout:Get:26 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3291-g994e2f20-1jammy [244 kB] 2024-04-25T16:47:09.434 INFO:teuthology.orchestra.run.smithi138.stdout:Get:27 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3291-g994e2f20-1jammy [125 kB] 2024-04-25T16:47:09.438 INFO:teuthology.orchestra.run.smithi138.stdout:Get:28 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3291-g994e2f20-1jammy [1,542 kB] 2024-04-25T16:47:09.533 INFO:teuthology.orchestra.run.smithi138.stdout:Get:29 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3291-g994e2f20-1jammy [6,148 kB] 2024-04-25T16:47:09.634 INFO:teuthology.orchestra.run.smithi094.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3291-g994e2f20-1jammy [244 kB] 2024-04-25T16:47:09.641 INFO:teuthology.orchestra.run.smithi094.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3291-g994e2f20-1jammy [125 kB] 2024-04-25T16:47:09.668 INFO:teuthology.orchestra.run.smithi094.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3291-g994e2f20-1jammy [1,542 kB] 2024-04-25T16:47:09.670 INFO:teuthology.orchestra.run.smithi138.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T16:47:09.687 INFO:teuthology.orchestra.run.smithi138.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T16:47:09.692 INFO:teuthology.orchestra.run.smithi094.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T16:47:09.703 INFO:teuthology.orchestra.run.smithi138.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T16:47:09.712 INFO:teuthology.orchestra.run.smithi138.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T16:47:09.714 INFO:teuthology.orchestra.run.smithi138.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T16:47:09.732 INFO:teuthology.orchestra.run.smithi138.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T16:47:09.767 INFO:teuthology.orchestra.run.smithi138.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T16:47:09.775 INFO:teuthology.orchestra.run.smithi094.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3291-g994e2f20-1jammy [6,148 kB] 2024-04-25T16:47:09.780 INFO:teuthology.orchestra.run.smithi138.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T16:47:09.784 INFO:teuthology.orchestra.run.smithi138.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T16:47:09.785 INFO:teuthology.orchestra.run.smithi138.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T16:47:09.812 INFO:teuthology.orchestra.run.smithi094.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T16:47:09.855 INFO:teuthology.orchestra.run.smithi094.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T16:47:09.913 INFO:teuthology.orchestra.run.smithi138.stdout:Get:40 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3291-g994e2f20-1jammy [22.5 MB] 2024-04-25T16:47:09.956 INFO:teuthology.orchestra.run.smithi138.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T16:47:09.956 INFO:teuthology.orchestra.run.smithi094.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T16:47:09.968 INFO:teuthology.orchestra.run.smithi094.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T16:47:09.987 INFO:teuthology.orchestra.run.smithi094.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T16:47:10.039 INFO:teuthology.orchestra.run.smithi094.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T16:47:10.040 INFO:teuthology.orchestra.run.smithi138.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T16:47:10.054 INFO:teuthology.orchestra.run.smithi138.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T16:47:10.123 INFO:teuthology.orchestra.run.smithi138.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T16:47:10.149 INFO:teuthology.orchestra.run.smithi138.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T16:47:10.165 INFO:teuthology.orchestra.run.smithi138.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T16:47:10.189 INFO:teuthology.orchestra.run.smithi138.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T16:47:10.207 INFO:teuthology.orchestra.run.smithi094.stdout:Get:70 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3291-g994e2f20-1jammy [22.5 MB] 2024-04-25T16:47:10.370 INFO:teuthology.orchestra.run.smithi138.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T16:47:10.379 INFO:teuthology.orchestra.run.smithi094.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T16:47:10.381 INFO:teuthology.orchestra.run.smithi094.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T16:47:10.385 INFO:teuthology.orchestra.run.smithi094.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T16:47:10.386 INFO:teuthology.orchestra.run.smithi138.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T16:47:10.393 INFO:teuthology.orchestra.run.smithi138.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T16:47:10.397 INFO:teuthology.orchestra.run.smithi138.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T16:47:10.551 INFO:teuthology.orchestra.run.smithi094.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T16:47:10.585 INFO:teuthology.orchestra.run.smithi138.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T16:47:10.669 INFO:teuthology.orchestra.run.smithi138.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T16:47:10.685 INFO:teuthology.orchestra.run.smithi138.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T16:47:10.873 INFO:teuthology.orchestra.run.smithi138.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T16:47:10.879 INFO:teuthology.orchestra.run.smithi094.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T16:47:10.924 INFO:teuthology.orchestra.run.smithi138.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T16:47:10.938 INFO:teuthology.orchestra.run.smithi138.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T16:47:11.097 INFO:teuthology.orchestra.run.smithi138.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T16:47:11.098 INFO:teuthology.orchestra.run.smithi138.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T16:47:11.103 INFO:teuthology.orchestra.run.smithi138.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T16:47:11.109 INFO:teuthology.orchestra.run.smithi138.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T16:47:11.123 INFO:teuthology.orchestra.run.smithi138.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T16:47:11.292 INFO:teuthology.orchestra.run.smithi094.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T16:47:11.302 INFO:teuthology.orchestra.run.smithi094.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T16:47:11.304 INFO:teuthology.orchestra.run.smithi094.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T16:47:11.307 INFO:teuthology.orchestra.run.smithi138.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T16:47:11.374 INFO:teuthology.orchestra.run.smithi094.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T16:47:11.375 INFO:teuthology.orchestra.run.smithi094.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T16:47:11.377 INFO:teuthology.orchestra.run.smithi094.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T16:47:11.378 INFO:teuthology.orchestra.run.smithi094.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T16:47:11.379 INFO:teuthology.orchestra.run.smithi094.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T16:47:11.443 INFO:teuthology.orchestra.run.smithi138.stdout:Get:64 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3291-g994e2f20-1jammy [14.1 kB] 2024-04-25T16:47:11.443 INFO:teuthology.orchestra.run.smithi138.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3291-g994e2f20-1jammy [856 kB] 2024-04-25T16:47:11.460 INFO:teuthology.orchestra.run.smithi138.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3291-g994e2f20-1jammy [2,367 kB] 2024-04-25T16:47:11.544 INFO:teuthology.orchestra.run.smithi138.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3291-g994e2f20-1jammy [791 kB] 2024-04-25T16:47:11.546 INFO:teuthology.orchestra.run.smithi094.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T16:47:11.570 INFO:teuthology.orchestra.run.smithi138.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T16:47:11.590 INFO:teuthology.orchestra.run.smithi138.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3291-g994e2f20-1jammy [148 kB] 2024-04-25T16:47:11.614 INFO:teuthology.orchestra.run.smithi138.stdout:Get:70 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3291-g994e2f20-1jammy [2,297 kB] 2024-04-25T16:47:11.645 INFO:teuthology.orchestra.run.smithi094.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T16:47:11.686 INFO:teuthology.orchestra.run.smithi138.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T16:47:11.732 INFO:teuthology.orchestra.run.smithi138.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T16:47:11.738 INFO:teuthology.orchestra.run.smithi094.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T16:47:11.756 INFO:teuthology.orchestra.run.smithi138.stdout:Get:73 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3291-g994e2f20-1jammy [8,625 kB] 2024-04-25T16:47:11.809 INFO:teuthology.orchestra.run.smithi094.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3291-g994e2f20-1jammy [14.1 kB] 2024-04-25T16:47:11.809 INFO:teuthology.orchestra.run.smithi094.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3291-g994e2f20-1jammy [856 kB] 2024-04-25T16:47:11.812 INFO:teuthology.orchestra.run.smithi138.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T16:47:11.824 INFO:teuthology.orchestra.run.smithi138.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T16:47:11.826 INFO:teuthology.orchestra.run.smithi094.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3291-g994e2f20-1jammy [2,367 kB] 2024-04-25T16:47:11.843 INFO:teuthology.orchestra.run.smithi138.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T16:47:11.882 INFO:teuthology.orchestra.run.smithi138.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T16:47:11.971 INFO:teuthology.orchestra.run.smithi094.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3291-g994e2f20-1jammy [791 kB] 2024-04-25T16:47:11.996 INFO:teuthology.orchestra.run.smithi094.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T16:47:12.023 INFO:teuthology.orchestra.run.smithi094.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3291-g994e2f20-1jammy [148 kB] 2024-04-25T16:47:12.027 INFO:teuthology.orchestra.run.smithi094.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3291-g994e2f20-1jammy [2,297 kB] 2024-04-25T16:47:12.054 INFO:teuthology.orchestra.run.smithi094.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T16:47:12.094 INFO:teuthology.orchestra.run.smithi094.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T16:47:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T16:47:12.149 INFO:teuthology.orchestra.run.smithi138.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T16:47:12.151 INFO:teuthology.orchestra.run.smithi138.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T16:47:12.154 INFO:teuthology.orchestra.run.smithi138.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T16:47:12.166 INFO:teuthology.orchestra.run.smithi094.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T16:47:12.179 INFO:teuthology.orchestra.run.smithi094.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3291-g994e2f20-1jammy [8,625 kB] 2024-04-25T16:47:12.210 INFO:teuthology.orchestra.run.smithi094.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T16:47:12.212 INFO:teuthology.orchestra.run.smithi094.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T16:47:12.222 INFO:teuthology.orchestra.run.smithi094.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T16:47:12.284 INFO:teuthology.orchestra.run.smithi138.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3291-g994e2f20-1jammy [14.2 kB] 2024-04-25T16:47:12.285 INFO:teuthology.orchestra.run.smithi138.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3291-g994e2f20-1jammy [44.9 MB] 2024-04-25T16:47:12.319 INFO:teuthology.orchestra.run.smithi138.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T16:47:12.413 INFO:teuthology.orchestra.run.smithi094.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T16:47:12.658 INFO:teuthology.orchestra.run.smithi138.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T16:47:12.765 INFO:teuthology.orchestra.run.smithi094.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3291-g994e2f20-1jammy [14.2 kB] 2024-04-25T16:47:12.766 INFO:teuthology.orchestra.run.smithi094.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3291-g994e2f20-1jammy [44.9 MB] 2024-04-25T16:47:13.067 INFO:teuthology.orchestra.run.smithi138.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T16:47:13.077 INFO:teuthology.orchestra.run.smithi138.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T16:47:13.078 INFO:teuthology.orchestra.run.smithi138.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T16:47:13.081 INFO:teuthology.orchestra.run.smithi094.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T16:47:13.093 INFO:teuthology.orchestra.run.smithi094.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T16:47:13.148 INFO:teuthology.orchestra.run.smithi138.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T16:47:13.149 INFO:teuthology.orchestra.run.smithi138.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T16:47:13.151 INFO:teuthology.orchestra.run.smithi138.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T16:47:13.152 INFO:teuthology.orchestra.run.smithi138.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T16:47:13.153 INFO:teuthology.orchestra.run.smithi138.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T16:47:13.184 INFO:teuthology.orchestra.run.smithi094.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T16:47:13.186 INFO:teuthology.orchestra.run.smithi094.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T16:47:13.222 INFO:teuthology.orchestra.run.smithi094.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T16:47:13.227 INFO:teuthology.orchestra.run.smithi094.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T16:47:13.272 INFO:teuthology.orchestra.run.smithi094.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T16:47:13.292 INFO:teuthology.orchestra.run.smithi094.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T16:47:13.331 INFO:teuthology.orchestra.run.smithi138.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T16:47:13.431 INFO:teuthology.orchestra.run.smithi138.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T16:47:13.434 INFO:teuthology.orchestra.run.smithi094.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T16:47:13.436 INFO:teuthology.orchestra.run.smithi094.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T16:47:13.527 INFO:teuthology.orchestra.run.smithi138.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T16:47:13.603 INFO:teuthology.orchestra.run.smithi094.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T16:47:13.838 INFO:teuthology.orchestra.run.smithi138.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T16:47:13.939 INFO:teuthology.orchestra.run.smithi138.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T16:47:14.003 INFO:teuthology.orchestra.run.smithi138.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T16:47:14.021 INFO:teuthology.orchestra.run.smithi138.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T16:47:14.043 INFO:teuthology.orchestra.run.smithi094.stdout:Get:116 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T16:47:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:Get:117 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T16:47:14.108 INFO:teuthology.orchestra.run.smithi138.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T16:47:14.161 INFO:teuthology.orchestra.run.smithi138.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T16:47:14.164 INFO:teuthology.orchestra.run.smithi138.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T16:47:14.178 INFO:teuthology.orchestra.run.smithi138.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T16:47:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3291-g994e2f20-1jammy [124 kB] 2024-04-25T16:47:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3291-g994e2f20-1jammy [236 kB] 2024-04-25T16:47:15.199 INFO:teuthology.orchestra.run.smithi138.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3291-g994e2f20-1jammy [33.9 kB] 2024-04-25T16:47:15.200 INFO:teuthology.orchestra.run.smithi138.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3291-g994e2f20-1jammy [23.4 kB] 2024-04-25T16:47:15.201 INFO:teuthology.orchestra.run.smithi138.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3291-g994e2f20-1jammy [39.0 kB] 2024-04-25T16:47:15.202 INFO:teuthology.orchestra.run.smithi138.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3291-g994e2f20-1jammy [14.2 MB] 2024-04-25T16:47:15.448 INFO:teuthology.orchestra.run.smithi138.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T16:47:15.897 INFO:teuthology.orchestra.run.smithi138.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3291-g994e2f20-1jammy [92.6 kB] 2024-04-25T16:47:15.985 INFO:teuthology.orchestra.run.smithi138.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T16:47:15.991 INFO:teuthology.orchestra.run.smithi138.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T16:47:16.061 INFO:teuthology.orchestra.run.smithi138.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T16:47:16.061 INFO:teuthology.orchestra.run.smithi138.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T16:47:16.068 INFO:teuthology.orchestra.run.smithi138.stdout:Get:116 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T16:47:16.071 INFO:teuthology.orchestra.run.smithi138.stdout:Get:117 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T16:47:16.090 INFO:teuthology.orchestra.run.smithi138.stdout:Get:118 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T16:47:16.104 INFO:teuthology.orchestra.run.smithi138.stdout:Get:119 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T16:47:16.131 INFO:teuthology.orchestra.run.smithi094.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3291-g994e2f20-1jammy [124 kB] 2024-04-25T16:47:16.134 INFO:teuthology.orchestra.run.smithi094.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3291-g994e2f20-1jammy [236 kB] 2024-04-25T16:47:16.139 INFO:teuthology.orchestra.run.smithi094.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3291-g994e2f20-1jammy [33.9 kB] 2024-04-25T16:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3291-g994e2f20-1jammy [23.4 kB] 2024-04-25T16:47:16.183 INFO:teuthology.orchestra.run.smithi138.stdout:Get:120 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T16:47:16.184 INFO:teuthology.orchestra.run.smithi138.stdout:Get:121 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T16:47:16.196 INFO:teuthology.orchestra.run.smithi094.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3291-g994e2f20-1jammy [39.0 kB] 2024-04-25T16:47:16.197 INFO:teuthology.orchestra.run.smithi094.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3291-g994e2f20-1jammy [14.2 MB] 2024-04-25T16:47:16.349 INFO:teuthology.orchestra.run.smithi138.stdout:Get:122 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T16:47:16.627 INFO:teuthology.orchestra.run.smithi094.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3291-g994e2f20-1jammy [92.6 kB] 2024-04-25T16:47:16.795 INFO:teuthology.orchestra.run.smithi138.stdout:Get:123 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T16:47:16.819 INFO:teuthology.orchestra.run.smithi138.stdout:Get:124 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T16:47:17.242 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 175 MB in 11s (15.6 MB/s) 2024-04-25T16:47:17.378 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T16:47:17.455 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T16:47:17.460 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T16:47:17.498 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T16:47:17.801 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T16:47:17.827 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T16:47:17.846 INFO:teuthology.orchestra.run.smithi138.stdout:Fetched 175 MB in 12s (14.7 MB/s) 2024-04-25T16:47:17.867 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T16:47:17.977 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T16:47:18.063 INFO:teuthology.orchestra.run.smithi138.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T16:47:18.069 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T16:47:18.109 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T16:47:18.170 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T16:47:18.196 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T16:47:18.236 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T16:47:18.362 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T16:47:18.388 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T16:47:18.420 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T16:47:18.598 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T16:47:18.624 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T16:47:18.690 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T16:47:18.739 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T16:47:18.765 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T16:47:18.806 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T16:47:18.968 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T16:47:18.994 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T16:47:19.034 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T16:47:19.117 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T16:47:19.143 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T16:47:19.227 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T16:47:19.579 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T16:47:19.579 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T16:47:19.605 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T16:47:19.606 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T16:47:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T16:47:19.646 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T16:47:19.974 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T16:47:20.002 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T16:47:20.040 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T16:47:20.091 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T16:47:20.118 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T16:47:20.166 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T16:47:20.485 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T16:47:20.512 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T16:47:20.527 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T16:47:20.553 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T16:47:20.560 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T16:47:20.585 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T16:47:20.921 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T16:47:20.947 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T16:47:20.955 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T16:47:20.981 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T16:47:20.988 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T16:47:21.022 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T16:47:21.318 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T16:47:21.341 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T16:47:21.366 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T16:47:21.376 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T16:47:21.393 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T16:47:21.433 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T16:47:21.688 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librados2. 2024-04-25T16:47:21.714 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../010-librados2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:21.754 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:21.795 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T16:47:21.820 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T16:47:21.861 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T16:47:22.181 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package librados2. 2024-04-25T16:47:22.207 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../010-librados2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:22.207 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librbd1. 2024-04-25T16:47:22.234 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../011-librbd1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:22.247 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:22.274 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:22.691 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package librbd1. 2024-04-25T16:47:22.707 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../011-librbd1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:22.741 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:22.761 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T16:47:22.787 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:22.827 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:23.088 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rados. 2024-04-25T16:47:23.103 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:23.138 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:23.244 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T16:47:23.271 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:23.466 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T16:47:23.492 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:47:23.533 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:23.793 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T16:47:23.809 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:23.847 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-rados. 2024-04-25T16:47:23.852 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:23.862 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:23.905 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:24.130 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T16:47:24.156 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:47:24.213 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:24.241 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T16:47:24.268 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:47:24.308 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:24.527 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T16:47:24.542 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:24.585 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:24.617 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T16:47:24.643 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T16:47:24.683 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T16:47:24.896 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T16:47:24.923 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:47:24.963 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:24.969 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T16:47:24.996 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:25.028 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:25.283 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T16:47:25.310 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T16:47:25.349 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T16:47:25.356 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T16:47:25.383 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T16:47:25.422 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T16:47:25.593 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T16:47:25.608 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:25.652 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:25.709 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T16:47:25.724 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T16:47:25.767 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T16:47:25.955 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T16:47:25.981 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T16:47:26.013 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T16:47:26.061 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T16:47:26.077 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T16:47:26.120 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T16:47:26.274 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T16:47:26.300 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T16:47:26.340 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T16:47:26.440 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua5.1. 2024-04-25T16:47:26.466 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T16:47:26.506 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T16:47:26.643 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T16:47:26.670 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T16:47:26.710 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T16:47:26.827 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua-any. 2024-04-25T16:47:26.853 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T16:47:26.893 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T16:47:27.030 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package lua5.1. 2024-04-25T16:47:27.056 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T16:47:27.096 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T16:47:27.137 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package zip. 2024-04-25T16:47:27.152 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T16:47:27.196 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T16:47:27.383 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package lua-any. 2024-04-25T16:47:27.398 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T16:47:27.433 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T16:47:27.474 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package unzip. 2024-04-25T16:47:27.500 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T16:47:27.532 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T16:47:27.710 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package zip. 2024-04-25T16:47:27.733 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T16:47:27.777 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T16:47:27.945 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package luarocks. 2024-04-25T16:47:27.971 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T16:47:28.011 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T16:47:28.097 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package unzip. 2024-04-25T16:47:28.123 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T16:47:28.164 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T16:47:28.428 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librgw2. 2024-04-25T16:47:28.454 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../027-librgw2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:28.494 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:28.550 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package luarocks. 2024-04-25T16:47:28.577 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T16:47:28.617 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T16:47:28.983 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package librgw2. 2024-04-25T16:47:29.009 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../027-librgw2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:29.049 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:29.130 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T16:47:29.146 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:29.189 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:29.509 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T16:47:29.535 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T16:47:29.575 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T16:47:29.660 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T16:47:29.687 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:29.727 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:29.887 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T16:47:29.913 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:29.953 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:30.063 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T16:47:30.090 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T16:47:30.130 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T16:47:30.324 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-common. 2024-04-25T16:47:30.350 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:30.381 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:30.441 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T16:47:30.467 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:30.507 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:30.894 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-common. 2024-04-25T16:47:30.921 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:30.952 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:31.929 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-base. 2024-04-25T16:47:31.956 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:32.005 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:32.474 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-base. 2024-04-25T16:47:32.501 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:32.550 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T16:47:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T16:47:32.692 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T16:47:32.953 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T16:47:32.980 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T16:47:33.020 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T16:47:33.104 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T16:47:33.130 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T16:47:33.162 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T16:47:33.348 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T16:47:33.375 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T16:47:33.414 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T16:47:33.456 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T16:47:33.482 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T16:47:33.531 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T16:47:33.718 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T16:47:33.745 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T16:47:33.784 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T16:47:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T16:47:34.044 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T16:47:34.062 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T16:47:34.078 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T16:47:34.084 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T16:47:34.113 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T16:47:34.387 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T16:47:34.390 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T16:47:34.413 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T16:47:34.417 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T16:47:34.453 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T16:47:34.457 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T16:47:34.723 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T16:47:34.750 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T16:47:34.806 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T16:47:34.952 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-portend. 2024-04-25T16:47:34.979 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T16:47:35.018 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T16:47:35.109 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T16:47:35.136 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T16:47:35.176 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T16:47:35.313 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T16:47:35.340 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T16:47:35.379 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T16:47:35.479 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-portend. 2024-04-25T16:47:35.505 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T16:47:35.545 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T16:47:35.649 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T16:47:35.676 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T16:47:35.716 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T16:47:35.824 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T16:47:35.849 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T16:47:35.890 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T16:47:36.062 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T16:47:36.089 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T16:47:36.128 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T16:47:36.142 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T16:47:36.169 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T16:47:36.209 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T16:47:36.406 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T16:47:36.433 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T16:47:36.465 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T16:47:36.554 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T16:47:36.582 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T16:47:36.621 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T16:47:36.776 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T16:47:36.802 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T16:47:36.843 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T16:47:36.899 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T16:47:36.914 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T16:47:36.949 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T16:47:37.146 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-mako. 2024-04-25T16:47:37.172 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T16:47:37.212 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T16:47:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T16:47:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T16:47:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T16:47:37.574 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T16:47:37.601 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T16:47:37.629 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-mako. 2024-04-25T16:47:37.640 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T16:47:37.656 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T16:47:37.696 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T16:47:37.902 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T16:47:37.928 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T16:47:37.960 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T16:47:38.008 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T16:47:38.034 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T16:47:38.074 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T16:47:38.280 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-webob. 2024-04-25T16:47:38.306 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T16:47:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T16:47:38.410 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T16:47:38.437 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T16:47:38.468 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T16:47:39.434 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T16:47:39.460 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T16:47:39.503 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T16:47:39.572 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-webob. 2024-04-25T16:47:39.599 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T16:47:39.638 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T16:47:39.812 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T16:47:39.838 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T16:47:39.878 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T16:47:39.983 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T16:47:40.009 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T16:47:40.053 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T16:47:40.148 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-paste. 2024-04-25T16:47:40.175 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T16:47:40.206 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T16:47:40.344 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T16:47:40.370 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T16:47:40.410 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T16:47:40.534 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T16:47:40.551 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T16:47:40.593 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T16:47:40.699 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-paste. 2024-04-25T16:47:40.714 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T16:47:40.757 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T16:47:40.862 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T16:47:40.889 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T16:47:40.920 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T16:47:41.069 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T16:47:41.081 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T16:47:41.119 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T16:47:41.216 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T16:47:41.242 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T16:47:41.282 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T16:47:41.406 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T16:47:41.433 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T16:47:41.472 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T16:47:41.627 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T16:47:41.654 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T16:47:41.693 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T16:47:41.776 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T16:47:41.802 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T16:47:41.843 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T16:47:42.057 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T16:47:42.080 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T16:47:42.123 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T16:47:42.181 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T16:47:42.206 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T16:47:42.245 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T16:47:42.460 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-idna. 2024-04-25T16:47:42.486 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T16:47:42.535 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T16:47:42.576 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T16:47:42.604 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T16:47:42.635 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T16:47:42.813 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T16:47:42.840 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T16:47:42.879 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T16:47:42.955 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-idna. 2024-04-25T16:47:42.981 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T16:47:43.021 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T16:47:43.166 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-requests. 2024-04-25T16:47:43.193 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T16:47:43.233 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T16:47:43.333 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T16:47:43.360 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T16:47:43.400 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T16:47:43.544 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T16:47:43.571 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T16:47:43.611 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T16:47:43.729 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-requests. 2024-04-25T16:47:43.755 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T16:47:43.795 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T16:47:43.881 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T16:47:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:47:43.947 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:44.074 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T16:47:44.100 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T16:47:44.132 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T16:47:44.303 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T16:47:44.330 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:44.361 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:44.585 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T16:47:44.601 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:47:44.644 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:44.681 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T16:47:44.708 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T16:47:44.747 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T16:47:45.025 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T16:47:45.051 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:45.075 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T16:47:45.091 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:45.102 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T16:47:45.141 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T16:47:45.369 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T16:47:45.396 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T16:47:45.428 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T16:47:45.453 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T16:47:45.480 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:45.511 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:45.748 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T16:47:45.775 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T16:47:45.823 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T16:47:45.940 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T16:47:45.966 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:46.006 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:46.160 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T16:47:46.186 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:46.226 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:46.588 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T16:47:46.615 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:46.643 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T16:47:46.646 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:46.670 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:46.710 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:47.284 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T16:47:47.311 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:47.350 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:47.763 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph. 2024-04-25T16:47:47.779 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../068-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:47.813 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:48.083 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T16:47:48.109 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:48.149 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:48.421 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph. 2024-04-25T16:47:48.448 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../068-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:48.487 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:48.545 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T16:47:48.571 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:48.611 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:48.790 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T16:47:48.817 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:48.857 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:48.998 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephadm. 2024-04-25T16:47:49.024 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../071-cephadm_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:49.056 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:49.252 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T16:47:49.279 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:49.309 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T16:47:49.319 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:49.335 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T16:47:49.375 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T16:47:49.712 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T16:47:49.727 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T16:47:49.770 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T16:47:49.772 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package cephadm. 2024-04-25T16:47:49.799 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../071-cephadm_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:47:49.839 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:50.142 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T16:47:50.169 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T16:47:50.209 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T16:47:50.512 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T16:47:50.539 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T16:47:50.578 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T16:47:51.017 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-babel. 2024-04-25T16:47:51.044 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T16:47:51.083 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T16:47:51.412 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T16:47:51.439 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T16:47:51.478 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T16:47:51.602 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-babel. 2024-04-25T16:47:51.628 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T16:47:51.668 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T16:47:51.831 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T16:47:51.858 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:47:51.898 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:51.988 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T16:47:52.014 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T16:47:52.054 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T16:47:52.235 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T16:47:52.261 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T16:47:52.301 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T16:47:52.400 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T16:47:52.427 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:47:52.458 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:52.562 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-routes. 2024-04-25T16:47:52.578 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T16:47:52.612 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T16:47:52.803 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T16:47:52.830 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T16:47:52.861 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T16:47:52.924 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T16:47:52.951 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:47:52.990 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:53.157 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-routes. 2024-04-25T16:47:53.184 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T16:47:53.223 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T16:47:53.535 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T16:47:53.562 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:47:53.602 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:54.221 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T16:47:54.237 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T16:47:54.279 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T16:47:54.775 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T16:47:54.802 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T16:47:54.841 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T16:47:55.008 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T16:47:55.024 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T16:47:55.058 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T16:47:55.254 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T16:47:55.280 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T16:47:55.406 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T16:47:55.570 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T16:47:55.597 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T16:47:55.637 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T16:47:55.700 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T16:47:55.727 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T16:47:55.758 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T16:47:55.999 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T16:47:56.026 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T16:47:56.152 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T16:47:56.460 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T16:47:56.463 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T16:47:56.487 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:47:56.490 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T16:47:56.526 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:56.530 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T16:47:57.208 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T16:47:57.235 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:47:57.274 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:47:57.513 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T16:47:57.541 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T16:47:57.580 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T16:47:57.883 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T16:47:57.910 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T16:47:57.949 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T16:47:58.269 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T16:47:58.296 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T16:47:58.320 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T16:47:58.336 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T16:47:58.346 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T16:47:58.386 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T16:47:58.614 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T16:47:58.641 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T16:47:58.681 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T16:47:58.706 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T16:47:58.733 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T16:47:58.772 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T16:47:59.010 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T16:47:59.035 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T16:47:59.037 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T16:47:59.059 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T16:47:59.076 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T16:47:59.093 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T16:47:59.388 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T16:47:59.407 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T16:47:59.415 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T16:47:59.434 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T16:47:59.454 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T16:47:59.473 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T16:47:59.759 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T16:47:59.785 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T16:47:59.786 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T16:47:59.812 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T16:47:59.825 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T16:47:59.851 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T16:48:00.163 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T16:48:00.190 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T16:48:00.229 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T16:48:00.272 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T16:48:00.299 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T16:48:00.371 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T16:48:00.666 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T16:48:00.693 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T16:48:00.733 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T16:48:01.012 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T16:48:01.039 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T16:48:01.070 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T16:48:01.136 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T16:48:01.164 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:48:01.202 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:01.497 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T16:48:01.524 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T16:48:01.597 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T16:48:01.751 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T16:48:01.767 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:48:01.809 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:01.942 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T16:48:01.969 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T16:48:02.008 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T16:48:02.096 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T16:48:02.123 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T16:48:02.163 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T16:48:02.261 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package jq. 2024-04-25T16:48:02.289 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T16:48:02.327 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T16:48:02.491 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T16:48:02.519 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T16:48:02.555 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package socat. 2024-04-25T16:48:02.558 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T16:48:02.571 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T16:48:02.614 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T16:48:02.819 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package jq. 2024-04-25T16:48:02.847 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T16:48:02.886 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T16:48:02.900 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T16:48:02.916 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T16:48:02.958 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T16:48:03.114 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package socat. 2024-04-25T16:48:03.141 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T16:48:03.180 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T16:48:03.330 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-test. 2024-04-25T16:48:03.357 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:48:03.396 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:03.485 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T16:48:03.512 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T16:48:03.551 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T16:48:03.873 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-test. 2024-04-25T16:48:03.888 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:48:03.922 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:05.751 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T16:48:05.778 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:48:05.817 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:06.155 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T16:48:06.183 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:48:06.221 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:06.361 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T16:48:06.389 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:48:06.427 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:06.542 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T16:48:06.569 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T16:48:06.608 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T16:48:06.799 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T16:48:06.827 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:48:06.858 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:06.869 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T16:48:06.896 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:48:06.935 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:07.178 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T16:48:07.188 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T16:48:07.203 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:48:07.205 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T16:48:07.238 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:07.244 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T16:48:07.474 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T16:48:07.502 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:48:07.505 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T16:48:07.533 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:48:07.541 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:07.572 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:07.825 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T16:48:07.841 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T16:48:07.843 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T16:48:07.859 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T16:48:07.875 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:07.902 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T16:48:08.103 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T16:48:08.130 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:48:08.169 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T16:48:08.245 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T16:48:08.290 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T16:48:08.489 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T16:48:08.517 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T16:48:08.556 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T16:48:08.652 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T16:48:08.680 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T16:48:08.718 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T16:48:08.886 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T16:48:08.914 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T16:48:08.952 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T16:48:09.133 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package pkg-config. 2024-04-25T16:48:09.160 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T16:48:09.199 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T16:48:09.307 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T16:48:09.335 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T16:48:09.374 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T16:48:09.410 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T16:48:09.426 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T16:48:09.460 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T16:48:09.805 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package pkg-config. 2024-04-25T16:48:09.833 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T16:48:09.871 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T16:48:09.872 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T16:48:09.900 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T16:48:09.930 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T16:48:10.134 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T16:48:10.149 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T16:48:10.192 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T16:48:10.218 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T16:48:10.244 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T16:48:10.283 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T16:48:10.546 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T16:48:10.570 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T16:48:10.603 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T16:48:10.604 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T16:48:10.631 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T16:48:10.670 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T16:48:10.915 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T16:48:10.943 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T16:48:10.982 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T16:48:10.990 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T16:48:11.017 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T16:48:11.056 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T16:48:11.303 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T16:48:11.330 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T16:48:11.369 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T16:48:11.393 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-py. 2024-04-25T16:48:11.420 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T16:48:11.459 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T16:48:11.631 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T16:48:11.658 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T16:48:11.689 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T16:48:11.788 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T16:48:11.815 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T16:48:11.846 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T16:48:12.035 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-py. 2024-04-25T16:48:12.062 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T16:48:12.101 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T16:48:12.261 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T16:48:12.286 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T16:48:12.325 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T16:48:12.413 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T16:48:12.441 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T16:48:12.480 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T16:48:12.578 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-toml. 2024-04-25T16:48:12.594 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T16:48:12.636 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T16:48:12.935 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T16:48:12.939 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T16:48:12.962 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T16:48:12.967 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T16:48:13.001 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T16:48:13.005 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T16:48:13.313 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-toml. 2024-04-25T16:48:13.340 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T16:48:13.343 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T16:48:13.371 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T16:48:13.379 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T16:48:13.409 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T16:48:13.666 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T16:48:13.679 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T16:48:13.694 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T16:48:13.707 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T16:48:13.745 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T16:48:13.757 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T16:48:14.078 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T16:48:14.095 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T16:48:14.128 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T16:48:14.390 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T16:48:14.406 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T16:48:14.448 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T16:48:14.483 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package radosgw. 2024-04-25T16:48:14.511 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../122-radosgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:48:14.550 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package radosgw. 2024-04-25T16:48:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../122-radosgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:48:15.219 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:15.536 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T16:48:15.564 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:48:15.602 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:15.925 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T16:48:16.214 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T16:48:16.231 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T16:48:16.273 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:16.317 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T16:48:16.543 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T16:48:16.574 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T16:48:16.779 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T16:48:16.839 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T16:48:16.965 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T16:48:17.127 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T16:48:17.223 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T16:48:17.540 INFO:teuthology.orchestra.run.smithi094.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T16:48:17.559 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T16:48:17.611 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:18.017 INFO:teuthology.orchestra.run.smithi138.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T16:48:18.093 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:18.103 INFO:teuthology.orchestra.run.smithi094.stdout:Adding system user cephadm....done 2024-04-25T16:48:18.284 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T16:48:18.553 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T16:48:18.812 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T16:48:18.938 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T16:48:19.044 INFO:teuthology.orchestra.run.smithi138.stdout:Adding system user cephadm....done 2024-04-25T16:48:19.193 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T16:48:19.198 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T16:48:19.441 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T16:48:19.478 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T16:48:19.567 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T16:48:19.729 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T16:48:19.856 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T16:48:19.877 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T16:48:20.107 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T16:48:20.237 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T16:48:20.375 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T16:48:20.493 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T16:48:20.497 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T16:48:20.757 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T16:48:20.787 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T16:48:21.025 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T16:48:21.277 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T16:48:21.297 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T16:48:21.574 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T16:48:21.707 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T16:48:21.825 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T16:48:21.987 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T16:48:22.077 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T16:48:22.239 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:22.354 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T16:48:22.515 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T16:48:22.641 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T16:48:22.755 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T16:48:22.767 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T16:48:23.010 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T16:48:23.014 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T16:48:23.111 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T16:48:23.274 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:23.353 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T16:48:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T16:48:23.643 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T16:48:23.756 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T16:48:23.770 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T16:48:23.786 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T16:48:23.857 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T16:48:23.983 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T16:48:24.029 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T16:48:24.155 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T16:48:24.343 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T16:48:24.407 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T16:48:24.444 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T16:48:24.687 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T16:48:24.856 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T16:48:24.889 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T16:48:24.944 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T16:48:24.954 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T16:48:25.087 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T16:48:25.272 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T16:48:25.407 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T16:48:25.446 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T16:48:25.573 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T16:48:25.832 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T16:48:26.083 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T16:48:26.092 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T16:48:26.352 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T16:48:26.393 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T16:48:26.478 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T16:48:26.520 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T16:48:26.604 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T16:48:26.914 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T16:48:27.205 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T16:48:27.315 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T16:48:27.457 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T16:48:27.558 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T16:48:27.566 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T16:48:27.684 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T16:48:27.960 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T16:48:27.977 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T16:48:28.244 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T16:48:28.370 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T16:48:28.562 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T16:48:28.621 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T16:48:28.914 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T16:48:28.990 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T16:48:29.148 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T16:48:29.249 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:29.266 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T16:48:29.551 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T16:48:29.551 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T16:48:29.860 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T16:48:29.894 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T16:48:30.154 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T16:48:30.280 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:30.565 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T16:48:30.866 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T16:48:31.091 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T16:48:31.343 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T16:48:31.452 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T16:48:31.729 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T16:48:31.871 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T16:48:31.981 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T16:48:32.040 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T16:48:32.266 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T16:48:32.283 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T16:48:32.418 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T16:48:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T16:48:32.686 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T16:48:32.760 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T16:48:32.812 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T16:48:32.861 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T16:48:32.939 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T16:48:33.146 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T16:48:33.232 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T16:48:33.397 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T16:48:33.475 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T16:48:33.656 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T16:48:33.718 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T16:48:33.844 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T16:48:33.908 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T16:48:34.146 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T16:48:34.167 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T16:48:34.405 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T16:48:34.435 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T16:48:34.536 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T16:48:34.665 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T16:48:34.804 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T16:48:34.900 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T16:48:34.955 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T16:48:35.134 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T16:48:35.231 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T16:48:35.394 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T16:48:35.520 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T16:48:35.524 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T16:48:35.775 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T16:48:35.830 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T16:48:35.893 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T16:48:35.956 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T16:48:36.002 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T16:48:36.075 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T16:48:36.249 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T16:48:36.321 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T16:48:36.447 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T16:48:36.734 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T16:48:36.823 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T16:48:36.993 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T16:48:37.108 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T16:48:37.119 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T16:48:37.234 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T16:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T16:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T16:48:37.477 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T16:48:37.564 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T16:48:37.603 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T16:48:37.674 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T16:48:37.912 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T16:48:38.059 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T16:48:38.238 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T16:48:38.319 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T16:48:38.445 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T16:48:38.498 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T16:48:38.693 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T16:48:38.696 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T16:48:38.822 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T16:48:38.970 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T16:48:39.132 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T16:48:39.254 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T16:48:39.408 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T16:48:39.514 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T16:48:39.676 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T16:48:39.897 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T16:48:39.899 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T16:48:40.182 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T16:48:40.184 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T16:48:40.433 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T16:48:40.520 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T16:48:40.663 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:40.693 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T16:48:40.772 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:40.874 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T16:48:41.095 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T16:48:41.202 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:41.328 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:41.380 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T16:48:41.454 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T16:48:41.725 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T16:48:41.747 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:41.834 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:41.848 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:41.950 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:41.961 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:42.087 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T16:48:42.172 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T16:48:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T16:48:42.424 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:42.541 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:42.659 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:42.668 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T16:48:42.785 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:42.912 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:42.945 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:43.038 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:43.071 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:43.164 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T16:48:43.197 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:43.408 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T16:48:43.408 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T16:48:43.857 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:43.983 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:44.084 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:44.219 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:44.345 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T16:48:44.367 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:44.469 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:44.570 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:44.696 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:44.847 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:44.973 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:45.492 INFO:teuthology.orchestra.run.smithi094.stdout:Adding group ceph....done 2024-04-25T16:48:45.608 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:45.734 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:45.844 INFO:teuthology.orchestra.run.smithi094.stdout:Adding system user ceph....done 2024-04-25T16:48:45.860 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:45.995 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:46.012 INFO:teuthology.orchestra.run.smithi094.stdout:Setting system user ceph properties....done 2024-04-25T16:48:46.029 INFO:teuthology.orchestra.run.smithi094.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T16:48:46.121 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T16:48:46.247 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:46.502 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T16:48:46.816 INFO:teuthology.orchestra.run.smithi138.stdout:Adding group ceph....done 2024-04-25T16:48:47.009 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:47.209 INFO:teuthology.orchestra.run.smithi138.stdout:Adding system user ceph....done 2024-04-25T16:48:47.216 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:48:47.217 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:48:47.370 INFO:teuthology.orchestra.run.smithi138.stdout:Setting system user ceph properties....done 2024-04-25T16:48:47.384 INFO:teuthology.orchestra.run.smithi138.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T16:48:47.522 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T16:48:47.727 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:47.870 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T16:48:48.327 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:48:48.327 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:48:48.534 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:48.744 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:48:48.744 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:48:48.788 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:49.134 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T16:48:49.219 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:49.345 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:49.615 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:49.814 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:48:49.814 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:48:49.847 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T16:48:49.847 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T16:48:50.289 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:50.324 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:50.536 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T16:48:50.537 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T16:48:50.622 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T16:48:50.993 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:51.074 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:51.276 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T16:48:51.276 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T16:48:51.299 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T16:48:51.300 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T16:48:51.760 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:51.795 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:51.920 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:52.041 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T16:48:52.041 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T16:48:52.071 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:52.287 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T16:48:52.287 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T16:48:52.520 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:52.768 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:52.791 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T16:48:52.791 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T16:48:52.899 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:53.025 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:53.185 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:53.289 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:53.424 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:53.529 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T16:48:53.567 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:53.630 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T16:48:53.765 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T16:48:53.814 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T16:48:53.814 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T16:48:54.260 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:54.411 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:54.538 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:54.689 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T16:48:54.849 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T16:48:55.034 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T16:48:55.151 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T16:48:55.304 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T16:48:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T16:48:55.510 DEBUG:teuthology.orchestra.run.smithi094:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T16:48:55.571 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-04-25T16:48:55.770 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2024-04-25T16:48:55.771 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2024-04-25T16:48:55.957 INFO:teuthology.orchestra.run.smithi094.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T16:48:55.958 INFO:teuthology.orchestra.run.smithi094.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T16:48:55.958 INFO:teuthology.orchestra.run.smithi094.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T16:48:55.958 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T16:48:55.958 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T16:48:55.958 INFO:teuthology.orchestra.run.smithi094.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T16:48:55.958 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T16:48:55.959 INFO:teuthology.orchestra.run.smithi094.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T16:48:55.959 INFO:teuthology.orchestra.run.smithi094.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T16:48:55.959 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T16:48:55.959 INFO:teuthology.orchestra.run.smithi094.stdout:The following additional packages will be installed: 2024-04-25T16:48:55.959 INFO:teuthology.orchestra.run.smithi094.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T16:48:55.959 INFO:teuthology.orchestra.run.smithi094.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T16:48:55.960 INFO:teuthology.orchestra.run.smithi094.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T16:48:55.960 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T16:48:55.960 INFO:teuthology.orchestra.run.smithi094.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T16:48:55.961 INFO:teuthology.orchestra.run.smithi094.stdout:Suggested packages: 2024-04-25T16:48:55.961 INFO:teuthology.orchestra.run.smithi094.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T16:48:55.961 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T16:48:55.961 INFO:teuthology.orchestra.run.smithi094.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T16:48:56.000 INFO:teuthology.orchestra.run.smithi094.stdout:The following NEW packages will be installed: 2024-04-25T16:48:56.001 INFO:teuthology.orchestra.run.smithi094.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T16:48:56.001 INFO:teuthology.orchestra.run.smithi094.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T16:48:56.001 INFO:teuthology.orchestra.run.smithi094.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T16:48:56.001 INFO:teuthology.orchestra.run.smithi094.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T16:48:56.001 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T16:48:56.002 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T16:48:56.002 INFO:teuthology.orchestra.run.smithi094.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T16:48:56.002 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages will be upgraded: 2024-04-25T16:48:56.003 INFO:teuthology.orchestra.run.smithi094.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T16:48:56.062 INFO:teuthology.orchestra.run.smithi094.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T16:48:56.062 INFO:teuthology.orchestra.run.smithi094.stdout:Need to get 55.7 MB of archives. 2024-04-25T16:48:56.062 INFO:teuthology.orchestra.run.smithi094.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T16:48:56.062 INFO:teuthology.orchestra.run.smithi094.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T16:48:56.222 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T16:48:56.229 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T16:48:56.269 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T16:48:56.269 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T16:48:56.270 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T16:48:56.281 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T16:48:56.282 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T16:48:56.284 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T16:48:56.286 INFO:teuthology.orchestra.run.smithi094.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T16:48:56.286 INFO:teuthology.orchestra.run.smithi094.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T16:48:56.287 INFO:teuthology.orchestra.run.smithi094.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T16:48:56.291 INFO:teuthology.orchestra.run.smithi094.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T16:48:56.310 INFO:teuthology.orchestra.run.smithi094.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T16:48:56.311 INFO:teuthology.orchestra.run.smithi094.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T16:48:56.313 INFO:teuthology.orchestra.run.smithi094.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T16:48:56.407 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T16:48:56.614 INFO:teuthology.orchestra.run.smithi094.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T16:48:56.616 INFO:teuthology.orchestra.run.smithi094.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T16:48:56.616 INFO:teuthology.orchestra.run.smithi094.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T16:48:56.618 INFO:teuthology.orchestra.run.smithi094.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T16:48:56.679 INFO:teuthology.orchestra.run.smithi094.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T16:48:56.684 INFO:teuthology.orchestra.run.smithi094.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T16:48:56.686 INFO:teuthology.orchestra.run.smithi094.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T16:48:56.694 INFO:teuthology.orchestra.run.smithi094.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T16:48:56.694 INFO:teuthology.orchestra.run.smithi094.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T16:48:56.719 INFO:teuthology.orchestra.run.smithi094.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T16:48:56.719 INFO:teuthology.orchestra.run.smithi094.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T16:48:56.730 INFO:teuthology.orchestra.run.smithi094.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T16:48:56.750 INFO:teuthology.orchestra.run.smithi094.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T16:48:56.753 INFO:teuthology.orchestra.run.smithi094.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T16:48:56.753 INFO:teuthology.orchestra.run.smithi094.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T16:48:56.754 INFO:teuthology.orchestra.run.smithi094.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T16:48:56.754 INFO:teuthology.orchestra.run.smithi094.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T16:48:56.764 INFO:teuthology.orchestra.run.smithi094.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T16:48:56.766 INFO:teuthology.orchestra.run.smithi094.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T16:48:57.032 INFO:teuthology.orchestra.run.smithi094.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T16:48:57.033 INFO:teuthology.orchestra.run.smithi094.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T16:48:57.033 INFO:teuthology.orchestra.run.smithi094.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T16:48:57.033 INFO:teuthology.orchestra.run.smithi094.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T16:48:57.033 INFO:teuthology.orchestra.run.smithi094.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T16:48:57.033 INFO:teuthology.orchestra.run.smithi094.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T16:48:57.045 INFO:teuthology.orchestra.run.smithi094.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T16:48:57.045 INFO:teuthology.orchestra.run.smithi094.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T16:48:57.064 INFO:teuthology.orchestra.run.smithi138.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T16:48:57.067 DEBUG:teuthology.orchestra.run.smithi138:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T16:48:57.168 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T16:48:57.369 INFO:teuthology.orchestra.run.smithi138.stdout:Building dependency tree... 2024-04-25T16:48:57.370 INFO:teuthology.orchestra.run.smithi138.stdout:Reading state information... 2024-04-25T16:48:57.447 INFO:teuthology.orchestra.run.smithi094.stdout:Preconfiguring packages ... 2024-04-25T16:48:57.564 INFO:teuthology.orchestra.run.smithi138.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T16:48:57.564 INFO:teuthology.orchestra.run.smithi138.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T16:48:57.564 INFO:teuthology.orchestra.run.smithi138.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T16:48:57.564 INFO:teuthology.orchestra.run.smithi138.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T16:48:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T16:48:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T16:48:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T16:48:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T16:48:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T16:48:57.565 INFO:teuthology.orchestra.run.smithi138.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T16:48:57.565 INFO:teuthology.orchestra.run.smithi138.stdout:The following additional packages will be installed: 2024-04-25T16:48:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T16:48:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T16:48:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T16:48:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T16:48:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T16:48:57.567 INFO:teuthology.orchestra.run.smithi138.stdout:Suggested packages: 2024-04-25T16:48:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T16:48:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T16:48:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T16:48:57.608 INFO:teuthology.orchestra.run.smithi138.stdout:The following NEW packages will be installed: 2024-04-25T16:48:57.609 INFO:teuthology.orchestra.run.smithi138.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T16:48:57.609 INFO:teuthology.orchestra.run.smithi138.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T16:48:57.610 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 55.7 MB in 1s (53.8 MB/s) 2024-04-25T16:48:57.611 INFO:teuthology.orchestra.run.smithi138.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T16:48:57.611 INFO:teuthology.orchestra.run.smithi138.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T16:48:57.611 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T16:48:57.611 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T16:48:57.611 INFO:teuthology.orchestra.run.smithi138.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T16:48:57.611 INFO:teuthology.orchestra.run.smithi138.stdout:The following packages will be upgraded: 2024-04-25T16:48:57.612 INFO:teuthology.orchestra.run.smithi138.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T16:48:57.670 INFO:teuthology.orchestra.run.smithi138.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T16:48:57.670 INFO:teuthology.orchestra.run.smithi138.stdout:Need to get 55.7 MB of archives. 2024-04-25T16:48:57.670 INFO:teuthology.orchestra.run.smithi138.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T16:48:57.670 INFO:teuthology.orchestra.run.smithi138.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T16:48:57.832 INFO:teuthology.orchestra.run.smithi138.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T16:48:57.839 INFO:teuthology.orchestra.run.smithi138.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T16:48:57.878 INFO:teuthology.orchestra.run.smithi138.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T16:48:57.879 INFO:teuthology.orchestra.run.smithi138.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T16:48:57.880 INFO:teuthology.orchestra.run.smithi138.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T16:48:57.891 INFO:teuthology.orchestra.run.smithi138.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T16:48:57.892 INFO:teuthology.orchestra.run.smithi138.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T16:48:57.894 INFO:teuthology.orchestra.run.smithi138.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T16:48:57.895 INFO:teuthology.orchestra.run.smithi138.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T16:48:57.896 INFO:teuthology.orchestra.run.smithi138.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T16:48:57.897 INFO:teuthology.orchestra.run.smithi138.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T16:48:57.900 INFO:teuthology.orchestra.run.smithi138.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T16:48:57.925 INFO:teuthology.orchestra.run.smithi138.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T16:48:57.925 INFO:teuthology.orchestra.run.smithi138.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T16:48:57.927 INFO:teuthology.orchestra.run.smithi138.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T16:48:57.928 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-25T16:48:57.934 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T16:48:58.070 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T16:48:58.368 INFO:teuthology.orchestra.run.smithi138.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T16:48:58.370 INFO:teuthology.orchestra.run.smithi138.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T16:48:58.371 INFO:teuthology.orchestra.run.smithi138.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T16:48:58.375 INFO:teuthology.orchestra.run.smithi138.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T16:48:58.375 INFO:teuthology.orchestra.run.smithi138.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T16:48:58.381 INFO:teuthology.orchestra.run.smithi138.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T16:48:58.383 INFO:teuthology.orchestra.run.smithi138.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T16:48:58.394 INFO:teuthology.orchestra.run.smithi138.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T16:48:58.394 INFO:teuthology.orchestra.run.smithi138.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T16:48:58.424 INFO:teuthology.orchestra.run.smithi138.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T16:48:58.424 INFO:teuthology.orchestra.run.smithi138.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T16:48:58.435 INFO:teuthology.orchestra.run.smithi138.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T16:48:58.455 INFO:teuthology.orchestra.run.smithi138.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T16:48:58.461 INFO:teuthology.orchestra.run.smithi138.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T16:48:58.463 INFO:teuthology.orchestra.run.smithi138.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T16:48:58.466 INFO:teuthology.orchestra.run.smithi138.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T16:48:58.467 INFO:teuthology.orchestra.run.smithi138.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T16:48:58.489 INFO:teuthology.orchestra.run.smithi138.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T16:48:58.493 INFO:teuthology.orchestra.run.smithi138.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T16:48:58.759 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T16:48:58.821 INFO:teuthology.orchestra.run.smithi138.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T16:48:58.821 INFO:teuthology.orchestra.run.smithi138.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T16:48:58.821 INFO:teuthology.orchestra.run.smithi138.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T16:48:58.821 INFO:teuthology.orchestra.run.smithi138.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T16:48:58.821 INFO:teuthology.orchestra.run.smithi138.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T16:48:58.822 INFO:teuthology.orchestra.run.smithi138.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T16:48:58.838 INFO:teuthology.orchestra.run.smithi138.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T16:48:58.839 INFO:teuthology.orchestra.run.smithi138.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T16:48:59.246 INFO:teuthology.orchestra.run.smithi138.stdout:Preconfiguring packages ... 2024-04-25T16:48:59.393 INFO:teuthology.orchestra.run.smithi138.stdout:Fetched 55.7 MB in 1s (45.6 MB/s) 2024-04-25T16:48:59.653 INFO:teuthology.orchestra.run.smithi138.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-25T16:48:59.659 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T16:48:59.693 INFO:teuthology.orchestra.run.smithi094.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T16:48:59.806 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T16:48:59.967 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package flex. 2024-04-25T16:49:00.053 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-25T16:49:00.058 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T16:49:00.099 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T16:49:00.405 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T16:49:00.573 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T16:49:00.730 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T16:49:01.123 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T16:49:01.369 INFO:teuthology.orchestra.run.smithi138.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T16:49:01.623 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package flex. 2024-04-25T16:49:01.651 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-25T16:49:01.656 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T16:49:01.721 INFO:teuthology.orchestra.run.smithi138.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-25T16:49:01.727 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T16:49:01.762 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T16:49:01.772 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T16:49:02.179 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T16:49:02.264 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T16:49:02.310 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T16:49:02.589 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T16:49:02.646 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T16:49:03.009 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T16:49:03.090 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package bison. 2024-04-25T16:49:03.194 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-25T16:49:03.200 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T16:49:03.247 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T16:49:03.521 INFO:teuthology.orchestra.run.smithi138.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-25T16:49:03.526 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T16:49:03.639 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T16:49:03.657 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T16:49:03.668 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T16:49:03.706 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T16:49:04.042 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package dump. 2024-04-25T16:49:04.068 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T16:49:04.070 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T16:49:04.109 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T16:49:04.199 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T16:49:04.388 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package indent. 2024-04-25T16:49:04.416 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:49:04.455 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T16:49:04.552 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T16:49:04.707 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T16:49:04.723 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T16:49:04.757 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T16:49:04.925 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package bison. 2024-04-25T16:49:05.030 INFO:teuthology.orchestra.run.smithi138.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-25T16:49:05.036 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T16:49:05.086 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T16:49:05.110 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T16:49:05.138 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T16:49:05.177 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T16:49:05.438 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T16:49:05.454 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T16:49:05.496 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T16:49:05.511 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T16:49:05.539 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T16:49:05.578 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T16:49:05.808 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T16:49:05.835 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T16:49:05.866 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T16:49:05.897 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package dump. 2024-04-25T16:49:05.925 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T16:49:05.966 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T16:49:06.110 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T16:49:06.137 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T16:49:06.177 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T16:49:06.243 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package indent. 2024-04-25T16:49:06.259 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:49:06.301 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T16:49:06.413 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T16:49:06.440 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T16:49:06.479 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T16:49:06.562 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T16:49:06.590 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T16:49:06.628 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T16:49:06.799 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T16:49:06.827 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T16:49:06.865 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T16:49:06.925 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T16:49:06.952 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T16:49:06.991 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T16:49:07.252 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T16:49:07.268 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T16:49:07.310 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T16:49:08.131 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T16:49:08.159 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T16:49:08.197 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T16:49:08.426 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T16:49:08.441 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T16:49:08.475 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T16:49:08.644 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T16:49:08.660 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T16:49:08.694 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T16:49:08.728 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T16:49:08.756 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T16:49:08.794 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T16:49:08.938 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T16:49:08.966 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T16:49:09.005 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T16:49:09.106 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T16:49:09.134 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T16:49:09.172 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T16:49:09.332 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T16:49:09.360 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T16:49:09.398 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T16:49:09.735 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T16:49:09.762 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T16:49:09.801 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T16:49:10.105 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T16:49:10.132 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T16:49:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T16:49:10.499 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T16:49:10.526 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:49:10.565 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T16:49:10.909 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T16:49:10.910 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T16:49:10.936 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T16:49:10.937 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:49:10.975 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T16:49:10.976 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T16:49:11.211 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T16:49:11.239 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T16:49:11.262 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T16:49:11.278 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T16:49:11.290 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:49:11.320 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T16:49:11.556 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T16:49:11.572 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T16:49:11.614 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T16:49:11.631 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T16:49:11.659 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T16:49:11.697 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T16:49:12.042 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T16:49:12.070 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T16:49:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T16:49:12.108 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T16:49:12.129 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T16:49:12.168 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T16:49:12.403 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T16:49:12.431 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T16:49:12.469 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T16:49:12.530 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package modemmanager. 2024-04-25T16:49:12.557 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T16:49:12.596 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T16:49:12.823 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T16:49:12.850 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:49:12.889 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T16:49:13.227 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package network-manager. 2024-04-25T16:49:13.255 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T16:49:13.285 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T16:49:13.302 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T16:49:13.313 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:49:13.351 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T16:49:13.638 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T16:49:13.666 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:49:13.704 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T16:49:13.957 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T16:49:13.968 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ppp. 2024-04-25T16:49:13.973 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T16:49:13.996 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T16:49:14.015 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T16:49:14.273 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T16:49:14.573 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T16:49:14.590 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T16:49:14.627 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T16:49:14.631 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T16:49:14.655 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T16:49:14.693 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T16:49:14.884 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T16:49:14.900 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T16:49:14.934 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T16:49:15.013 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package modemmanager. 2024-04-25T16:49:15.041 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T16:49:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T16:49:15.187 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T16:49:15.214 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T16:49:15.253 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T16:49:15.574 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T16:49:15.601 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:49:15.640 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T16:49:15.728 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package network-manager. 2024-04-25T16:49:15.756 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T16:49:15.803 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T16:49:16.161 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T16:49:16.189 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T16:49:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T16:49:16.276 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-common (238) ... 2024-04-25T16:49:16.402 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ppp. 2024-04-25T16:49:16.430 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T16:49:16.688 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T16:49:16.743 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T16:49:16.771 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:49:16.820 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T16:49:17.007 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T16:49:17.035 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T16:49:17.074 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T16:49:17.326 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T16:49:17.342 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T16:49:17.376 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T16:49:17.671 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T16:49:17.698 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T16:49:17.737 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T16:49:18.037 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql. 2024-04-25T16:49:18.041 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T16:49:18.054 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T16:49:18.069 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:49:18.108 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T16:49:18.179 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql (14+238) ... 2024-04-25T16:49:18.564 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T16:49:18.592 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T16:49:18.782 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T16:49:18.855 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T16:49:18.883 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T16:49:18.941 INFO:teuthology.orchestra.run.smithi138.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T16:49:18.971 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql-common (238) ... 2024-04-25T16:49:19.025 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T16:49:19.042 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T16:49:19.086 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T16:49:19.361 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T16:49:19.389 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T16:49:19.419 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T16:49:19.471 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T16:49:19.499 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:49:19.541 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T16:49:19.706 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package tcl. 2024-04-25T16:49:19.734 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T16:49:19.777 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T16:49:20.100 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T16:49:20.128 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T16:49:20.166 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T16:49:20.514 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T16:49:20.540 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T16:49:20.578 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T16:49:20.766 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql. 2024-04-25T16:49:20.794 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T16:49:20.832 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql (14+238) ... 2024-04-25T16:49:20.957 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T16:49:20.985 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T16:49:21.031 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T16:49:21.059 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T16:49:21.076 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T16:49:21.117 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T16:49:21.310 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T16:49:21.336 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T16:49:21.353 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T16:49:21.386 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T16:49:21.654 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T16:49:21.705 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T16:49:21.734 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T16:49:21.772 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T16:49:21.805 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T16:49:21.931 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T16:49:22.042 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package tcl. 2024-04-25T16:49:22.057 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T16:49:22.070 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T16:49:22.113 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T16:49:22.182 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T16:49:22.308 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T16:49:22.478 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T16:49:22.506 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T16:49:22.544 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T16:49:22.785 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T16:49:22.881 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T16:49:22.890 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T16:49:22.918 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T16:49:22.957 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T16:49:23.293 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T16:49:23.299 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T16:49:23.299 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T16:49:23.321 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T16:49:23.368 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T16:49:23.656 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T16:49:23.891 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T16:49:24.033 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T16:49:24.159 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T16:49:24.230 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T16:49:24.285 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T16:49:24.359 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T16:49:24.411 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T16:49:24.468 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T16:49:24.537 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T16:49:24.569 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T16:49:24.670 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T16:49:24.821 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T16:49:24.947 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T16:49:25.026 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T16:49:25.073 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T16:49:25.093 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T16:49:25.458 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T16:49:25.458 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T16:49:25.720 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T16:49:25.821 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T16:49:25.915 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T16:49:25.946 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T16:49:25.972 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T16:49:26.047 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T16:49:26.098 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T16:49:26.173 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T16:49:26.224 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T16:49:26.299 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T16:49:26.350 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T16:49:26.425 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T16:49:26.476 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T16:49:26.567 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T16:49:26.567 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T16:49:26.628 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T16:49:26.694 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T16:49:26.737 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T16:49:26.819 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T16:49:26.899 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T16:49:27.025 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T16:49:27.151 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T16:49:27.319 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T16:49:27.420 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T16:49:27.441 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T16:49:27.567 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T16:49:27.662 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T16:49:27.736 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T16:49:27.862 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T16:49:27.904 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:27.904 INFO:teuthology.orchestra.run.smithi094.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T16:49:27.904 INFO:teuthology.orchestra.run.smithi094.stdout:which means they are currently configured by ifupdown: 2024-04-25T16:49:27.904 INFO:teuthology.orchestra.run.smithi094.stdout:- enp3s0f1 2024-04-25T16:49:27.904 INFO:teuthology.orchestra.run.smithi094.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T16:49:27.904 INFO:teuthology.orchestra.run.smithi094.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T16:49:27.904 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:27.979 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T16:49:28.015 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T16:49:28.082 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T16:49:28.206 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T16:49:28.298 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T16:49:28.352 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T16:49:28.366 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T16:49:28.687 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T16:49:28.901 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T16:49:29.069 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T16:49:29.195 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T16:49:29.321 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T16:49:29.495 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T16:49:29.514 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T16:49:29.640 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T16:49:29.758 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T16:49:29.885 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T16:49:30.011 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-common (238) ... 2024-04-25T16:49:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:30.128 INFO:teuthology.orchestra.run.smithi138.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T16:49:30.128 INFO:teuthology.orchestra.run.smithi138.stdout:which means they are currently configured by ifupdown: 2024-04-25T16:49:30.128 INFO:teuthology.orchestra.run.smithi138.stdout:- enp3s0f1 2024-04-25T16:49:30.128 INFO:teuthology.orchestra.run.smithi138.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T16:49:30.128 INFO:teuthology.orchestra.run.smithi138.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T16:49:30.128 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:30.258 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T16:49:30.589 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T16:49:30.866 INFO:teuthology.orchestra.run.smithi094.stdout:Adding user postgres to group ssl-cert 2024-04-25T16:49:30.895 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T16:49:31.091 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:31.092 INFO:teuthology.orchestra.run.smithi094.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T16:49:31.280 INFO:teuthology.orchestra.run.smithi094.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T16:49:31.280 INFO:teuthology.orchestra.run.smithi094.stdout:Removing obsolete dictionary files: 2024-04-25T16:49:31.401 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T16:49:31.536 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T16:49:31.662 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T16:49:31.779 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql-common (238) ... 2024-04-25T16:49:31.868 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T16:49:32.264 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T16:49:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T16:49:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T16:49:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T16:49:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:This user must also own the server process. 2024-04-25T16:49:32.650 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T16:49:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T16:49:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:The default text search configuration will be set to "english". 2024-04-25T16:49:32.651 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:Data page checksums are disabled. 2024-04-25T16:49:32.651 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T16:49:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:creating subdirectories ... ok 2024-04-25T16:49:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T16:49:32.666 INFO:teuthology.orchestra.run.smithi094.stdout:selecting default max_connections ... 100 2024-04-25T16:49:32.668 INFO:teuthology.orchestra.run.smithi138.stdout:Adding user postgres to group ssl-cert 2024-04-25T16:49:32.684 INFO:teuthology.orchestra.run.smithi094.stdout:selecting default shared_buffers ... 128MB 2024-04-25T16:49:32.694 INFO:teuthology.orchestra.run.smithi094.stdout:selecting default time zone ... Etc/UTC 2024-04-25T16:49:32.694 INFO:teuthology.orchestra.run.smithi094.stdout:creating configuration files ... ok 2024-04-25T16:49:32.804 INFO:teuthology.orchestra.run.smithi094.stdout:running bootstrap script ... ok 2024-04-25T16:49:32.889 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:32.889 INFO:teuthology.orchestra.run.smithi138.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T16:49:33.081 INFO:teuthology.orchestra.run.smithi138.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T16:49:33.081 INFO:teuthology.orchestra.run.smithi138.stdout:Removing obsolete dictionary files: 2024-04-25T16:49:33.445 INFO:teuthology.orchestra.run.smithi094.stdout:performing post-bootstrap initialization ... ok 2024-04-25T16:49:33.610 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T16:49:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T16:49:34.316 INFO:teuthology.orchestra.run.smithi138.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T16:49:34.341 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T16:49:34.342 INFO:teuthology.orchestra.run.smithi094.stdout:syncing data to disk ... ok 2024-04-25T16:49:34.403 INFO:teuthology.orchestra.run.smithi138.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T16:49:34.404 INFO:teuthology.orchestra.run.smithi138.stdout:This user must also own the server process. 2024-04-25T16:49:34.404 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:34.404 INFO:teuthology.orchestra.run.smithi138.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T16:49:34.404 INFO:teuthology.orchestra.run.smithi138.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T16:49:34.404 INFO:teuthology.orchestra.run.smithi138.stdout:The default text search configuration will be set to "english". 2024-04-25T16:49:34.404 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:34.404 INFO:teuthology.orchestra.run.smithi138.stdout:Data page checksums are disabled. 2024-04-25T16:49:34.404 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:34.404 INFO:teuthology.orchestra.run.smithi138.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T16:49:34.404 INFO:teuthology.orchestra.run.smithi138.stdout:creating subdirectories ... ok 2024-04-25T16:49:34.404 INFO:teuthology.orchestra.run.smithi138.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T16:49:34.421 INFO:teuthology.orchestra.run.smithi138.stdout:selecting default max_connections ... 100 2024-04-25T16:49:34.441 INFO:teuthology.orchestra.run.smithi138.stdout:selecting default shared_buffers ... 128MB 2024-04-25T16:49:34.451 INFO:teuthology.orchestra.run.smithi138.stdout:selecting default time zone ... Etc/UTC 2024-04-25T16:49:34.451 INFO:teuthology.orchestra.run.smithi138.stdout:creating configuration files ... ok 2024-04-25T16:49:34.566 INFO:teuthology.orchestra.run.smithi138.stdout:running bootstrap script ... ok 2024-04-25T16:49:34.721 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T16:49:35.229 INFO:teuthology.orchestra.run.smithi138.stdout:performing post-bootstrap initialization ... ok 2024-04-25T16:49:35.884 INFO:teuthology.orchestra.run.smithi138.stdout:syncing data to disk ... ok 2024-04-25T16:49:36.261 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T16:49:37.506 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T16:49:37.632 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T16:49:37.758 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql (14+238) ... 2024-04-25T16:49:37.885 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T16:49:38.011 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T16:49:38.267 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T16:49:38.267 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T16:49:38.830 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T16:49:38.931 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T16:49:38.979 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T16:49:39.105 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T16:49:39.234 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T16:49:39.239 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql (14+238) ... 2024-04-25T16:49:39.365 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T16:49:39.370 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T16:49:39.491 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T16:49:39.726 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T16:49:39.726 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T16:49:40.269 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T16:49:40.361 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T16:49:40.664 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T16:49:40.725 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T16:49:40.816 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T16:49:40.824 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T16:49:41.344 INFO:teuthology.orchestra.run.smithi094.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T16:49:41.350 DEBUG:teuthology.parallel:result is None 2024-04-25T16:49:42.177 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T16:49:42.260 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T16:49:42.753 INFO:teuthology.orchestra.run.smithi138.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T16:49:42.757 DEBUG:teuthology.parallel:result is None 2024-04-25T16:49:42.758 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:49:42.912 DEBUG:teuthology.orchestra.run.smithi094:> dpkg-query -W -f '${Version}' ceph 2024-04-25T16:49:42.932 INFO:teuthology.orchestra.run.smithi094.stdout:19.0.0-3291-g994e2f20-1jammy 2024-04-25T16:49:42.932 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T16:49:42.933 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291-g994e2f20-1jammy is installed. 2024-04-25T16:49:42.934 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:49:43.080 DEBUG:teuthology.orchestra.run.smithi138:> dpkg-query -W -f '${Version}' ceph 2024-04-25T16:49:43.099 INFO:teuthology.orchestra.run.smithi138.stdout:19.0.0-3291-g994e2f20-1jammy 2024-04-25T16:49:43.099 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T16:49:43.099 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291-g994e2f20-1jammy is installed. 2024-04-25T16:49:43.101 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T16:49:43.101 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:49:43.101 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T16:49:43.115 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:49:43.115 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T16:49:43.153 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T16:49:43.153 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:49:43.154 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T16:49:43.167 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T16:49:43.220 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:49:43.227 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T16:49:43.241 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T16:49:43.295 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T16:49:43.295 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:49:43.296 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T16:49:43.308 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T16:49:43.364 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:49:43.364 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T16:49:43.376 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T16:49:43.432 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T16:49:43.432 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:49:43.432 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T16:49:43.444 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T16:49:43.499 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:49:43.500 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T16:49:43.512 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T16:49:43.567 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T16:49:43.653 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T16:49:43.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /var/log/ceph 2024-04-25T16:49:43.656 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2024-04-25T16:49:43.668 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T16:49:43.668 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T16:49:43.710 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T16:49:43.723 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T16:49:43.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T16:49:43.762 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T16:49:43.782 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T16:49:43.783 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T16:49:43.783 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T16:49:43.783 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:49:43.783 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T16:49:43.821 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T16:49:43.898 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:49:43.898 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T16:49:43.904 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T16:49:43.984 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T16:49:43.984 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T16:49:43.984 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673280', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi094.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGu8NGEuGQkb1jV9Xb27i1LpCP0Jj07s6pidfWmZr3LyeUMosqY0oO8/gJWyAfPqsll1QwH4HI+dPvOXZsNhRbE=', 'smithi138.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBFEDbrXlp5aEZYUnjRaYzIiE2ZC7qbzwLmxJvka+WOTYpvlaj5Y/OswrFC2ZJZx2t7z6yFZqE062kaVp4nguaU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T16:49:43.985 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T16:49:43.987 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T16:49:44.025 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T16:49:44.034 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T16:49:44.080 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:49:44.080 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T16:49:44.093 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T16:49:44.093 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-04-25T16:49:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T16:49:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:49:44.141 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 881 Links: 1 2024-04-25T16:49:44.141 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:49:44.141 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 16:43:05.389502742 +0000 2024-04-25T16:49:44.141 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 16:43:05.117505170 +0000 2024-04-25T16:49:44.142 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 16:43:05.117505170 +0000 2024-04-25T16:49:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: - 2024-04-25T16:49:44.142 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T16:49:44.195 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T16:49:44.195 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T16:49:44.195 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000288394 s, 1.8 MB/s 2024-04-25T16:49:44.197 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T16:49:44.246 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-04-25T16:49:44.293 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T16:49:44.293 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:49:44.293 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 898 Links: 1 2024-04-25T16:49:44.293 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:49:44.293 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 16:43:05.881498354 +0000 2024-04-25T16:49:44.293 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 16:43:05.593500923 +0000 2024-04-25T16:49:44.293 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 16:43:05.593500923 +0000 2024-04-25T16:49:44.293 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: - 2024-04-25T16:49:44.294 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T16:49:44.347 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T16:49:44.347 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T16:49:44.347 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000321683 s, 1.6 MB/s 2024-04-25T16:49:44.348 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T16:49:44.398 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-04-25T16:49:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T16:49:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:49:44.445 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 917 Links: 1 2024-04-25T16:49:44.445 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:49:44.445 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 16:43:06.365494036 +0000 2024-04-25T16:49:44.445 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 16:43:06.117496249 +0000 2024-04-25T16:49:44.445 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 16:43:06.117496249 +0000 2024-04-25T16:49:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: - 2024-04-25T16:49:44.446 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T16:49:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T16:49:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T16:49:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000310281 s, 1.7 MB/s 2024-04-25T16:49:44.500 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T16:49:44.550 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-04-25T16:49:44.597 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T16:49:44.597 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:49:44.597 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 935 Links: 1 2024-04-25T16:49:44.597 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:49:44.597 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 16:43:06.837489825 +0000 2024-04-25T16:49:44.597 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 16:43:06.585492074 +0000 2024-04-25T16:49:44.597 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 16:43:06.585492074 +0000 2024-04-25T16:49:44.598 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: - 2024-04-25T16:49:44.598 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T16:49:44.651 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T16:49:44.651 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T16:49:44.651 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000243378 s, 2.1 MB/s 2024-04-25T16:49:44.652 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T16:49:44.706 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T16:49:44.707 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:49:44.707 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T16:49:44.712 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T16:49:44.712 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-04-25T16:49:44.761 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T16:49:44.761 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:49:44.761 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-04-25T16:49:44.761 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:49:44.761 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 16:43:05.401290532 +0000 2024-04-25T16:49:44.761 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 16:43:05.125316181 +0000 2024-04-25T16:49:44.761 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 16:43:05.125316181 +0000 2024-04-25T16:49:44.761 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-04-25T16:49:44.762 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T16:49:44.815 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T16:49:44.815 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T16:49:44.815 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000436247 s, 1.2 MB/s 2024-04-25T16:49:44.816 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T16:49:44.865 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-04-25T16:49:44.912 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T16:49:44.912 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:49:44.912 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-04-25T16:49:44.912 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:49:44.912 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 16:43:05.917242579 +0000 2024-04-25T16:49:44.912 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 16:43:05.641268228 +0000 2024-04-25T16:49:44.912 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 16:43:05.641268228 +0000 2024-04-25T16:49:44.912 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-04-25T16:49:44.913 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T16:49:44.965 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T16:49:44.965 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T16:49:44.966 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000341195 s, 1.5 MB/s 2024-04-25T16:49:44.967 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T16:49:45.013 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-04-25T16:49:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T16:49:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:49:45.060 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 915 Links: 1 2024-04-25T16:49:45.060 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:49:45.060 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 16:43:06.401198797 +0000 2024-04-25T16:49:45.060 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 16:43:06.165220021 +0000 2024-04-25T16:49:45.060 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 16:43:06.165220021 +0000 2024-04-25T16:49:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-04-25T16:49:45.061 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T16:49:45.114 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T16:49:45.114 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T16:49:45.114 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000333592 s, 1.5 MB/s 2024-04-25T16:49:45.115 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T16:49:45.165 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-04-25T16:49:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T16:49:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:49:45.212 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 940 Links: 1 2024-04-25T16:49:45.213 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:49:45.213 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 16:43:06.645176854 +0000 2024-04-25T16:49:45.213 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 16:43:06.641177214 +0000 2024-04-25T16:49:45.213 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 16:43:06.641177214 +0000 2024-04-25T16:49:45.213 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-04-25T16:49:45.213 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T16:49:45.266 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T16:49:45.266 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T16:49:45.266 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000284769 s, 1.8 MB/s 2024-04-25T16:49:45.267 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T16:49:45.317 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T16:49:45.318 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T16:49:45.318 INFO:tasks.ceph:Generating config... 2024-04-25T16:49:45.319 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T16:49:45.319 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T16:49:45.319 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T16:49:45.319 INFO:tasks.ceph:[client] pid file = 2024-04-25T16:49:45.319 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T16:49:45.319 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T16:49:45.319 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T16:49:45.319 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T16:49:45.319 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T16:49:45.320 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T16:49:45.321 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T16:49:45.321 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T16:49:45.321 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T16:49:45.321 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T16:49:45.321 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T16:49:45.321 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T16:49:45.321 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T16:49:45.321 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T16:49:45.321 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T16:49:45.321 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T16:49:45.321 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T16:49:45.321 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T16:49:45.321 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T16:49:45.321 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T16:49:45.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T16:49:45.387 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T16:49:45.402 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T16:49:45.469 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T16:49:45.479 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.94'), ('mon.b', '172.21.15.138'), ('mon.c', '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]')] 2024-04-25T16:49:45.480 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.94,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T16:49:45.481 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:49:45.481 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T16:49:45.525 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.94 --add b 172.21.15.138 --addv c '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:49:45.638 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:49:45.639 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: generated fsid cc26689a-ed7a-405e-9abc-b7fa2c1906cf 2024-04-25T16:49:45.639 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = quincy 2024-04-25T16:49:45.639 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 0 2024-04-25T16:49:45.639 INFO:teuthology.orchestra.run.smithi094.stdout:fsid cc26689a-ed7a-405e-9abc-b7fa2c1906cf 2024-04-25T16:49:45.639 INFO:teuthology.orchestra.run.smithi094.stdout:last_changed 2024-04-25T16:49:45.643473+0000 2024-04-25T16:49:45.639 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-25T16:49:45.643473+0000 2024-04-25T16:49:45.639 INFO:teuthology.orchestra.run.smithi094.stdout:min_mon_release 17 (quincy) 2024-04-25T16:49:45.639 INFO:teuthology.orchestra.run.smithi094.stdout:election_strategy: 1 2024-04-25T16:49:45.639 INFO:teuthology.orchestra.run.smithi094.stdout:0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T16:49:45.639 INFO:teuthology.orchestra.run.smithi094.stdout:1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-04-25T16:49:45.639 INFO:teuthology.orchestra.run.smithi094.stdout:2: [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] mon.c 2024-04-25T16:49:45.639 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T16:49:45.644 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T16:49:45.650 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID cc26689a-ed7a-405e-9abc-b7fa2c1906cf... 2024-04-25T16:49:45.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T16:49:45.694 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T16:49:45.718 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-04-25T16:49:45.718 INFO:teuthology.orchestra.run.smithi094.stdout: chdir = "" 2024-04-25T16:49:45.718 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: auth supported = cephx 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: filestore xattr use omap = true 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: mon clock drift allowed = 1.000 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: osd crush chooseleaf type = 0 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: auth debug = true 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on old message = true 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on bug = true 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: mon pg warn max object skew = 0 2024-04-25T16:49:45.719 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default size = 2 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow primary affinity = true 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow pg remap = true 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on legacy crush tunables = false 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on crush straw calc version zero = false 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on no sortbitwise = false 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on osd down out interval zero = false 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on too few osds = false 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T16:49:45.720 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: mon_allow_pool_size_one = true 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: osd default data pool replay window = 5 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: mon allow pool delete = true 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: mon cluster log file level = debug 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: debug asserts on shutdown = true 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: mon health detail to clog = false 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: mon host = "172.21.15.94,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = cc26689a-ed7a-405e-9abc-b7fa2c1906cf 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.721 INFO:teuthology.orchestra.run.smithi094.stdout:[osd] 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: osd journal size = 100 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub load threshold = 5.0 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub max interval = 600 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: osd mclock profile = high_recovery_ops 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: osd recover clone overlap = true 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: osd recovery max chunk = 1048576 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug shutdown = true 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug op order = true 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug verify stray on activate = true 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug trim objects = true 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.722 INFO:teuthology.orchestra.run.smithi094.stdout: osd open classes on start = true 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug pg log writeout = true 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: osd deep scrub update digest min age = 30 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: osd map max advance = 10 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: journal zero on create = true 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: filestore ondisk finisher threads = 3 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: filestore apply finisher threads = 3 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: bdev debug aio = true 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug misdirected ops = true 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore block size = 96636764160 2024-04-25T16:49:45.723 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore fsck on mount = True 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluefs = 20 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluestore = 20 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: debug osd = 20 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: debug rocksdb = 10 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd full ratio = 0.9 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd nearfull ratio = 0.8 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: osd failsafe full ratio = 0.95 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: osd objectstore = bluestore 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: osd op complaint time = 180 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr] 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: debug mgr = 20 2024-04-25T16:49:45.724 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: mgr/telemetry/nag = false 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout:[mon] 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: debug paxos = 20 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: mon data avail warn = 5 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: mon mgr mkfs grace = 240 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-04-25T16:49:45.725 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd reporter subtree level = osd 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd prime pg temp = true 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: # 1m isn't quite enough 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: mon_down_mkfs_grace = 2m 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.726 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_filestore_osds = false 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: mon down mkfs grace = 300 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: mon op complaint time = 120 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout:[client] 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: rgw cache enabled = true 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable ops log = true 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable usage log = true 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: client mount timeout = 600 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = "" 2024-04-25T16:49:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: rados mon op timeout = 900 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout: rados osd op timeout = 900 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.a] 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.b] 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.c] 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout:[mds] 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout: debug mds = 20 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout: debug mds balancer = 20 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout: mds debug frag = True 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout: mds debug scatterstat = True 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout: mds op complaint time = 180 2024-04-25T16:49:45.728 INFO:teuthology.orchestra.run.smithi094.stdout: mds verify scatter = True 2024-04-25T16:49:45.729 INFO:teuthology.orchestra.run.smithi094.stdout: osd op complaint time = 180 2024-04-25T16:49:45.729 INFO:teuthology.orchestra.run.smithi094.stdout: rados mon op timeout = 900 2024-04-25T16:49:45.729 INFO:teuthology.orchestra.run.smithi094.stdout: rados osd op timeout = 900 2024-04-25T16:49:45.729 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-04-25T16:49:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-04-25T16:49:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T16:49:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-04-25T16:49:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-04-25T16:49:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-04-25T16:49:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-04-25T16:49:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-04-25T16:49:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-04-25T16:49:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-04-25T16:49:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-04-25T16:49:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-04-25T16:49:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.94,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = cc26689a-ed7a-405e-9abc-b7fa2c1906cf 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-04-25T16:49:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug trim objects = true 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-04-25T16:49:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 20 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 20 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 10 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-04-25T16:49:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-04-25T16:49:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: mon down mkfs grace = 300 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: mon op complaint time = 120 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-04-25T16:49:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: client mount timeout = 600 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = "" 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-04-25T16:49:45.739 INFO:teuthology.orchestra.run.smithi138.stdout:[mds] 2024-04-25T16:49:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds = 20 2024-04-25T16:49:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds balancer = 20 2024-04-25T16:49:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T16:49:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug frag = True 2024-04-25T16:49:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug scatterstat = True 2024-04-25T16:49:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: mds op complaint time = 180 2024-04-25T16:49:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: mds verify scatter = True 2024-04-25T16:49:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-04-25T16:49:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-04-25T16:49:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-04-25T16:49:45.741 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T16:49:45.742 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T16:49:45.863 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T16:49:45.864 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:49:45.864 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T16:49:45.869 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:49:45.869 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T16:49:45.917 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:49:45.917 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:49:45.917 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T16:49:45.917 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T16:49:45.981 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:49:45.981 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:49:46.029 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:49:46.029 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:49:46.029 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T16:49:46.030 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T16:49:46.053 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:49:46.054 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:49:46.101 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T16:49:46.102 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T16:49:46.102 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T16:49:46.175 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T16:49:46.182 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T16:49:46.256 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T16:49:46.272 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T16:49:46.273 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T16:49:46.342 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T16:49:46.349 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T16:49:46.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T16:49:46.468 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T16:49:46.474 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T16:49:46.486 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T16:49:46.561 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T16:49:46.567 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T16:49:46.580 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T16:49:46.691 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T16:49:46.697 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T16:49:46.709 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T16:49:46.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T16:49:46.776 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T16:49:46.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T16:49:46.854 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T16:49:46.868 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T16:49:46.935 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T16:49:46.949 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T16:49:47.009 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T16:49:47.022 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T16:49:47.022 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T16:49:47.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T16:49:47.036 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T16:49:47.036 INFO:tasks.ceph:role: osd.0 2024-04-25T16:49:47.036 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:49:47.036 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T16:49:47.097 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:49:47.097 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:49:47.097 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:49:47.097 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T16:49:47.097 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:49:47.097 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:49:47.097 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:49:47.097 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T16:49:47.097 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:49:47.097 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:49:47.120 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-04-25T16:49:47.122 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-04-25T16:49:47.122 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T16:49:47.179 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T16:49:47.229 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T16:49:47.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:49:47.230 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T16:49:47.284 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T16:49:47.284 INFO:tasks.ceph:role: osd.1 2024-04-25T16:49:47.285 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:49:47.285 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T16:49:47.340 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:49:47.341 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:49:47.341 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:49:47.341 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T16:49:47.341 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:49:47.341 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:49:47.341 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:49:47.341 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T16:49:47.341 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:49:47.341 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:49:47.364 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-04-25T16:49:47.366 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-04-25T16:49:47.366 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T16:49:47.424 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T16:49:47.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:49:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T16:49:47.479 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T16:49:47.530 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T16:49:47.531 INFO:tasks.ceph:role: osd.2 2024-04-25T16:49:47.531 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:49:47.531 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T16:49:47.585 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:49:47.585 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:49:47.585 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:49:47.585 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T16:49:47.585 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:49:47.585 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:49:47.585 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:49:47.585 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T16:49:47.586 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:49:47.586 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:49:47.608 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-04-25T16:49:47.610 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-04-25T16:49:47.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T16:49:47.667 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T16:49:47.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:49:47.717 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T16:49:47.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T16:49:47.772 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T16:49:47.772 INFO:tasks.ceph:role: osd.3 2024-04-25T16:49:47.772 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:49:47.772 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T16:49:47.828 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:49:47.829 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:49:47.829 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:49:47.829 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T16:49:47.829 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:49:47.829 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:49:47.829 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:49:47.829 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T16:49:47.829 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:49:47.829 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:49:47.852 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-04-25T16:49:47.854 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-04-25T16:49:47.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T16:49:47.910 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T16:49:47.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:49:47.962 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T16:49:47.963 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:49:48.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:48.113+0000 7fe5099377c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T16:49:48.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:48.113+0000 7fe5099377c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T16:49:48.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:48.113+0000 7fe5099377c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T16:49:48.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:48.113+0000 7fe5099377c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T16:49:48.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:48.113+0000 7fe5099377c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T16:49:48.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:48.113+0000 7fe5099377c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T16:49:51.096 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T16:49:51.110 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:49:51.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:51.224+0000 7f71230d97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T16:49:51.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:51.224+0000 7f71230d97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T16:49:51.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:51.224+0000 7f71230d97c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T16:49:51.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:51.224+0000 7f71230d97c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T16:49:51.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:51.224+0000 7f71230d97c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T16:49:51.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:51.224+0000 7f71230d97c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T16:49:54.189 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T16:49:54.202 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:49:54.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:54.320+0000 7f98aba197c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T16:49:54.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:54.320+0000 7f98aba197c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T16:49:54.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:54.320+0000 7f98aba197c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T16:49:54.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:54.320+0000 7f98aba197c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T16:49:54.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:54.320+0000 7f98aba197c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T16:49:54.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:54.320+0000 7f98aba197c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T16:49:57.304 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T16:49:57.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:49:57.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:57.432+0000 7f84f66607c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T16:49:57.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:57.432+0000 7f84f66607c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T16:49:57.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:57.432+0000 7f84f66607c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T16:49:57.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:57.432+0000 7f84f66607c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T16:49:57.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:57.432+0000 7f84f66607c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T16:49:57.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:49:57.436+0000 7f84f66607c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T16:50:00.416 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T16:50:00.429 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T16:50:00.446 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T16:50:00.446 INFO:tasks.ceph:role: osd.4 2024-04-25T16:50:00.446 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:50:00.446 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T16:50:00.507 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:50:00.507 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:50:00.507 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:50:00.508 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T16:50:00.508 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:50:00.508 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:50:00.508 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:50:00.508 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T16:50:00.508 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:50:00.508 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:50:00.992 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-25T16:50:00.994 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-25T16:50:00.994 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T16:50:01.145 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T16:50:01.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:50:01.157 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T16:50:01.157 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T16:50:01.211 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T16:50:01.211 INFO:tasks.ceph:role: osd.5 2024-04-25T16:50:01.211 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:50:01.212 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T16:50:01.276 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:50:01.276 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:50:01.276 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:50:01.276 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T16:50:01.276 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:50:01.276 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:50:01.276 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:50:01.276 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T16:50:01.276 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:50:01.277 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:50:01.723 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-25T16:50:01.725 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-25T16:50:01.725 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T16:50:01.807 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T16:50:01.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:50:01.861 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T16:50:01.861 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T16:50:01.915 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T16:50:01.915 INFO:tasks.ceph:role: osd.6 2024-04-25T16:50:01.915 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:50:01.915 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T16:50:01.984 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:50:01.984 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:50:01.984 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:50:01.984 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T16:50:01.984 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:50:01.984 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:50:01.985 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:50:01.985 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T16:50:01.985 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:50:01.985 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:50:02.436 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-25T16:50:02.438 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-25T16:50:02.438 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T16:50:02.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T16:50:02.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:50:02.566 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T16:50:02.566 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T16:50:02.618 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T16:50:02.619 INFO:tasks.ceph:role: osd.7 2024-04-25T16:50:02.619 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:50:02.619 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T16:50:02.679 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T16:50:02.679 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T16:50:02.679 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T16:50:02.679 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T16:50:02.679 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T16:50:02.680 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-25T16:50:02.680 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T16:50:02.680 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T16:50:02.680 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T16:50:02.680 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T16:50:03.160 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-25T16:50:03.163 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-25T16:50:03.163 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T16:50:03.297 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T16:50:03.307 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T16:50:03.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:50:03.308 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:50:03.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:03.460+0000 7f5dacf3f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T16:50:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:03.464+0000 7f5dacf3f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T16:50:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:03.464+0000 7f5dacf3f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T16:50:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:03.464+0000 7f5dacf3f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T16:50:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:03.464+0000 7f5dacf3f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T16:50:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:03.464+0000 7f5dacf3f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T16:50:06.468 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T16:50:06.482 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:50:06.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:06.596+0000 7f36ba5ec7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T16:50:06.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:06.596+0000 7f36ba5ec7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T16:50:06.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:06.596+0000 7f36ba5ec7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T16:50:06.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:06.596+0000 7f36ba5ec7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T16:50:06.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:06.596+0000 7f36ba5ec7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T16:50:06.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:06.596+0000 7f36ba5ec7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T16:50:09.591 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T16:50:09.604 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:50:09.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:09.716+0000 7fa05a7197c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T16:50:09.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:09.716+0000 7fa05a7197c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T16:50:09.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:09.716+0000 7fa05a7197c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T16:50:09.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:09.716+0000 7fa05a7197c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T16:50:09.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:09.716+0000 7fa05a7197c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T16:50:09.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:09.716+0000 7fa05a7197c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T16:50:12.716 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T16:50:12.730 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:50:12.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:12.844+0000 7fab424b77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T16:50:12.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:12.844+0000 7fab424b77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T16:50:12.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:12.844+0000 7fab424b77c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T16:50:12.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:12.844+0000 7fab424b77c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T16:50:12.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:12.844+0000 7fab424b77c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T16:50:12.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:12.844+0000 7fab424b77c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T16:50:13.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:50:13.990 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:50:15.840 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T16:50:15.853 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T16:50:15.854 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:50:15.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T16:50:15.868 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:50:15.868 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T16:50:15.925 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:50:15.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T16:50:15.980 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:50:15.980 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T16:50:16.036 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:50:16.036 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T16:50:16.091 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:50:16.091 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T16:50:16.145 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:50:16.145 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T16:50:16.200 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:50:16.200 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T16:50:16.213 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:50:16.213 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T16:50:16.269 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:50:16.269 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T16:50:16.323 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:50:16.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T16:50:16.379 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:50:16.379 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T16:50:16.435 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:50:16.435 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T16:50:16.491 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:50:16.492 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T16:50:16.547 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:50:16.547 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T16:50:16.553 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:50:16.553 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T16:50:16.600 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:50:16.600 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T16:50:16.606 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:50:16.606 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T16:50:16.652 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T16:50:16.652 DEBUG:teuthology.orchestra.run.smithi094:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T16:50:16.655 DEBUG:teuthology.orchestra.run.smithi138:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T16:50:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.y] 2024-04-25T16:50:16.697 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCqiSpmkuy3ChAAK079xLd0VAzJ2fHWGSqAKg== 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCqiSpmgKayFBAAU7YU3CXQPeY2aOZRmZpT/w== 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCqiSpmC24sHBAAdDAXtY+87Lt/TawZKwTdcA== 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.0] 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCsiSpmgSfGBhAAD6ugAvKS2vtU9ufyA8T2UQ== 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.1] 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCviSpmEXKZDRAA2/cTL5Irr+vAXiKKpYzxDg== 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.2] 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCyiSpm3IxSExAAzs5LaP8CwRLgPVAps2ccvQ== 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.3] 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC1iSpmukcIGhAAFmLhwf4KHPJJvNYzSop6jA== 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.x] 2024-04-25T16:50:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCqiSpmo1esDxAAxzoc9ioXsQxGPWDgEh4Jgg== 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCqiSpmRpnTIRAAiPFPCHSxVWe9q0F0deXgaA== 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCqiSpmyy+aKRAARdfTN3rKu+E6bDvZZbD47g== 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.4] 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC7iSpm0RWhGxAA4KKPB45Iod7zoVw6IW4bJQ== 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.5] 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC+iSpmetnNIxAAF+1t+rPK23w/VvsRnJgyyw== 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.6] 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDBiSpmyEXoKhAAx5EDhq1pz+2EeD/dWUParg== 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.7] 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDEiSpmDeSDMhAArpZmLFT/VyYTzp/gq6v87w== 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCqiSpmzcOULhAAGKJvQYLzCNAQecQi22z2GQ== 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:[client.2] 2024-04-25T16:50:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCqiSpmm3c+MxAAJUovvS+EqAog+i2eUBeh9Q== 2024-04-25T16:50:16.700 INFO:teuthology.orchestra.run.smithi094.stdout:[client.1] 2024-04-25T16:50:16.700 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCqiSpmjfIfOBAAs0gA62roRxxeirvZV36x+g== 2024-04-25T16:50:16.700 INFO:teuthology.orchestra.run.smithi094.stdout:[client.3] 2024-04-25T16:50:16.700 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCriSpmefDuABAA7Uo70NTIVW+wZYj15VbmEg== 2024-04-25T16:50:16.706 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.y] 2024-04-25T16:50:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCqiSpmkuy3ChAAK079xLd0VAzJ2fHWGSqAKg== 2024-04-25T16:50:16.706 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-04-25T16:50:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCqiSpmgKayFBAAU7YU3CXQPeY2aOZRmZpT/w== 2024-04-25T16:50:16.706 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-04-25T16:50:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCqiSpmC24sHBAAdDAXtY+87Lt/TawZKwTdcA== 2024-04-25T16:50:16.706 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.0] 2024-04-25T16:50:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCsiSpmgSfGBhAAD6ugAvKS2vtU9ufyA8T2UQ== 2024-04-25T16:50:16.706 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.1] 2024-04-25T16:50:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCviSpmEXKZDRAA2/cTL5Irr+vAXiKKpYzxDg== 2024-04-25T16:50:16.706 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.2] 2024-04-25T16:50:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCyiSpm3IxSExAAzs5LaP8CwRLgPVAps2ccvQ== 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.3] 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC1iSpmukcIGhAAFmLhwf4KHPJJvNYzSop6jA== 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.x] 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCqiSpmo1esDxAAxzoc9ioXsQxGPWDgEh4Jgg== 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCqiSpmRpnTIRAAiPFPCHSxVWe9q0F0deXgaA== 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCqiSpmyy+aKRAARdfTN3rKu+E6bDvZZbD47g== 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.4] 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC7iSpm0RWhGxAA4KKPB45Iod7zoVw6IW4bJQ== 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.5] 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC+iSpmetnNIxAAF+1t+rPK23w/VvsRnJgyyw== 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.6] 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDBiSpmyEXoKhAAx5EDhq1pz+2EeD/dWUParg== 2024-04-25T16:50:16.707 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.7] 2024-04-25T16:50:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDEiSpmDeSDMhAArpZmLFT/VyYTzp/gq6v87w== 2024-04-25T16:50:16.708 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-04-25T16:50:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCqiSpmzcOULhAAGKJvQYLzCNAQecQi22z2GQ== 2024-04-25T16:50:16.708 INFO:teuthology.orchestra.run.smithi138.stdout:[client.2] 2024-04-25T16:50:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCqiSpmm3c+MxAAJUovvS+EqAog+i2eUBeh9Q== 2024-04-25T16:50:16.708 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2024-04-25T16:50:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCqiSpmjfIfOBAAs0gA62roRxxeirvZV36x+g== 2024-04-25T16:50:16.708 INFO:teuthology.orchestra.run.smithi138.stdout:[client.3] 2024-04-25T16:50:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCriSpmefDuABAA7Uo70NTIVW+wZYj15VbmEg== 2024-04-25T16:50:16.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T16:50:16.746 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T16:50:16.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:50:16.834 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:50:16.908 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:50:16.910 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:50:16.983 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:16.985 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.056 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.059 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.130 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.199 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.201 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.269 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T16:50:17.272 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T16:50:17.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:50:17.340 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:50:17.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:50:17.413 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T16:50:17.480 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.482 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.551 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.619 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.689 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T16:50:17.752 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:50:17.754 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:50:17.819 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:50:17.821 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:50:17.887 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:50:17.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:50:17.954 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:50:17.956 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T16:50:18.027 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T16:50:18.027 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T16:50:18.036 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T16:50:18.149 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T16:50:18.159 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T16:50:18.171 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T16:50:18.293 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T16:50:18.302 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T16:50:18.352 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T16:50:18.471 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T16:50:18.481 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:50:18.483 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T16:50:18.526 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T16:50:18.527 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T16:50:18.527 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T16:50:18.529 INFO:tasks.ceph.mon.a:Started 2024-04-25T16:50:18.530 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T16:50:18.530 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T16:50:18.568 INFO:tasks.ceph.mon.b:Started 2024-04-25T16:50:18.569 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T16:50:18.569 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T16:50:18.571 INFO:tasks.ceph.mon.c:Started 2024-04-25T16:50:18.571 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T16:50:18.571 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T16:50:18.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T16:50:18.574 INFO:tasks.ceph.mgr.y:Started 2024-04-25T16:50:18.574 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T16:50:18.575 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T16:50:18.576 INFO:tasks.ceph.mgr.x:Started 2024-04-25T16:50:18.577 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T16:50:18.577 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T16:50:18.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.720+0000 7f9943bcc640 1 Processor -- start 2024-04-25T16:50:18.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.720+0000 7f9943bcc640 1 -- start start 2024-04-25T16:50:18.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.720+0000 7f9943bcc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:18.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.720+0000 7f9943bcc640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f993c0b1110 con 0x7f993c0b6b80 2024-04-25T16:50:18.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.720+0000 7f9943bcc640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f993c0b13f0 con 0x7f993c0b0bf0 2024-04-25T16:50:18.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.720+0000 7f9943bcc640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f993c0b1d20 con 0x7f993c0b07e0 2024-04-25T16:50:18.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.720+0000 7f9942169640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f993c0b6b80 0x7f993c0d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49956/0 (socket says 172.21.15.94:49956) 2024-04-25T16:50:18.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.720+0000 7f9941968640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:18.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.720+0000 7f9942169640 1 -- 172.21.15.94:0/3535421645 learned_addr learned my addr 172.21.15.94:0/3535421645 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:50:18.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.720+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f993c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:18.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.720+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f993c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:18.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.720+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:18.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.720+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:50:18.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.924+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:18.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.924+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f993c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:18.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.924+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f993c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:18.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.924+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:18.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:18.924+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:50:19.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:19.328+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:19.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:19.328+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f993c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:19.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:19.328+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f993c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:19.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:19.328+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:19.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:19.328+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T16:50:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:20.132+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:20.132+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f993c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:20.132+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f993c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:20.132+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:20.132+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T16:50:21.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f993c0702c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:50:21.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f99337fe640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f993c0702c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 >> v1:172.21.15.138:6789/0 conn(0x7f993c0b0bf0 legacy=0x7f993c0b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:21.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 >> v1:172.21.15.94:6789/0 conn(0x7f993c0b6b80 legacy=0x7f993c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:21.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f99337fe640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f99337fe640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f993c0b1d20 con 0x7f99200036d0 2024-04-25T16:50:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9920008dd0 con 0x7f99200061e0 2024-04-25T16:50:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 msgr2=0x7f9920005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T16:50:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 msgr2=0x7f9920005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:50:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 msgr2=0x7f99200086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 msgr2=0x7f99200086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.720+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:50:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.924+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.924+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.924+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 msgr2=0x7f99200086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.924+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 msgr2=0x7f99200086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.924+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 msgr2=0x7f9920005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T16:50:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.924+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.924+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 msgr2=0x7f9920005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.924+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:21.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.924+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:50:21.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:21.924+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:50:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:22.328+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:22.328+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:22.328+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 msgr2=0x7f99200086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T16:50:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:22.328+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 msgr2=0x7f99200086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:22.328+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:22.328+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 msgr2=0x7f9920005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:22.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:22.328+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 msgr2=0x7f9920005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:22.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:22.328+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:22.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:22.328+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T16:50:22.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:22.328+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T16:50:23.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:23.132+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:23.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:23.132+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:23.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:23.132+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 msgr2=0x7f9920005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T16:50:23.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:23.132+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 msgr2=0x7f9920005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:23.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:23.132+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:23.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:23.132+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 msgr2=0x7f99200086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:23.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:23.132+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 msgr2=0x7f99200086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:23.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:23.132+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T16:50:23.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:23.132+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:23.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:23.132+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T16:50:24.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 msgr2=0x7f99200086b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f99337fe640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99200061e0 0x7f99200086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 msgr2=0x7f9920005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f99337fe640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f99200036d0 0x7f9920005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f9942169640 1 -- 172.21.15.94:0/3535421645 reap_dead start 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f99337fe640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f99337fe640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f993c0b16d0 con 0x7f9920006be0 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f993c0b1d20 con 0x7f993c0b07e0 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9920008dd0 con 0x7f9920003b90 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f992000b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f992000b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:24.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:50:24.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 msgr2=0x7f99200066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:24.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 msgr2=0x7f99200066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:24.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:24.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.720+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:50:24.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.924+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:24.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.924+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:24.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.924+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f992000b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:24.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.924+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f992000b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:24.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.924+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:24.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.924+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:50:24.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.924+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 msgr2=0x7f99200066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T16:50:24.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.924+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 msgr2=0x7f99200066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:24.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.924+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:24.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:24.924+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:50:25.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:25.328+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:25.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:25.328+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:25.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:25.328+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f992000b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:25.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:25.328+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f992000b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:25.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:25.328+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:25.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:25.328+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T16:50:25.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:25.328+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 msgr2=0x7f99200066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T16:50:25.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:25.328+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 msgr2=0x7f99200066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:25.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:25.328+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:25.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:25.328+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T16:50:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:26.132+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:26.132+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:26.132+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f992000b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:26.132+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f992000b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:26.132+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:26.132+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T16:50:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:26.132+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 msgr2=0x7f99200066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T16:50:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:26.132+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 msgr2=0x7f99200066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:26.132+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:26.132+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T16:50:27.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 msgr2=0x7f99200066a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:50:27.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.720+0000 7f99337fe640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9920003b90 0x7f99200066a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:27.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 msgr2=0x7f992000b2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:50:27.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.720+0000 7f99337fe640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.94:3300/0 conn(0x7f993c0b07e0 0x7f992000b2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:27.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 >> v1:172.21.15.94:6789/0 conn(0x7f9920006be0 legacy=0x7f9920006ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:27.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.720+0000 7f99337fe640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 0x7f9920013c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:27.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99200089a0 con 0x7f992000d920 2024-04-25T16:50:27.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.720+0000 7f99337fe640 1 -- 172.21.15.94:0/3535421645 --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f993c0b1d20 con 0x7f9920011860 2024-04-25T16:50:27.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.720+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 0x7f9920013c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:27.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.720+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 msgr2=0x7f9920013c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:27.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.720+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 msgr2=0x7f9920013c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:27.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.720+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 0x7f9920013c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:27.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.720+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 0x7f9920013c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:50:27.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.924+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 0x7f9920013c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:27.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.924+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 msgr2=0x7f9920013c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:27.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.924+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 msgr2=0x7f9920013c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:27.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.924+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 0x7f9920013c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:27.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:27.924+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 0x7f9920013c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:50:28.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.328+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 0x7f9920013c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:28.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.328+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 msgr2=0x7f9920013c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T16:50:28.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.328+0000 7f9941167640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 msgr2=0x7f9920013c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T16:50:28.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.328+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 0x7f9920013c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:50:28.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.328+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 0x7f9920013c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T16:50:28.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 -- 172.21.15.94:0/3535421645 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2698955211 0 0) 0x7f99200089a0 con 0x7f992000d920 2024-04-25T16:50:28.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 -- 172.21.15.94:0/3535421645 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9928003530 con 0x7f992000d920 2024-04-25T16:50:28.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 -- 172.21.15.94:0/3535421645 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2845092142 0 0) 0x7f9928003530 con 0x7f992000d920 2024-04-25T16:50:28.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 -- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 msgr2=0x7f9920013c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:50:28.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 --2- 172.21.15.94:0/3535421645 >> v2:172.21.15.138:3300/0 conn(0x7f9920011860 0x7f9920013c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:28.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 -- 172.21.15.94:0/3535421645 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9920015200 con 0x7f992000d920 2024-04-25T16:50:28.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 -- 172.21.15.94:0/3535421645 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f9938017600 con 0x7f992000d920 2024-04-25T16:50:28.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 -- 172.21.15.94:0/3535421645 >> v1:172.21.15.138:6789/0 conn(0x7f992000d920 legacy=0x7f992000dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:28.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9942169640 1 -- 172.21.15.94:0/3535421645 reap_dead start 2024-04-25T16:50:28.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280039c0 0x7f9928003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:28.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9928004810 0x7f9928004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:28.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928005360 0x7f992800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:28.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 -- 172.21.15.94:0/3535421645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f993c0b1d20 con 0x7f9928005360 2024-04-25T16:50:28.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 -- 172.21.15.94:0/3535421645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f992800c290 con 0x7f99280039c0 2024-04-25T16:50:28.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9940966640 1 -- 172.21.15.94:0/3535421645 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f992800c490 con 0x7f9928004810 2024-04-25T16:50:28.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280039c0 0x7f9928003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:28.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9942169640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928005360 0x7f992800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:28.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9941167640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9928004810 0x7f9928004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:28.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9928004810 msgr2=0x7f9928004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:28.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9928004810 0x7f9928004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:28.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928005360 msgr2=0x7f992800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:28.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928005360 0x7f992800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:28.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9941968640 1 -- 172.21.15.94:0/3535421645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9928003530 con 0x7f99280039c0 2024-04-25T16:50:28.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.984+0000 7f9942169640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928005360 0x7f992800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:28.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.992+0000 7f9941968640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280039c0 0x7f9928003e10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9938018720 tx=0x7f9938007210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a8f4deb424dce89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:28.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.992+0000 7f9940966640 1 -- 172.21.15.94:0/3535421645 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99380b6070 con 0x7f99280039c0 2024-04-25T16:50:28.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.992+0000 7f9940966640 1 -- 172.21.15.94:0/3535421645 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99380b1600 con 0x7f99280039c0 2024-04-25T16:50:28.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.992+0000 7f9940966640 1 -- 172.21.15.94:0/3535421645 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938018450 con 0x7f99280039c0 2024-04-25T16:50:28.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.992+0000 7f9943bcc640 1 -- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280039c0 msgr2=0x7f9928003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:28.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.992+0000 7f9943bcc640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280039c0 0x7f9928003e10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9938018720 tx=0x7f9938007210 comp rx=0 tx=0).stop 2024-04-25T16:50:28.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.992+0000 7f9943bcc640 1 -- 172.21.15.94:0/3535421645 shutdown_connections 2024-04-25T16:50:28.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.992+0000 7f9943bcc640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928005360 0x7f992800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:28.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.992+0000 7f9943bcc640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9928004810 0x7f9928004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:28.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.992+0000 7f9943bcc640 1 --2- 172.21.15.94:0/3535421645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280039c0 0x7f9928003e10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:28.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.992+0000 7f9943bcc640 1 -- 172.21.15.94:0/3535421645 >> 172.21.15.94:0/3535421645 conn(0x7f993c071f00 msgr2=0x7f993c073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:28.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.992+0000 7f9943bcc640 1 -- 172.21.15.94:0/3535421645 shutdown_connections 2024-04-25T16:50:28.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:28.996+0000 7f9943bcc640 1 -- 172.21.15.94:0/3535421645 wait complete. 2024-04-25T16:50:28.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.000+0000 7f9943bcc640 1 Processor -- start 2024-04-25T16:50:28.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.000+0000 7f9943bcc640 1 -- start start 2024-04-25T16:50:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9943bcc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280039c0 0x7f993c1f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9943bcc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928004810 0x7f993c13a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9943bcc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9928005360 0x7f993c13dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9943bcc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f993c0b65f0 con 0x7f9928004810 2024-04-25T16:50:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9943bcc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f993c0b6470 con 0x7f99280039c0 2024-04-25T16:50:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9943bcc640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f993c0b6770 con 0x7f9928005360 2024-04-25T16:50:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9941167640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928004810 0x7f993c13a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9941167640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928004810 0x7f993c13a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48070/0 (socket says 172.21.15.94:48070) 2024-04-25T16:50:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9941167640 1 -- 172.21.15.94:0/2219203679 learned_addr learned my addr 172.21.15.94:0/2219203679 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9941968640 1 --2- 172.21.15.94:0/2219203679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280039c0 0x7f993c1f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9942169640 1 --2- 172.21.15.94:0/2219203679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9928005360 0x7f993c13dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9941167640 1 -- 172.21.15.94:0/2219203679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9928005360 msgr2=0x7f993c13dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9941167640 1 --2- 172.21.15.94:0/2219203679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9928005360 0x7f993c13dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9941167640 1 -- 172.21.15.94:0/2219203679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280039c0 msgr2=0x7f993c1f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9941167640 1 --2- 172.21.15.94:0/2219203679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280039c0 0x7f993c1f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9941167640 1 -- 172.21.15.94:0/2219203679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c13e110 con 0x7f9928004810 2024-04-25T16:50:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9941167640 1 --2- 172.21.15.94:0/2219203679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928004810 0x7f993c13a830 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f992c005630 tx=0x7f992c017af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f99327fc640 1 -- 172.21.15.94:0/2219203679 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c001650 con 0x7f9928004810 2024-04-25T16:50:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f99327fc640 1 -- 172.21.15.94:0/2219203679 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f992c0017f0 con 0x7f9928004810 2024-04-25T16:50:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9943bcc640 1 -- 172.21.15.94:0/2219203679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993c13cb30 con 0x7f9928004810 2024-04-25T16:50:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f99327fc640 1 -- 172.21.15.94:0/2219203679 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c001be0 con 0x7f9928004810 2024-04-25T16:50:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9943bcc640 1 -- 172.21.15.94:0/2219203679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993c13d0f0 con 0x7f9928004810 2024-04-25T16:50:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f99327fc640 1 -- 172.21.15.94:0/2219203679 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f992c013040 con 0x7f9928004810 2024-04-25T16:50:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f99327fc640 1 -- 172.21.15.94:0/2219203679 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f992c021500 con 0x7f9928004810 2024-04-25T16:50:29.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.004+0000 7f9943bcc640 1 -- 172.21.15.94:0/2219203679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f993c06a730 con 0x7f9928004810 2024-04-25T16:50:29.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.016+0000 7f99327fc640 1 -- 172.21.15.94:0/2219203679 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f992c01e070 con 0x7f9928004810 2024-04-25T16:50:29.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.080+0000 7f9943bcc640 1 -- 172.21.15.94:0/2219203679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f993c0b03e0 con 0x7f9928004810 2024-04-25T16:50:29.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.084+0000 7f99327fc640 1 -- 172.21.15.94:0/2219203679 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f992c021700 con 0x7f9928004810 2024-04-25T16:50:29.080 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-04-25T16:50:29.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.084+0000 7f9943bcc640 1 -- 172.21.15.94:0/2219203679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928004810 msgr2=0x7f993c13a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:29.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.084+0000 7f9943bcc640 1 --2- 172.21.15.94:0/2219203679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928004810 0x7f993c13a830 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f992c005630 tx=0x7f992c017af0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.084+0000 7f9943bcc640 1 -- 172.21.15.94:0/2219203679 shutdown_connections 2024-04-25T16:50:29.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.084+0000 7f9943bcc640 1 --2- 172.21.15.94:0/2219203679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9928005360 0x7f993c13dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.084+0000 7f9943bcc640 1 --2- 172.21.15.94:0/2219203679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928004810 0x7f993c13a830 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.084+0000 7f9943bcc640 1 --2- 172.21.15.94:0/2219203679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280039c0 0x7f993c1f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.084+0000 7f9943bcc640 1 -- 172.21.15.94:0/2219203679 >> 172.21.15.94:0/2219203679 conn(0x7f993c071f00 msgr2=0x7f993c0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:29.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.084+0000 7f9943bcc640 1 -- 172.21.15.94:0/2219203679 shutdown_connections 2024-04-25T16:50:29.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:29.084+0000 7f9943bcc640 1 -- 172.21.15.94:0/2219203679 wait complete. 2024-04-25T16:50:29.101 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T16:50:29.101 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T16:50:29.101 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:50:29.102 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T16:50:29.114 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:50:29.114 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T16:50:29.166 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:50:29.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T16:50:29.220 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:50:29.220 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T16:50:29.278 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:50:29.278 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T16:50:29.291 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:50:29.292 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T16:50:29.307 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:29.308+0000 7ff5e660f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T16:50:29.348 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:50:29.348 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T16:50:29.388 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:29.387+0000 7f4e3b4e8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T16:50:29.399 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:50:29.399 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T16:50:29.451 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 394e2fce-26fe-473e-930b-ce703137119c 0 2024-04-25T16:50:29.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efffed2c640 1 Processor -- start 2024-04-25T16:50:29.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efffed2c640 1 -- start start 2024-04-25T16:50:29.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efffed2c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efff80d3f70 0x7efff80d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:29.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efffed2c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efff8164510 con 0x7efff80d3450 2024-04-25T16:50:29.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efffed2c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efff81647b0 con 0x7efff80d1970 2024-04-25T16:50:29.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efffed2c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efff8164a90 con 0x7efff80d3f70 2024-04-25T16:50:29.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efffd2c9640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7efff80d1970 0x7efff80d1d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47576/0 (socket says 172.21.15.138:47576) 2024-04-25T16:50:29.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efff7fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efff80d3f70 0x7efff80d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:29.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efffd2c9640 1 -- 172.21.15.138:0/2009189844 learned_addr learned my addr 172.21.15.138:0/2009189844 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-25T16:50:29.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efff77fe640 1 -- 172.21.15.138:0/2009189844 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3480537922 0 0) 0x7efff81647b0 con 0x7efff80d1970 2024-04-25T16:50:29.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efff77fe640 1 -- 172.21.15.138:0/2009189844 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effe0003530 con 0x7efff80d1970 2024-04-25T16:50:29.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efff7fff640 1 -- 172.21.15.138:0/2009189844 >> v1:172.21.15.138:6789/0 conn(0x7efff80d1970 legacy=0x7efff80d1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:29.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efff7fff640 1 -- 172.21.15.138:0/2009189844 >> v1:172.21.15.94:6789/0 conn(0x7efff80d3450 legacy=0x7efff80d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:29.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.643+0000 7efff7fff640 1 -- 172.21.15.138:0/2009189844 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff80d4b60 con 0x7efff80d3f70 2024-04-25T16:50:29.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.647+0000 7efff7fff640 1 --2- 172.21.15.138:0/2009189844 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efff80d3f70 0x7efff80d1430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efff8164c10 tx=0x7effec01c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a51189b1b1140e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:29.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.647+0000 7efff77fe640 1 -- 172.21.15.138:0/2009189844 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0b0070 con 0x7efff80d3f70 2024-04-25T16:50:29.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.647+0000 7efff77fe640 1 -- 172.21.15.138:0/2009189844 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effec0ab600 con 0x7efff80d3f70 2024-04-25T16:50:29.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.647+0000 7efff77fe640 1 -- 172.21.15.138:0/2009189844 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0ab9d0 con 0x7efff80d3f70 2024-04-25T16:50:29.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.647+0000 7efffed2c640 1 -- 172.21.15.138:0/2009189844 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efff80d3f70 msgr2=0x7efff80d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:29.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.647+0000 7efffed2c640 1 --2- 172.21.15.138:0/2009189844 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efff80d3f70 0x7efff80d1430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efff8164c10 tx=0x7effec01c430 comp rx=0 tx=0).stop 2024-04-25T16:50:29.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.647+0000 7efffed2c640 1 -- 172.21.15.138:0/2009189844 shutdown_connections 2024-04-25T16:50:29.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.647+0000 7efffed2c640 1 --2- 172.21.15.138:0/2009189844 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efff80d3f70 0x7efff80d1430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.647+0000 7efffed2c640 1 -- 172.21.15.138:0/2009189844 >> 172.21.15.138:0/2009189844 conn(0x7efff809e8d0 msgr2=0x7efff80b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:29.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.647+0000 7efffed2c640 1 -- 172.21.15.138:0/2009189844 shutdown_connections 2024-04-25T16:50:29.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.647+0000 7efffed2c640 1 -- 172.21.15.138:0/2009189844 wait complete. 2024-04-25T16:50:29.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffed2c640 1 Processor -- start 2024-04-25T16:50:29.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffed2c640 1 -- start start 2024-04-25T16:50:29.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffed2c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efff80d1970 0x7efff815a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:29.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffed2c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efff80d3450 0x7efff815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:29.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffed2c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff80d3f70 0x7efff80c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:29.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffed2c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efff816e850 con 0x7efff80d3450 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffed2c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efff816e6d0 con 0x7efff80d3f70 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffed2c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efff816e9d0 con 0x7efff80d1970 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffcac8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efff80d1970 0x7efff815a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffd2c9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff80d3f70 0x7efff80c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffd2c9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff80d3f70 0x7efff80c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44830/0 (socket says 172.21.15.138:44830) 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffd2c9640 1 -- 172.21.15.138:0/56190056 learned_addr learned my addr 172.21.15.138:0/56190056 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffcac8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efff80d1970 0x7efff815a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:47380/0 (socket says 172.21.15.138:47380) 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efff7fff640 1 --2- 172.21.15.138:0/56190056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efff80d3450 0x7efff815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffd2c9640 1 -- 172.21.15.138:0/56190056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efff80d1970 msgr2=0x7efff815a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffd2c9640 1 --2- 172.21.15.138:0/56190056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efff80d1970 0x7efff815a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffd2c9640 1 -- 172.21.15.138:0/56190056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efff80d3450 msgr2=0x7efff815a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffd2c9640 1 --2- 172.21.15.138:0/56190056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efff80d3450 0x7efff815a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffd2c9640 1 -- 172.21.15.138:0/56190056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff815e020 con 0x7efff80d3f70 2024-04-25T16:50:29.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffcac8640 1 --2- 172.21.15.138:0/56190056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efff80d1970 0x7efff815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:29.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffd2c9640 1 --2- 172.21.15.138:0/56190056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff80d3f70 0x7efff80c9d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efff000cad0 tx=0x7efff000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:29.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efff57fa640 1 -- 172.21.15.138:0/56190056 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0017070 con 0x7efff80d3f70 2024-04-25T16:50:29.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efff57fa640 1 -- 172.21.15.138:0/56190056 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff000ac30 con 0x7efff80d3f70 2024-04-25T16:50:29.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efff57fa640 1 -- 172.21.15.138:0/56190056 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0012400 con 0x7efff80d3f70 2024-04-25T16:50:29.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffed2c640 1 -- 172.21.15.138:0/56190056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff815e300 con 0x7efff80d3f70 2024-04-25T16:50:29.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.651+0000 7efffed2c640 1 -- 172.21.15.138:0/56190056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff815e8c0 con 0x7efff80d3f70 2024-04-25T16:50:29.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.655+0000 7efffed2c640 1 -- 172.21.15.138:0/56190056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff8076830 con 0x7efff80d3f70 2024-04-25T16:50:29.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.655+0000 7efff57fa640 1 -- 172.21.15.138:0/56190056 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efff001a070 con 0x7efff80d3f70 2024-04-25T16:50:29.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.655+0000 7efff57fa640 1 -- 172.21.15.138:0/56190056 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7efff001cbc0 con 0x7efff80d3f70 2024-04-25T16:50:29.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.655+0000 7efff57fa640 1 -- 172.21.15.138:0/56190056 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7efff00125a0 con 0x7efff80d3f70 2024-04-25T16:50:29.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.719+0000 7efffed2c640 1 -- 172.21.15.138:0/56190056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "394e2fce-26fe-473e-930b-ce703137119c", "id": 0} v 0) -- 0x7efff80d3970 con 0x7efff80d3f70 2024-04-25T16:50:29.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.723+0000 7efff57fa640 1 -- 172.21.15.138:0/56190056 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "394e2fce-26fe-473e-930b-ce703137119c", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7efff0012780 con 0x7efff80d3f70 2024-04-25T16:50:29.723 INFO:teuthology.orchestra.run.smithi138.stdout:0 2024-04-25T16:50:29.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.723+0000 7efffed2c640 1 -- 172.21.15.138:0/56190056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff80d3f70 msgr2=0x7efff80c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:29.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.727+0000 7efffed2c640 1 --2- 172.21.15.138:0/56190056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff80d3f70 0x7efff80c9d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efff000cad0 tx=0x7efff000cf90 comp rx=0 tx=0).stop 2024-04-25T16:50:29.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.727+0000 7efffed2c640 1 -- 172.21.15.138:0/56190056 shutdown_connections 2024-04-25T16:50:29.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.727+0000 7efffed2c640 1 --2- 172.21.15.138:0/56190056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff80d3f70 0x7efff80c9d00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.727+0000 7efffed2c640 1 --2- 172.21.15.138:0/56190056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efff80d3450 0x7efff815a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.727+0000 7efffed2c640 1 --2- 172.21.15.138:0/56190056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efff80d1970 0x7efff815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.727+0000 7efffed2c640 1 -- 172.21.15.138:0/56190056 >> 172.21.15.138:0/56190056 conn(0x7efff809e8d0 msgr2=0x7efff80d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:29.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.727+0000 7efffed2c640 1 -- 172.21.15.138:0/56190056 shutdown_connections 2024-04-25T16:50:29.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.727+0000 7efffed2c640 1 -- 172.21.15.138:0/56190056 wait complete. 2024-04-25T16:50:29.739 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:29.740+0000 7ff5e660f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T16:50:29.741 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 47b97498-86d9-4503-897a-96333028e6b7 1 2024-04-25T16:50:29.862 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:29.863+0000 7f4e3b4e8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T16:50:29.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.887+0000 7f02f67eb640 1 Processor -- start 2024-04-25T16:50:29.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.887+0000 7f02f67eb640 1 -- start start 2024-04-25T16:50:29.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f67eb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02f0160d00 0x7f02f00b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:29.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f67eb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f02f006dea0 0x7f02f006e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:29.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f67eb640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f02f01636f0 0x7f02f0163b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:29.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f67eb640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f02f01604d0 con 0x7f02f006dea0 2024-04-25T16:50:29.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f67eb640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f02f0161110 con 0x7f02f01636f0 2024-04-25T16:50:29.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f67eb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f02f00b5d60 con 0x7f02f0160d00 2024-04-25T16:50:29.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f4d88640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f02f01636f0 0x7f02f0163b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:29.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02effff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02f0160d00 0x7f02f00b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:29.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02effff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02f0160d00 0x7f02f00b4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:47388/0 (socket says 172.21.15.138:47388) 2024-04-25T16:50:29.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f4d88640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f02f01636f0 0x7f02f0163b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44838/0 (socket says 172.21.15.138:44838) 2024-04-25T16:50:29.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02effff640 1 -- 172.21.15.138:0/2853488724 learned_addr learned my addr 172.21.15.138:0/2853488724 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:50:29.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f4d88640 1 -- 172.21.15.138:0/2853488724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02f0160d00 msgr2=0x7f02f00b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:29.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f4d88640 1 --2- 172.21.15.138:0/2853488724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02f0160d00 0x7f02f00b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02ef7fe640 1 --2- 172.21.15.138:0/2853488724 >> v2:172.21.15.94:3300/0 conn(0x7f02f006dea0 0x7f02f006e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:29.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f4d88640 1 -- 172.21.15.138:0/2853488724 >> v2:172.21.15.94:3300/0 conn(0x7f02f006dea0 msgr2=0x7f02f006e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:29.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f4d88640 1 --2- 172.21.15.138:0/2853488724 >> v2:172.21.15.94:3300/0 conn(0x7f02f006dea0 0x7f02f006e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f4d88640 1 -- 172.21.15.138:0/2853488724 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f006e7f0 con 0x7f02f01636f0 2024-04-25T16:50:29.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02effff640 1 --2- 172.21.15.138:0/2853488724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02f0160d00 0x7f02f00b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:29.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.891+0000 7f02f4d88640 1 --2- 172.21.15.138:0/2853488724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f01636f0 0x7f02f0163b00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02f0071200 tx=0x7f02e001c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcc1f93c129f73ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:29.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.895+0000 7f02eeffd640 1 -- 172.21.15.138:0/2853488724 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e00b1070 con 0x7f02f01636f0 2024-04-25T16:50:29.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.895+0000 7f02eeffd640 1 -- 172.21.15.138:0/2853488724 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e00ac410 con 0x7f02f01636f0 2024-04-25T16:50:29.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.895+0000 7f02eeffd640 1 -- 172.21.15.138:0/2853488724 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e00ac7e0 con 0x7f02f01636f0 2024-04-25T16:50:29.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.895+0000 7f02f67eb640 1 -- 172.21.15.138:0/2853488724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f01636f0 msgr2=0x7f02f0163b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:29.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.895+0000 7f02f67eb640 1 --2- 172.21.15.138:0/2853488724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f01636f0 0x7f02f0163b00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02f0071200 tx=0x7f02e001c3f0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.895+0000 7f02f67eb640 1 -- 172.21.15.138:0/2853488724 shutdown_connections 2024-04-25T16:50:29.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.895+0000 7f02f67eb640 1 --2- 172.21.15.138:0/2853488724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f01636f0 0x7f02f0163b00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.895+0000 7f02f67eb640 1 --2- 172.21.15.138:0/2853488724 >> v2:172.21.15.94:3300/0 conn(0x7f02f006dea0 0x7f02f006e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.895+0000 7f02f67eb640 1 --2- 172.21.15.138:0/2853488724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02f0160d00 0x7f02f00b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.895+0000 7f02f67eb640 1 -- 172.21.15.138:0/2853488724 >> 172.21.15.138:0/2853488724 conn(0x7f02f0072fd0 msgr2=0x7f02f00733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.895+0000 7f02f67eb640 1 -- 172.21.15.138:0/2853488724 shutdown_connections 2024-04-25T16:50:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.895+0000 7f02f67eb640 1 -- 172.21.15.138:0/2853488724 wait complete. 2024-04-25T16:50:29.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02f67eb640 1 Processor -- start 2024-04-25T16:50:29.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02f67eb640 1 -- start start 2024-04-25T16:50:29.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02f67eb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f006dea0 0x7f02f014d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02f67eb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02f0160d00 0x7f02f0148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02f67eb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02f01636f0 0x7f02f00cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02f67eb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f02f00b6c80 con 0x7f02f0160d00 2024-04-25T16:50:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02f67eb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02f0164450 con 0x7f02f006dea0 2024-04-25T16:50:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02f67eb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f02f00b61f0 con 0x7f02f01636f0 2024-04-25T16:50:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02effff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f006dea0 0x7f02f014d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02effff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f006dea0 0x7f02f014d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44846/0 (socket says 172.21.15.138:44846) 2024-04-25T16:50:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02effff640 1 -- 172.21.15.138:0/2505476905 learned_addr learned my addr 172.21.15.138:0/2505476905 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:50:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02f4d88640 1 --2- 172.21.15.138:0/2505476905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02f01636f0 0x7f02f00cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02ef7fe640 1 --2- 172.21.15.138:0/2505476905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02f0160d00 0x7f02f0148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02effff640 1 -- 172.21.15.138:0/2505476905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02f01636f0 msgr2=0x7f02f00cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02effff640 1 --2- 172.21.15.138:0/2505476905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02f01636f0 0x7f02f00cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02effff640 1 -- 172.21.15.138:0/2505476905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02f0160d00 msgr2=0x7f02f0148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:29.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02effff640 1 --2- 172.21.15.138:0/2505476905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02f0160d00 0x7f02f0148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02effff640 1 -- 172.21.15.138:0/2505476905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f0148e50 con 0x7f02f006dea0 2024-04-25T16:50:29.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02f4d88640 1 --2- 172.21.15.138:0/2505476905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02f01636f0 0x7f02f00cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:29.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02effff640 1 --2- 172.21.15.138:0/2505476905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f006dea0 0x7f02f014d380 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f02e800efc0 tx=0x7f02e800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:29.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02ecff9640 1 -- 172.21.15.138:0/2505476905 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e800cda0 con 0x7f02f006dea0 2024-04-25T16:50:29.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02ecff9640 1 -- 172.21.15.138:0/2505476905 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e8019070 con 0x7f02f006dea0 2024-04-25T16:50:29.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.899+0000 7f02ecff9640 1 -- 172.21.15.138:0/2505476905 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e80146a0 con 0x7f02f006dea0 2024-04-25T16:50:29.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.903+0000 7f02f67eb640 1 -- 172.21.15.138:0/2505476905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f00ce450 con 0x7f02f006dea0 2024-04-25T16:50:29.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.903+0000 7f02f67eb640 1 -- 172.21.15.138:0/2505476905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f00cec60 con 0x7f02f006dea0 2024-04-25T16:50:29.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.903+0000 7f02ecff9640 1 -- 172.21.15.138:0/2505476905 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f02e8012050 con 0x7f02f006dea0 2024-04-25T16:50:29.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.903+0000 7f02ecff9640 1 -- 172.21.15.138:0/2505476905 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f02e80073f0 con 0x7f02f006dea0 2024-04-25T16:50:29.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.903+0000 7f02f67eb640 1 -- 172.21.15.138:0/2505476905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f006a730 con 0x7f02f006dea0 2024-04-25T16:50:29.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.903+0000 7f02ecff9640 1 -- 172.21.15.138:0/2505476905 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f02e8010040 con 0x7f02f006dea0 2024-04-25T16:50:29.935 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:29.936+0000 7ff5e660f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T16:50:29.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.971+0000 7f02f67eb640 1 -- 172.21.15.138:0/2505476905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "47b97498-86d9-4503-897a-96333028e6b7", "id": 1} v 0) -- 0x7f02f00d1820 con 0x7f02f006dea0 2024-04-25T16:50:29.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.983+0000 7f02ecff9640 1 -- 172.21.15.138:0/2505476905 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "47b97498-86d9-4503-897a-96333028e6b7", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f02e8014db0 con 0x7f02f006dea0 2024-04-25T16:50:29.983 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-04-25T16:50:29.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.987+0000 7f02f67eb640 1 -- 172.21.15.138:0/2505476905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f006dea0 msgr2=0x7f02f014d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:29.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.987+0000 7f02f67eb640 1 --2- 172.21.15.138:0/2505476905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f006dea0 0x7f02f014d380 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f02e800efc0 tx=0x7f02e800c490 comp rx=0 tx=0).stop 2024-04-25T16:50:29.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.987+0000 7f02f67eb640 1 -- 172.21.15.138:0/2505476905 shutdown_connections 2024-04-25T16:50:29.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.987+0000 7f02f67eb640 1 --2- 172.21.15.138:0/2505476905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02f01636f0 0x7f02f00cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.987+0000 7f02f67eb640 1 --2- 172.21.15.138:0/2505476905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02f0160d00 0x7f02f0148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.987+0000 7f02f67eb640 1 --2- 172.21.15.138:0/2505476905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f006dea0 0x7f02f014d380 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:29.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.987+0000 7f02f67eb640 1 -- 172.21.15.138:0/2505476905 >> 172.21.15.138:0/2505476905 conn(0x7f02f0072fd0 msgr2=0x7f02f00733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:29.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.987+0000 7f02f67eb640 1 -- 172.21.15.138:0/2505476905 shutdown_connections 2024-04-25T16:50:29.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:29.987+0000 7f02f67eb640 1 -- 172.21.15.138:0/2505476905 wait complete. 2024-04-25T16:50:30.002 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 606ece18-5501-48bd-b735-45a4c435c03a 2 2024-04-25T16:50:30.061 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:30.063+0000 7f4e3b4e8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T16:50:30.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.147+0000 7fd0f6277640 1 Processor -- start 2024-04-25T16:50:30.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0f6277640 1 -- start start 2024-04-25T16:50:30.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0f6277640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd0f00b5050 0x7fd0f00b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0f6277640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd0f006f6a0 0x7fd0f006fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0f6277640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f00b5cf0 con 0x7fd0f00b6b90 2024-04-25T16:50:30.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0f6277640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f00b5fd0 con 0x7fd0f00b5050 2024-04-25T16:50:30.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0f6277640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f00b62b0 con 0x7fd0f006f6a0 2024-04-25T16:50:30.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0f4814640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd0f006f6a0 0x7fd0f006fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0f4814640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd0f006f6a0 0x7fd0f006fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44860/0 (socket says 172.21.15.138:44860) 2024-04-25T16:50:30.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0f4814640 1 -- 172.21.15.138:0/1305576913 learned_addr learned my addr 172.21.15.138:0/1305576913 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:50:30.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0ef7fe640 1 --2- 172.21.15.138:0/1305576913 >> v2:172.21.15.94:3300/0 conn(0x7fd0f00b5050 0x7fd0f00b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0f4814640 1 -- 172.21.15.138:0/1305576913 >> v2:172.21.15.94:3300/0 conn(0x7fd0f00b5050 msgr2=0x7fd0f00b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:30.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0f4814640 1 --2- 172.21.15.138:0/1305576913 >> v2:172.21.15.94:3300/0 conn(0x7fd0f00b5050 0x7fd0f00b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0f4814640 1 -- 172.21.15.138:0/1305576913 >> v1:172.21.15.94:6789/0 conn(0x7fd0f00b6b90 legacy=0x7fd0f00b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.151+0000 7fd0f4814640 1 -- 172.21.15.138:0/1305576913 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f006f180 con 0x7fd0f006f6a0 2024-04-25T16:50:30.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0f4814640 1 --2- 172.21.15.138:0/1305576913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f006f6a0 0x7fd0f006fab0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0071410 tx=0x7fd0e801eb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c396edaec544590e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:30.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0eeffd640 1 -- 172.21.15.138:0/1305576913 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e80ad070 con 0x7fd0f006f6a0 2024-04-25T16:50:30.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0eeffd640 1 -- 172.21.15.138:0/1305576913 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e80a7ac0 con 0x7fd0f006f6a0 2024-04-25T16:50:30.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0eeffd640 1 -- 172.21.15.138:0/1305576913 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e80b1400 con 0x7fd0f006f6a0 2024-04-25T16:50:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0f6277640 1 -- 172.21.15.138:0/1305576913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f006f6a0 msgr2=0x7fd0f006fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0f6277640 1 --2- 172.21.15.138:0/1305576913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f006f6a0 0x7fd0f006fab0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0071410 tx=0x7fd0e801eb60 comp rx=0 tx=0).stop 2024-04-25T16:50:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0f6277640 1 -- 172.21.15.138:0/1305576913 shutdown_connections 2024-04-25T16:50:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0f6277640 1 --2- 172.21.15.138:0/1305576913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f006f6a0 0x7fd0f006fab0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0f6277640 1 --2- 172.21.15.138:0/1305576913 >> v2:172.21.15.94:3300/0 conn(0x7fd0f00b5050 0x7fd0f00b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0f6277640 1 -- 172.21.15.138:0/1305576913 >> 172.21.15.138:0/1305576913 conn(0x7fd0f0073570 msgr2=0x7fd0f0073980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0f6277640 1 -- 172.21.15.138:0/1305576913 shutdown_connections 2024-04-25T16:50:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0f6277640 1 -- 172.21.15.138:0/1305576913 wait complete. 2024-04-25T16:50:30.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0f6277640 1 Processor -- start 2024-04-25T16:50:30.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.155+0000 7fd0f6277640 1 -- start start 2024-04-25T16:50:30.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0f6277640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0f006f6a0 0x7fd0f0135c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0f6277640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f00b5050 0x7fd0f0158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0f6277640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0f00b6b90 0x7fd0f0140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0f6277640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f0168fa0 con 0x7fd0f00b6b90 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0f6277640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f0168e20 con 0x7fd0f00b5050 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0f6277640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f0169120 con 0x7fd0f006f6a0 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0effff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0f006f6a0 0x7fd0f0135c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0ef7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f00b5050 0x7fd0f0158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0effff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0f006f6a0 0x7fd0f0135c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:47396/0 (socket says 172.21.15.138:47396) 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0ef7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f00b5050 0x7fd0f0158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44874/0 (socket says 172.21.15.138:44874) 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0effff640 1 -- 172.21.15.138:0/660734312 learned_addr learned my addr 172.21.15.138:0/660734312 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0f4814640 1 --2- 172.21.15.138:0/660734312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0f00b6b90 0x7fd0f0140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0effff640 1 -- 172.21.15.138:0/660734312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f00b5050 msgr2=0x7fd0f0158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0effff640 1 --2- 172.21.15.138:0/660734312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f00b5050 0x7fd0f0158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0effff640 1 -- 172.21.15.138:0/660734312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0f00b6b90 msgr2=0x7fd0f0140ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0effff640 1 --2- 172.21.15.138:0/660734312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0f00b6b90 0x7fd0f0140ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0effff640 1 -- 172.21.15.138:0/660734312 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f0142b00 con 0x7fd0f006f6a0 2024-04-25T16:50:30.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0ef7fe640 1 --2- 172.21.15.138:0/660734312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f00b5050 0x7fd0f0158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:30.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0effff640 1 --2- 172.21.15.138:0/660734312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0f006f6a0 0x7fd0f0135c50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000c3c0 tx=0x7fd0e000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:30.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0ecff9640 1 -- 172.21.15.138:0/660734312 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0017070 con 0x7fd0f006f6a0 2024-04-25T16:50:30.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0ecff9640 1 -- 172.21.15.138:0/660734312 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e00052d0 con 0x7fd0f006f6a0 2024-04-25T16:50:30.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0f6277640 1 -- 172.21.15.138:0/660734312 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f0142de0 con 0x7fd0f006f6a0 2024-04-25T16:50:30.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0ecff9640 1 -- 172.21.15.138:0/660734312 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0003820 con 0x7fd0f006f6a0 2024-04-25T16:50:30.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0f6277640 1 -- 172.21.15.138:0/660734312 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f01433a0 con 0x7fd0f006f6a0 2024-04-25T16:50:30.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0ecff9640 1 -- 172.21.15.138:0/660734312 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd0e001a070 con 0x7fd0f006f6a0 2024-04-25T16:50:30.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0f6277640 1 -- 172.21.15.138:0/660734312 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f0076830 con 0x7fd0f006f6a0 2024-04-25T16:50:30.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.159+0000 7fd0ecff9640 1 -- 172.21.15.138:0/660734312 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fd0e0012420 con 0x7fd0f006f6a0 2024-04-25T16:50:30.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.163+0000 7fd0ecff9640 1 -- 172.21.15.138:0/660734312 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd0e0021070 con 0x7fd0f006f6a0 2024-04-25T16:50:30.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.231+0000 7fd0f6277640 1 -- 172.21.15.138:0/660734312 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "606ece18-5501-48bd-b735-45a4c435c03a", "id": 2} v 0) -- 0x7fd0f00714b0 con 0x7fd0f006f6a0 2024-04-25T16:50:30.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.235+0000 7fd0ecff9640 1 -- 172.21.15.138:0/660734312 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "606ece18-5501-48bd-b735-45a4c435c03a", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fd0e0005480 con 0x7fd0f006f6a0 2024-04-25T16:50:30.236 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-04-25T16:50:30.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.239+0000 7fd0f6277640 1 -- 172.21.15.138:0/660734312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0f006f6a0 msgr2=0x7fd0f0135c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.239+0000 7fd0f6277640 1 --2- 172.21.15.138:0/660734312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0f006f6a0 0x7fd0f0135c50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000c3c0 tx=0x7fd0e000c880 comp rx=0 tx=0).stop 2024-04-25T16:50:30.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.239+0000 7fd0f6277640 1 -- 172.21.15.138:0/660734312 shutdown_connections 2024-04-25T16:50:30.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.239+0000 7fd0f6277640 1 --2- 172.21.15.138:0/660734312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0f00b6b90 0x7fd0f0140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.239+0000 7fd0f6277640 1 --2- 172.21.15.138:0/660734312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f00b5050 0x7fd0f0158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.239+0000 7fd0f6277640 1 --2- 172.21.15.138:0/660734312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0f006f6a0 0x7fd0f0135c50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.239+0000 7fd0f6277640 1 -- 172.21.15.138:0/660734312 >> 172.21.15.138:0/660734312 conn(0x7fd0f0073570 msgr2=0x7fd0f015d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:30.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.239+0000 7fd0f6277640 1 -- 172.21.15.138:0/660734312 shutdown_connections 2024-04-25T16:50:30.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.239+0000 7fd0f6277640 1 -- 172.21.15.138:0/660734312 wait complete. 2024-04-25T16:50:30.254 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 8463d045-b8bb-493a-a1fb-7b3029fbe685 3 2024-04-25T16:50:30.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.399+0000 7f4434b01640 1 Processor -- start 2024-04-25T16:50:30.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.399+0000 7f4434b01640 1 -- start start 2024-04-25T16:50:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f4434b01640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443015de30 0x7f4430157d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f4434b01640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44300701b0 0x7f44300705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f4434b01640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4430070b00 con 0x7f443006ce90 2024-04-25T16:50:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f4434b01640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f44301582b0 con 0x7f44300701b0 2024-04-25T16:50:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f4434b01640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4430162250 con 0x7f443015de30 2024-04-25T16:50:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442e59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443015de30 0x7f4430157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442dd9b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44300701b0 0x7f44300705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442e59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443015de30 0x7f4430157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:47402/0 (socket says 172.21.15.138:47402) 2024-04-25T16:50:30.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442dd9b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44300701b0 0x7f44300705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44880/0 (socket says 172.21.15.138:44880) 2024-04-25T16:50:30.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442e59c640 1 -- 172.21.15.138:0/1826398878 learned_addr learned my addr 172.21.15.138:0/1826398878 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:50:30.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442d59a640 1 -- 172.21.15.138:0/1826398878 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3908951390 0 0) 0x7f44200014a0 con 0x7f443006ce90 2024-04-25T16:50:30.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442e59c640 1 -- 172.21.15.138:0/1826398878 >> v2:172.21.15.138:3300/0 conn(0x7f44300701b0 msgr2=0x7f44300705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:30.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442e59c640 1 --2- 172.21.15.138:0/1826398878 >> v2:172.21.15.138:3300/0 conn(0x7f44300701b0 0x7f44300705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442e59c640 1 -- 172.21.15.138:0/1826398878 >> v1:172.21.15.138:6789/0 conn(0x7f443006ce90 legacy=0x7f443006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442e59c640 1 -- 172.21.15.138:0/1826398878 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443015ee20 con 0x7f443015de30 2024-04-25T16:50:30.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442dd9b640 1 --2- 172.21.15.138:0/1826398878 >> v2:172.21.15.138:3300/0 conn(0x7f44300701b0 0x7f44300705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:30.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442e59c640 1 --2- 172.21.15.138:0/1826398878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443015de30 0x7f4430157d70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4428009870 tx=0x7f442801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d07ef19c3a581172 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:30.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442d59a640 1 -- 172.21.15.138:0/1826398878 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44280a9620 con 0x7f443015de30 2024-04-25T16:50:30.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442d59a640 1 -- 172.21.15.138:0/1826398878 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44280a97c0 con 0x7f443015de30 2024-04-25T16:50:30.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.403+0000 7f442d59a640 1 -- 172.21.15.138:0/1826398878 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44280a9b90 con 0x7f443015de30 2024-04-25T16:50:30.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 -- 172.21.15.138:0/1826398878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443015de30 msgr2=0x7f4430157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 --2- 172.21.15.138:0/1826398878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443015de30 0x7f4430157d70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4428009870 tx=0x7f442801c560 comp rx=0 tx=0).stop 2024-04-25T16:50:30.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 -- 172.21.15.138:0/1826398878 shutdown_connections 2024-04-25T16:50:30.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 --2- 172.21.15.138:0/1826398878 >> v2:172.21.15.138:3300/0 conn(0x7f44300701b0 0x7f44300705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 --2- 172.21.15.138:0/1826398878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443015de30 0x7f4430157d70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 -- 172.21.15.138:0/1826398878 >> 172.21.15.138:0/1826398878 conn(0x7f4430071f00 msgr2=0x7f4430073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:30.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 -- 172.21.15.138:0/1826398878 shutdown_connections 2024-04-25T16:50:30.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 -- 172.21.15.138:0/1826398878 wait complete. 2024-04-25T16:50:30.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 Processor -- start 2024-04-25T16:50:30.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 -- start start 2024-04-25T16:50:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443006ce90 0x7f443015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44300701b0 0x7f44301521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f443015de30 0x7f4430152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f443015e240 con 0x7f44300701b0 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f443015eb40 con 0x7f443015de30 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f4434b01640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f443016afd0 con 0x7f443006ce90 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f442e59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443006ce90 0x7f443015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f442ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f443015de30 0x7f4430152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.407+0000 7f442e59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443006ce90 0x7f443015a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:47412/0 (socket says 172.21.15.138:47412) 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f442ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f443015de30 0x7f4430152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44888/0 (socket says 172.21.15.138:44888) 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f442e59c640 1 -- 172.21.15.138:0/764887805 learned_addr learned my addr 172.21.15.138:0/764887805 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f442dd9b640 1 --2- 172.21.15.138:0/764887805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44300701b0 0x7f44301521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f442e59c640 1 -- 172.21.15.138:0/764887805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f443015de30 msgr2=0x7f4430152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f442e59c640 1 --2- 172.21.15.138:0/764887805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f443015de30 0x7f4430152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f442e59c640 1 -- 172.21.15.138:0/764887805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44300701b0 msgr2=0x7f44301521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f442e59c640 1 --2- 172.21.15.138:0/764887805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44300701b0 0x7f44301521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f442e59c640 1 -- 172.21.15.138:0/764887805 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4430158500 con 0x7f443006ce90 2024-04-25T16:50:30.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f442dd9b640 1 --2- 172.21.15.138:0/764887805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44300701b0 0x7f44301521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:30.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f442e59c640 1 --2- 172.21.15.138:0/764887805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443006ce90 0x7f443015a610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f442801ac10 tx=0x7f4428019300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:30.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f441effd640 1 -- 172.21.15.138:0/764887805 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44280b9630 con 0x7f443006ce90 2024-04-25T16:50:30.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f441effd640 1 -- 172.21.15.138:0/764887805 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44280b97f0 con 0x7f443006ce90 2024-04-25T16:50:30.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f441effd640 1 -- 172.21.15.138:0/764887805 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44280b9be0 con 0x7f443006ce90 2024-04-25T16:50:30.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f4434b01640 1 -- 172.21.15.138:0/764887805 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44301551b0 con 0x7f443006ce90 2024-04-25T16:50:30.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f4434b01640 1 -- 172.21.15.138:0/764887805 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4430158a80 con 0x7f443006ce90 2024-04-25T16:50:30.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f441effd640 1 -- 172.21.15.138:0/764887805 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f44280b1780 con 0x7f443006ce90 2024-04-25T16:50:30.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f441effd640 1 -- 172.21.15.138:0/764887805 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f44280a88b0 con 0x7f443006ce90 2024-04-25T16:50:30.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.411+0000 7f4434b01640 1 -- 172.21.15.138:0/764887805 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443006a730 con 0x7f443006ce90 2024-04-25T16:50:30.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.415+0000 7f441effd640 1 -- 172.21.15.138:0/764887805 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f44280c0030 con 0x7f443006ce90 2024-04-25T16:50:30.477 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:30.480+0000 7ff5e660f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T16:50:30.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.483+0000 7f4434b01640 1 -- 172.21.15.138:0/764887805 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8463d045-b8bb-493a-a1fb-7b3029fbe685", "id": 3} v 0) -- 0x7f44300b1c50 con 0x7f443006ce90 2024-04-25T16:50:30.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.487+0000 7f441effd640 1 -- 172.21.15.138:0/764887805 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8463d045-b8bb-493a-a1fb-7b3029fbe685", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f44280ad020 con 0x7f443006ce90 2024-04-25T16:50:30.487 INFO:teuthology.orchestra.run.smithi138.stdout:3 2024-04-25T16:50:30.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.491+0000 7f4434b01640 1 -- 172.21.15.138:0/764887805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443006ce90 msgr2=0x7f443015a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.491+0000 7f4434b01640 1 --2- 172.21.15.138:0/764887805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443006ce90 0x7f443015a610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f442801ac10 tx=0x7f4428019300 comp rx=0 tx=0).stop 2024-04-25T16:50:30.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.491+0000 7f4434b01640 1 -- 172.21.15.138:0/764887805 shutdown_connections 2024-04-25T16:50:30.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.491+0000 7f4434b01640 1 --2- 172.21.15.138:0/764887805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f443015de30 0x7f4430152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.491+0000 7f4434b01640 1 --2- 172.21.15.138:0/764887805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44300701b0 0x7f44301521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.491+0000 7f4434b01640 1 --2- 172.21.15.138:0/764887805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443006ce90 0x7f443015a610 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.491+0000 7f4434b01640 1 -- 172.21.15.138:0/764887805 >> 172.21.15.138:0/764887805 conn(0x7f4430071f00 msgr2=0x7f4430150520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:30.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.491+0000 7f4434b01640 1 -- 172.21.15.138:0/764887805 shutdown_connections 2024-04-25T16:50:30.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.491+0000 7f4434b01640 1 -- 172.21.15.138:0/764887805 wait complete. 2024-04-25T16:50:30.513 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 1768cc78-6c07-42e8-b4df-a24f2b58540c 4 2024-04-25T16:50:30.531 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:30.532+0000 7ff5e660f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T16:50:30.583 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:30.584+0000 7ff5e660f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T16:50:30.637 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:30.639+0000 7f4e3b4e8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T16:50:30.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.659+0000 7f30cab94640 1 Processor -- start 2024-04-25T16:50:30.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.663+0000 7f30cab94640 1 -- start start 2024-04-25T16:50:30.660 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:30.660+0000 7ff5e660f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T16:50:30.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.663+0000 7f30cab94640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f30c4145cb0 0x7f30c40d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.663+0000 7f30cab94640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30c40b6950 con 0x7f30c4142e20 2024-04-25T16:50:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.663+0000 7f30cab94640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30c40b6c30 con 0x7f30c41460c0 2024-04-25T16:50:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.663+0000 7f30cab94640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f30c40b6f10 con 0x7f30c4145cb0 2024-04-25T16:50:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.663+0000 7f30c9131640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f30c41460c0 0x7f30c41434d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47590/0 (socket says 172.21.15.138:47590) 2024-04-25T16:50:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.663+0000 7f30c9131640 1 -- 172.21.15.138:0/305870887 learned_addr learned my addr 172.21.15.138:0/305870887 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-25T16:50:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.663+0000 7f30c3fff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f30c4145cb0 0x7f30c40d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.663+0000 7f30c3fff640 1 -- 172.21.15.138:0/305870887 >> v1:172.21.15.138:6789/0 conn(0x7f30c41460c0 legacy=0x7f30c41434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.663+0000 7f30c3fff640 1 -- 172.21.15.138:0/305870887 >> v1:172.21.15.94:6789/0 conn(0x7f30c4142e20 legacy=0x7f30c406fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.663+0000 7f30c3fff640 1 -- 172.21.15.138:0/305870887 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c406f560 con 0x7f30c4145cb0 2024-04-25T16:50:30.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.667+0000 7f30c3fff640 1 --2- 172.21.15.138:0/305870887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4145cb0 0x7f30c40d1450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f30b00097c0 tx=0x7f30b001c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b41ffe9670a3cac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:30.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.667+0000 7f30c37fe640 1 -- 172.21.15.138:0/305870887 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b00b0070 con 0x7f30c4145cb0 2024-04-25T16:50:30.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.667+0000 7f30c37fe640 1 -- 172.21.15.138:0/305870887 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b00ab600 con 0x7f30c4145cb0 2024-04-25T16:50:30.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.667+0000 7f30c37fe640 1 -- 172.21.15.138:0/305870887 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b00ab9d0 con 0x7f30c4145cb0 2024-04-25T16:50:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.667+0000 7f30cab94640 1 -- 172.21.15.138:0/305870887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4145cb0 msgr2=0x7f30c40d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.667+0000 7f30cab94640 1 --2- 172.21.15.138:0/305870887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4145cb0 0x7f30c40d1450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f30b00097c0 tx=0x7f30b001c380 comp rx=0 tx=0).stop 2024-04-25T16:50:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.667+0000 7f30cab94640 1 -- 172.21.15.138:0/305870887 shutdown_connections 2024-04-25T16:50:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.667+0000 7f30cab94640 1 --2- 172.21.15.138:0/305870887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4145cb0 0x7f30c40d1450 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.667+0000 7f30cab94640 1 -- 172.21.15.138:0/305870887 >> 172.21.15.138:0/305870887 conn(0x7f30c4071f00 msgr2=0x7f30c4073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.667+0000 7f30cab94640 1 -- 172.21.15.138:0/305870887 shutdown_connections 2024-04-25T16:50:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.667+0000 7f30cab94640 1 -- 172.21.15.138:0/305870887 wait complete. 2024-04-25T16:50:30.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30cab94640 1 Processor -- start 2024-04-25T16:50:30.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30cab94640 1 -- start start 2024-04-25T16:50:30.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30cab94640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4142e20 0x7f30c415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30cab94640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30c4145cb0 0x7f30c4158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30cab94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c41460c0 0x7f30c413f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30cab94640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f30c416b010 con 0x7f30c41460c0 2024-04-25T16:50:30.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30cab94640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30c416ae90 con 0x7f30c4142e20 2024-04-25T16:50:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30cab94640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f30c416b190 con 0x7f30c4145cb0 2024-04-25T16:50:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c3fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30c4145cb0 0x7f30c4158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c8930640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4142e20 0x7f30c415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c3fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30c4145cb0 0x7f30c4158e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:47426/0 (socket says 172.21.15.138:47426) 2024-04-25T16:50:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c3fff640 1 -- 172.21.15.138:0/1262900064 learned_addr learned my addr 172.21.15.138:0/1262900064 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:50:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c9131640 1 --2- 172.21.15.138:0/1262900064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c41460c0 0x7f30c413f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c3fff640 1 -- 172.21.15.138:0/1262900064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4142e20 msgr2=0x7f30c415c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c3fff640 1 --2- 172.21.15.138:0/1262900064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4142e20 0x7f30c415c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c3fff640 1 -- 172.21.15.138:0/1262900064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c41460c0 msgr2=0x7f30c413f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c3fff640 1 --2- 172.21.15.138:0/1262900064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c41460c0 0x7f30c413f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c3fff640 1 -- 172.21.15.138:0/1262900064 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c40b19c0 con 0x7f30c4145cb0 2024-04-25T16:50:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c3fff640 1 --2- 172.21.15.138:0/1262900064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30c4145cb0 0x7f30c4158e50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f30b0018f90 tx=0x7f30b0019120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c17fa640 1 -- 172.21.15.138:0/1262900064 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b00b0040 con 0x7f30c4145cb0 2024-04-25T16:50:30.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c17fa640 1 -- 172.21.15.138:0/1262900064 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b00ba070 con 0x7f30c4145cb0 2024-04-25T16:50:30.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30cab94640 1 -- 172.21.15.138:0/1262900064 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c40aa600 con 0x7f30c4145cb0 2024-04-25T16:50:30.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c17fa640 1 -- 172.21.15.138:0/1262900064 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b00b4a70 con 0x7f30c4145cb0 2024-04-25T16:50:30.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30cab94640 1 -- 172.21.15.138:0/1262900064 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c40ac160 con 0x7f30c4145cb0 2024-04-25T16:50:30.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c17fa640 1 -- 172.21.15.138:0/1262900064 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f30b00b3070 con 0x7f30c4145cb0 2024-04-25T16:50:30.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.671+0000 7f30c17fa640 1 -- 172.21.15.138:0/1262900064 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f30b00b5df0 con 0x7f30c4145cb0 2024-04-25T16:50:30.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.675+0000 7f30cab94640 1 -- 172.21.15.138:0/1262900064 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c40b1720 con 0x7f30c4145cb0 2024-04-25T16:50:30.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.675+0000 7f30c17fa640 1 -- 172.21.15.138:0/1262900064 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f30c40b1720 con 0x7f30c4145cb0 2024-04-25T16:50:30.694 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:30.695+0000 7f4e3b4e8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T16:50:30.712 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:30.716+0000 7ff5e660f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T16:50:30.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.743+0000 7f30cab94640 1 -- 172.21.15.138:0/1262900064 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1768cc78-6c07-42e8-b4df-a24f2b58540c", "id": 4} v 0) -- 0x7f30c406a730 con 0x7f30c4145cb0 2024-04-25T16:50:30.748 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:30.751+0000 7f4e3b4e8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T16:50:30.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.751+0000 7f30c17fa640 1 -- 172.21.15.138:0/1262900064 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1768cc78-6c07-42e8-b4df-a24f2b58540c", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f30b0016090 con 0x7f30c4145cb0 2024-04-25T16:50:30.751 INFO:teuthology.orchestra.run.smithi138.stdout:4 2024-04-25T16:50:30.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.755+0000 7f30cab94640 1 -- 172.21.15.138:0/1262900064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30c4145cb0 msgr2=0x7f30c4158e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.755+0000 7f30cab94640 1 --2- 172.21.15.138:0/1262900064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30c4145cb0 0x7f30c4158e50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f30b0018f90 tx=0x7f30b0019120 comp rx=0 tx=0).stop 2024-04-25T16:50:30.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.755+0000 7f30cab94640 1 -- 172.21.15.138:0/1262900064 shutdown_connections 2024-04-25T16:50:30.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.755+0000 7f30cab94640 1 --2- 172.21.15.138:0/1262900064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c41460c0 0x7f30c413f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.755+0000 7f30cab94640 1 --2- 172.21.15.138:0/1262900064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30c4145cb0 0x7f30c4158e50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.755+0000 7f30cab94640 1 --2- 172.21.15.138:0/1262900064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4142e20 0x7f30c415c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.755+0000 7f30cab94640 1 -- 172.21.15.138:0/1262900064 >> 172.21.15.138:0/1262900064 conn(0x7f30c4071f00 msgr2=0x7f30c4144200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:30.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.755+0000 7f30cab94640 1 -- 172.21.15.138:0/1262900064 shutdown_connections 2024-04-25T16:50:30.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.755+0000 7f30cab94640 1 -- 172.21.15.138:0/1262900064 wait complete. 2024-04-25T16:50:30.769 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 8ee52e85-51e1-4444-92ba-696df3573c6e 5 2024-04-25T16:50:30.789 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:30.792+0000 7ff5e660f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T16:50:30.831 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:30.831+0000 7f4e3b4e8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T16:50:30.842 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:30.844+0000 7ff5e660f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T16:50:30.885 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:30.887+0000 7f4e3b4e8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T16:50:30.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.915+0000 7fe44baed640 1 Processor -- start 2024-04-25T16:50:30.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.915+0000 7fe44baed640 1 -- start start 2024-04-25T16:50:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44baed640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe444143f20 0x7fe4440d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44baed640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe4440b73d0 0x7fe4440b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44baed640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe4441614a0 0x7fe4441618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44baed640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe44415e980 con 0x7fe4441614a0 2024-04-25T16:50:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44baed640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe4440b7d20 con 0x7fe4440b73d0 2024-04-25T16:50:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44baed640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe444144330 con 0x7fe444143f20 2024-04-25T16:50:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe449889640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe444143f20 0x7fe4440d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe449088640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe4440b73d0 0x7fe4440b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe449889640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe444143f20 0x7fe4440d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:47438/0 (socket says 172.21.15.138:47438) 2024-04-25T16:50:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe449889640 1 -- 172.21.15.138:0/758901875 learned_addr learned my addr 172.21.15.138:0/758901875 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:50:30.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44a08a640 1 --2- 172.21.15.138:0/758901875 >> v2:172.21.15.94:3300/0 conn(0x7fe4441614a0 0x7fe4441618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe449088640 1 -- 172.21.15.138:0/758901875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe444143f20 msgr2=0x7fe4440d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:30.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe449088640 1 --2- 172.21.15.138:0/758901875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe444143f20 0x7fe4440d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe449088640 1 -- 172.21.15.138:0/758901875 >> v2:172.21.15.94:3300/0 conn(0x7fe4441614a0 msgr2=0x7fe4441618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:30.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe449088640 1 --2- 172.21.15.138:0/758901875 >> v2:172.21.15.94:3300/0 conn(0x7fe4441614a0 0x7fe4441618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe449088640 1 -- 172.21.15.138:0/758901875 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44415ee10 con 0x7fe4440b73d0 2024-04-25T16:50:30.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe449889640 1 --2- 172.21.15.138:0/758901875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe444143f20 0x7fe4440d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:30.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe449088640 1 --2- 172.21.15.138:0/758901875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4440b73d0 0x7fe4440b77e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe4440d44d0 tx=0x7fe43401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5a0fc95e1cc8cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:30.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe448887640 1 -- 172.21.15.138:0/758901875 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4340ab610 con 0x7fe4440b73d0 2024-04-25T16:50:30.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe448887640 1 -- 172.21.15.138:0/758901875 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4340b0070 con 0x7fe4440b73d0 2024-04-25T16:50:30.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe448887640 1 -- 172.21.15.138:0/758901875 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4340ab9d0 con 0x7fe4440b73d0 2024-04-25T16:50:30.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44baed640 1 -- 172.21.15.138:0/758901875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4440b73d0 msgr2=0x7fe4440b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44baed640 1 --2- 172.21.15.138:0/758901875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4440b73d0 0x7fe4440b77e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe4440d44d0 tx=0x7fe43401c450 comp rx=0 tx=0).stop 2024-04-25T16:50:30.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44baed640 1 -- 172.21.15.138:0/758901875 shutdown_connections 2024-04-25T16:50:30.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44baed640 1 --2- 172.21.15.138:0/758901875 >> v2:172.21.15.94:3300/0 conn(0x7fe4441614a0 0x7fe4441618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44baed640 1 --2- 172.21.15.138:0/758901875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4440b73d0 0x7fe4440b77e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44baed640 1 --2- 172.21.15.138:0/758901875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe444143f20 0x7fe4440d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.919+0000 7fe44baed640 1 -- 172.21.15.138:0/758901875 >> 172.21.15.138:0/758901875 conn(0x7fe4440d33d0 msgr2=0x7fe4440d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:30.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe44baed640 1 -- 172.21.15.138:0/758901875 shutdown_connections 2024-04-25T16:50:30.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe44baed640 1 -- 172.21.15.138:0/758901875 wait complete. 2024-04-25T16:50:30.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe44baed640 1 Processor -- start 2024-04-25T16:50:30.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe44baed640 1 -- start start 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe44baed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4440b73d0 0x7fe44413fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe44baed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe444143f20 0x7fe4441469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe44baed640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4441614a0 0x7fe444146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe44baed640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440d0ca0 con 0x7fe444143f20 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe44baed640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440d0fb0 con 0x7fe4440b73d0 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe44baed640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe444162200 con 0x7fe4441614a0 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe449889640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4440b73d0 0x7fe44413fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe44a08a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4441614a0 0x7fe444146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe449889640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4440b73d0 0x7fe44413fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44922/0 (socket says 172.21.15.138:44922) 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe449889640 1 -- 172.21.15.138:0/419436472 learned_addr learned my addr 172.21.15.138:0/419436472 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe449088640 1 --2- 172.21.15.138:0/419436472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe444143f20 0x7fe4441469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe449889640 1 -- 172.21.15.138:0/419436472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4441614a0 msgr2=0x7fe444146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe449889640 1 --2- 172.21.15.138:0/419436472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4441614a0 0x7fe444146f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe449889640 1 -- 172.21.15.138:0/419436472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe444143f20 msgr2=0x7fe4441469e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:30.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe449889640 1 --2- 172.21.15.138:0/419436472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe444143f20 0x7fe4441469e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:30.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe449889640 1 -- 172.21.15.138:0/419436472 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44415a5b0 con 0x7fe4440b73d0 2024-04-25T16:50:30.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe44a08a640 1 --2- 172.21.15.138:0/419436472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4441614a0 0x7fe444146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:30.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.923+0000 7fe449889640 1 --2- 172.21.15.138:0/419436472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4440b73d0 0x7fe44413fcb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe44000e8f0 tx=0x7fe44000edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:30.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.927+0000 7fe43a7fc640 1 -- 172.21.15.138:0/419436472 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440017020 con 0x7fe4440b73d0 2024-04-25T16:50:30.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.927+0000 7fe43a7fc640 1 -- 172.21.15.138:0/419436472 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe440004240 con 0x7fe4440b73d0 2024-04-25T16:50:30.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.927+0000 7fe44baed640 1 -- 172.21.15.138:0/419436472 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44406eac0 con 0x7fe4440b73d0 2024-04-25T16:50:30.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.927+0000 7fe43a7fc640 1 -- 172.21.15.138:0/419436472 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44000c730 con 0x7fe4440b73d0 2024-04-25T16:50:30.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.927+0000 7fe44baed640 1 -- 172.21.15.138:0/419436472 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44406f080 con 0x7fe4440b73d0 2024-04-25T16:50:30.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.927+0000 7fe43a7fc640 1 -- 172.21.15.138:0/419436472 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe44001a020 con 0x7fe4440b73d0 2024-04-25T16:50:30.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.927+0000 7fe43a7fc640 1 -- 172.21.15.138:0/419436472 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fe440003d80 con 0x7fe4440b73d0 2024-04-25T16:50:30.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.927+0000 7fe44baed640 1 -- 172.21.15.138:0/419436472 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44406a730 con 0x7fe4440b73d0 2024-04-25T16:50:30.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:30.927+0000 7fe43a7fc640 1 -- 172.21.15.138:0/419436472 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe440021030 con 0x7fe4440b73d0 2024-04-25T16:50:30.979 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:30.979+0000 7f4e3b4e8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T16:50:30.992 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:30.996+0000 7ff5e660f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T16:50:31.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.003+0000 7fe44baed640 1 -- 172.21.15.138:0/419436472 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8ee52e85-51e1-4444-92ba-696df3573c6e", "id": 5} v 0) -- 0x7fe4440d02a0 con 0x7fe4440b73d0 2024-04-25T16:50:31.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.011+0000 7fe43a7fc640 1 -- 172.21.15.138:0/419436472 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8ee52e85-51e1-4444-92ba-696df3573c6e", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fe440003800 con 0x7fe4440b73d0 2024-04-25T16:50:31.008 INFO:teuthology.orchestra.run.smithi138.stdout:5 2024-04-25T16:50:31.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.011+0000 7fe44baed640 1 -- 172.21.15.138:0/419436472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4440b73d0 msgr2=0x7fe44413fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.011+0000 7fe44baed640 1 --2- 172.21.15.138:0/419436472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4440b73d0 0x7fe44413fcb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe44000e8f0 tx=0x7fe44000edb0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.011+0000 7fe44baed640 1 -- 172.21.15.138:0/419436472 shutdown_connections 2024-04-25T16:50:31.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.011+0000 7fe44baed640 1 --2- 172.21.15.138:0/419436472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4441614a0 0x7fe444146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.011+0000 7fe44baed640 1 --2- 172.21.15.138:0/419436472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe444143f20 0x7fe4441469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.011+0000 7fe44baed640 1 --2- 172.21.15.138:0/419436472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4440b73d0 0x7fe44413fcb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.011+0000 7fe44baed640 1 -- 172.21.15.138:0/419436472 >> 172.21.15.138:0/419436472 conn(0x7fe4440d33d0 msgr2=0x7fe4440d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:31.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.011+0000 7fe44baed640 1 -- 172.21.15.138:0/419436472 shutdown_connections 2024-04-25T16:50:31.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.011+0000 7fe44baed640 1 -- 172.21.15.138:0/419436472 wait complete. 2024-04-25T16:50:31.026 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 8df799bb-47cd-4458-b347-bed7f533b6a9 6 2024-04-25T16:50:31.038 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:31.039+0000 7f4e3b4e8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T16:50:31.054 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:31.056+0000 7ff5e660f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T16:50:31.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.175+0000 7f9f557bd640 1 Processor -- start 2024-04-25T16:50:31.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.175+0000 7f9f557bd640 1 -- start start 2024-04-25T16:50:31.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f557bd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9f50071740 0x7f9f5006e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f557bd640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f50071340 con 0x7f9f500cf200 2024-04-25T16:50:31.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f557bd640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f5006f180 con 0x7f9f50158170 2024-04-25T16:50:31.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f557bd640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f9f5015ee80 con 0x7f9f50071740 2024-04-25T16:50:31.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4e7fc640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9f50158170 0x7f9f50070520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47604/0 (socket says 172.21.15.138:47604) 2024-04-25T16:50:31.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4e7fc640 1 -- 172.21.15.138:0/3460131680 learned_addr learned my addr 172.21.15.138:0/3460131680 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-25T16:50:31.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4effd640 1 --2- 172.21.15.138:0/3460131680 >> v2:172.21.15.94:3300/0 conn(0x7f9f50071740 0x7f9f5006e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1642180167 0 0) 0x7f9f5006f180 con 0x7f9f50158170 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f3c003530 con 0x7f9f50158170 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1187916023 0 0) 0x7f9f3c003530 con 0x7f9f50158170 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 >> v2:172.21.15.94:3300/0 conn(0x7f9f50071740 msgr2=0x7f9f5006e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 --2- 172.21.15.138:0/3460131680 >> v2:172.21.15.94:3300/0 conn(0x7f9f50071740 0x7f9f5006e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 >> v1:172.21.15.94:6789/0 conn(0x7f9f500cf200 legacy=0x7f9f50070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5006f460 con 0x7f9f50158170 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f9f40001680 con 0x7f9f50158170 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 >> v1:172.21.15.138:6789/0 conn(0x7f9f50158170 legacy=0x7f9f50070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 --2- 172.21.15.138:0/3460131680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f3c0039c0 0x7f9f3c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 --2- 172.21.15.138:0/3460131680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f3c004810 0x7f9f3c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 --2- 172.21.15.138:0/3460131680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3c0051d0 0x7f9f3c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5015ee80 con 0x7f9f3c004810 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c00c170 con 0x7f9f3c0039c0 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c00c370 con 0x7f9f3c0051d0 2024-04-25T16:50:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4effd640 1 --2- 172.21.15.138:0/3460131680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f3c0039c0 0x7f9f3c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4f7fe640 1 --2- 172.21.15.138:0/3460131680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3c0051d0 0x7f9f3c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4e7fc640 1 --2- 172.21.15.138:0/3460131680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f3c004810 0x7f9f3c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4effd640 1 -- 172.21.15.138:0/3460131680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3c0051d0 msgr2=0x7f9f3c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:31.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4effd640 1 --2- 172.21.15.138:0/3460131680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3c0051d0 0x7f9f3c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4effd640 1 -- 172.21.15.138:0/3460131680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f3c004810 msgr2=0x7f9f3c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:31.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4effd640 1 --2- 172.21.15.138:0/3460131680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f3c004810 0x7f9f3c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4effd640 1 -- 172.21.15.138:0/3460131680 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c003530 con 0x7f9f3c0039c0 2024-04-25T16:50:31.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.179+0000 7f9f4f7fe640 1 -- 172.21.15.138:0/3460131680 reap_dead start 2024-04-25T16:50:31.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.183+0000 7f9f4effd640 1 --2- 172.21.15.138:0/3460131680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f3c0039c0 0x7f9f3c003e30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009ab0 tx=0x7f9f3801c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=179a32d8ba981797 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:31.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.183+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f380096b0 con 0x7f9f3c0039c0 2024-04-25T16:50:31.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.183+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f380a6480 con 0x7f9f3c0039c0 2024-04-25T16:50:31.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.183+0000 7f9f4dffb640 1 -- 172.21.15.138:0/3460131680 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f380a6870 con 0x7f9f3c0039c0 2024-04-25T16:50:31.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.183+0000 7f9f557bd640 1 -- 172.21.15.138:0/3460131680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f3c0039c0 msgr2=0x7f9f3c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.183+0000 7f9f557bd640 1 --2- 172.21.15.138:0/3460131680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f3c0039c0 0x7f9f3c003e30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009ab0 tx=0x7f9f3801c990 comp rx=0 tx=0).stop 2024-04-25T16:50:31.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.183+0000 7f9f557bd640 1 -- 172.21.15.138:0/3460131680 shutdown_connections 2024-04-25T16:50:31.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.183+0000 7f9f557bd640 1 --2- 172.21.15.138:0/3460131680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f3c0039c0 0x7f9f3c003e30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.183+0000 7f9f557bd640 1 -- 172.21.15.138:0/3460131680 >> 172.21.15.138:0/3460131680 conn(0x7f9f50073570 msgr2=0x7f9f50073980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:31.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.183+0000 7f9f557bd640 1 -- 172.21.15.138:0/3460131680 shutdown_connections 2024-04-25T16:50:31.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.183+0000 7f9f557bd640 1 -- 172.21.15.138:0/3460131680 wait complete. 2024-04-25T16:50:31.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f557bd640 1 Processor -- start 2024-04-25T16:50:31.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f557bd640 1 -- start start 2024-04-25T16:50:31.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f557bd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3c0039c0 0x7f9f5014f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f557bd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f5015cec0 0x7f9f501f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f557bd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f5014fdb0 0x7f9f501f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f557bd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5006fa20 con 0x7f9f5015cec0 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f557bd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f500cf6e0 con 0x7f9f5014fdb0 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f557bd640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f501614b0 con 0x7f9f3c0039c0 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f4effd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3c0039c0 0x7f9f5014f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f4f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f5014fdb0 0x7f9f501f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f4effd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3c0039c0 0x7f9f5014f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:47472/0 (socket says 172.21.15.138:47472) 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f4effd640 1 -- 172.21.15.138:0/3418757602 learned_addr learned my addr 172.21.15.138:0/3418757602 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f4f7fe640 1 -- 172.21.15.138:0/3418757602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3c0039c0 msgr2=0x7f9f5014f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f4e7fc640 1 --2- 172.21.15.138:0/3418757602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f5015cec0 0x7f9f501f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f4f7fe640 1 --2- 172.21.15.138:0/3418757602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3c0039c0 0x7f9f5014f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f4f7fe640 1 -- 172.21.15.138:0/3418757602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f5015cec0 msgr2=0x7f9f501f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f4f7fe640 1 --2- 172.21.15.138:0/3418757602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f5015cec0 0x7f9f501f36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f4f7fe640 1 -- 172.21.15.138:0/3418757602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f50154400 con 0x7f9f5014fdb0 2024-04-25T16:50:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f4f7fe640 1 --2- 172.21.15.138:0/3418757602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f5014fdb0 0x7f9f501f3c00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9f500d0910 tx=0x7f9f4800fd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f477fe640 1 -- 172.21.15.138:0/3418757602 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f48003b90 con 0x7f9f5014fdb0 2024-04-25T16:50:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f477fe640 1 -- 172.21.15.138:0/3418757602 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f48018070 con 0x7f9f5014fdb0 2024-04-25T16:50:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f477fe640 1 -- 172.21.15.138:0/3418757602 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f480050d0 con 0x7f9f5014fdb0 2024-04-25T16:50:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f557bd640 1 -- 172.21.15.138:0/3418757602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5015d560 con 0x7f9f5014fdb0 2024-04-25T16:50:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f557bd640 1 -- 172.21.15.138:0/3418757602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f501546c0 con 0x7f9f5014fdb0 2024-04-25T16:50:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f477fe640 1 -- 172.21.15.138:0/3418757602 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9f48005270 con 0x7f9f5014fdb0 2024-04-25T16:50:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.187+0000 7f9f477fe640 1 -- 172.21.15.138:0/3418757602 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f9f4800dd50 con 0x7f9f5014fdb0 2024-04-25T16:50:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.191+0000 7f9f557bd640 1 -- 172.21.15.138:0/3418757602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5015d2d0 con 0x7f9f5014fdb0 2024-04-25T16:50:31.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.211+0000 7f9f477fe640 1 -- 172.21.15.138:0/3418757602 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9f48005560 con 0x7f9f5014fdb0 2024-04-25T16:50:31.209 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:31.211+0000 7f4e3b4e8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T16:50:31.272 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:31.275+0000 7f4e3b4e8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T16:50:31.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.279+0000 7f9f557bd640 1 -- 172.21.15.138:0/3418757602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8df799bb-47cd-4458-b347-bed7f533b6a9", "id": 6} v 0) -- 0x7f9f50076830 con 0x7f9f5014fdb0 2024-04-25T16:50:31.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.283+0000 7f9f477fe640 1 -- 172.21.15.138:0/3418757602 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8df799bb-47cd-4458-b347-bed7f533b6a9", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f9f48005740 con 0x7f9f5014fdb0 2024-04-25T16:50:31.284 INFO:teuthology.orchestra.run.smithi138.stdout:6 2024-04-25T16:50:31.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.287+0000 7f9f557bd640 1 -- 172.21.15.138:0/3418757602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f5014fdb0 msgr2=0x7f9f501f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.287+0000 7f9f557bd640 1 --2- 172.21.15.138:0/3418757602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f5014fdb0 0x7f9f501f3c00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9f500d0910 tx=0x7f9f4800fd40 comp rx=0 tx=0).stop 2024-04-25T16:50:31.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.287+0000 7f9f557bd640 1 -- 172.21.15.138:0/3418757602 shutdown_connections 2024-04-25T16:50:31.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.287+0000 7f9f557bd640 1 --2- 172.21.15.138:0/3418757602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f5014fdb0 0x7f9f501f3c00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.287+0000 7f9f557bd640 1 --2- 172.21.15.138:0/3418757602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f5015cec0 0x7f9f501f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.287+0000 7f9f557bd640 1 --2- 172.21.15.138:0/3418757602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3c0039c0 0x7f9f5014f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.287+0000 7f9f557bd640 1 -- 172.21.15.138:0/3418757602 >> 172.21.15.138:0/3418757602 conn(0x7f9f50073570 msgr2=0x7f9f50070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:31.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.287+0000 7f9f557bd640 1 -- 172.21.15.138:0/3418757602 shutdown_connections 2024-04-25T16:50:31.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.287+0000 7f9f557bd640 1 -- 172.21.15.138:0/3418757602 wait complete. 2024-04-25T16:50:31.303 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 27e6f89e-2d85-4888-b0d3-bbb58d15544b 7 2024-04-25T16:50:31.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb0af95640 1 Processor -- start 2024-04-25T16:50:31.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb0af95640 1 -- start start 2024-04-25T16:50:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb0af95640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb040b7060 0x7ffb040b7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb0af95640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffb0406dfa0 0x7ffb0406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb0af95640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb04070170 con 0x7ffb040cd1c0 2024-04-25T16:50:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb0af95640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ffb040b79b0 con 0x7ffb0406dfa0 2024-04-25T16:50:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb0af95640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffb0406e8f0 con 0x7ffb040b7060 2024-04-25T16:50:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb03fff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffb0406dfa0 0x7ffb0406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb08d31640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb040b7060 0x7ffb040b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb09532640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ffb040cd1c0 0x7ffb040cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47620/0 (socket says 172.21.15.138:47620) 2024-04-25T16:50:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb09532640 1 -- 172.21.15.138:0/3060547475 learned_addr learned my addr 172.21.15.138:0/3060547475 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-25T16:50:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb03fff640 1 --2- 172.21.15.138:0/3060547475 >> v2:172.21.15.138:3300/0 conn(0x7ffb0406dfa0 0x7ffb0406e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44948/0 (socket says 172.21.15.138:44948) 2024-04-25T16:50:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb03fff640 1 -- 172.21.15.138:0/3060547475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb040b7060 msgr2=0x7ffb040b7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb03fff640 1 --2- 172.21.15.138:0/3060547475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb040b7060 0x7ffb040b7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb03fff640 1 -- 172.21.15.138:0/3060547475 >> v1:172.21.15.138:6789/0 conn(0x7ffb040cd1c0 legacy=0x7ffb040cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb037fe640 1 -- 172.21.15.138:0/3060547475 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 703327114 0 0) 0x7ffafc0014a0 con 0x7ffb040cd1c0 2024-04-25T16:50:31.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb03fff640 1 -- 172.21.15.138:0/3060547475 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb040b1340 con 0x7ffb0406dfa0 2024-04-25T16:50:31.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.487+0000 7ffb08d31640 1 --2- 172.21.15.138:0/3060547475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb040b7060 0x7ffb040b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:31.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb03fff640 1 --2- 172.21.15.138:0/3060547475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb0406dfa0 0x7ffb0406e3b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00097c0 tx=0x7ffaf001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58b66178789a37ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:31.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb037fe640 1 -- 172.21.15.138:0/3060547475 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00b0070 con 0x7ffb0406dfa0 2024-04-25T16:50:31.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb037fe640 1 -- 172.21.15.138:0/3060547475 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf00ab600 con 0x7ffb0406dfa0 2024-04-25T16:50:31.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb037fe640 1 -- 172.21.15.138:0/3060547475 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00ab9d0 con 0x7ffb0406dfa0 2024-04-25T16:50:31.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb0af95640 1 -- 172.21.15.138:0/3060547475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb0406dfa0 msgr2=0x7ffb0406e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb0af95640 1 --2- 172.21.15.138:0/3060547475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb0406dfa0 0x7ffb0406e3b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00097c0 tx=0x7ffaf001c3e0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb0af95640 1 -- 172.21.15.138:0/3060547475 shutdown_connections 2024-04-25T16:50:31.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb0af95640 1 --2- 172.21.15.138:0/3060547475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb0406dfa0 0x7ffb0406e3b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb0af95640 1 --2- 172.21.15.138:0/3060547475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb040b7060 0x7ffb040b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb0af95640 1 -- 172.21.15.138:0/3060547475 >> 172.21.15.138:0/3060547475 conn(0x7ffb040ce490 msgr2=0x7ffb040ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:31.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb0af95640 1 -- 172.21.15.138:0/3060547475 shutdown_connections 2024-04-25T16:50:31.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb0af95640 1 -- 172.21.15.138:0/3060547475 wait complete. 2024-04-25T16:50:31.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb0af95640 1 Processor -- start 2024-04-25T16:50:31.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.491+0000 7ffb0af95640 1 -- start start 2024-04-25T16:50:31.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb0af95640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0406dfa0 0x7ffb040c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb0af95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb040b7060 0x7ffb0415cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb0af95640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb040cd1c0 0x7ffb040b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb0af95640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0406fc50 con 0x7ffb040b7060 2024-04-25T16:50:31.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb0af95640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0406f970 con 0x7ffb040cd1c0 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb0af95640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffb040d05f0 con 0x7ffb0406dfa0 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb09532640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb040cd1c0 0x7ffb040b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb08d31640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0406dfa0 0x7ffb040c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb09532640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb040cd1c0 0x7ffb040b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44956/0 (socket says 172.21.15.138:44956) 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb08d31640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0406dfa0 0x7ffb040c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:47488/0 (socket says 172.21.15.138:47488) 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb09532640 1 -- 172.21.15.138:0/3449912240 learned_addr learned my addr 172.21.15.138:0/3449912240 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb03fff640 1 --2- 172.21.15.138:0/3449912240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb040b7060 0x7ffb0415cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb09532640 1 -- 172.21.15.138:0/3449912240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0406dfa0 msgr2=0x7ffb040c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb09532640 1 --2- 172.21.15.138:0/3449912240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0406dfa0 0x7ffb040c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb09532640 1 -- 172.21.15.138:0/3449912240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb040b7060 msgr2=0x7ffb0415cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb09532640 1 --2- 172.21.15.138:0/3449912240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb040b7060 0x7ffb0415cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb09532640 1 -- 172.21.15.138:0/3449912240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb04150a00 con 0x7ffb040cd1c0 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb09532640 1 --2- 172.21.15.138:0/3449912240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb040cd1c0 0x7ffb040b5740 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00c6d0 tx=0x7ffafc00cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb017fa640 1 -- 172.21.15.138:0/3449912240 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc018070 con 0x7ffb040cd1c0 2024-04-25T16:50:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb017fa640 1 -- 172.21.15.138:0/3449912240 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffafc004d70 con 0x7ffb040cd1c0 2024-04-25T16:50:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb0af95640 1 -- 172.21.15.138:0/3449912240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb04150ce0 con 0x7ffb040cd1c0 2024-04-25T16:50:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb017fa640 1 -- 172.21.15.138:0/3449912240 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00a880 con 0x7ffb040cd1c0 2024-04-25T16:50:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb0af95640 1 -- 172.21.15.138:0/3449912240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb041512a0 con 0x7ffb040cd1c0 2024-04-25T16:50:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb017fa640 1 -- 172.21.15.138:0/3449912240 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffafc01b070 con 0x7ffb040cd1c0 2024-04-25T16:50:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb017fa640 1 -- 172.21.15.138:0/3449912240 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7ffafc012a60 con 0x7ffb040cd1c0 2024-04-25T16:50:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.495+0000 7ffb0af95640 1 -- 172.21.15.138:0/3449912240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb0406a730 con 0x7ffb040cd1c0 2024-04-25T16:50:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.499+0000 7ffb017fa640 1 -- 172.21.15.138:0/3449912240 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ffb04070170 con 0x7ffb040cd1c0 2024-04-25T16:50:31.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.563+0000 7ffb0af95640 1 -- 172.21.15.138:0/3449912240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "27e6f89e-2d85-4888-b0d3-bbb58d15544b", "id": 7} v 0) -- 0x7ffb040cd820 con 0x7ffb040cd1c0 2024-04-25T16:50:31.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.571+0000 7ffb017fa640 1 -- 172.21.15.138:0/3449912240 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "27e6f89e-2d85-4888-b0d3-bbb58d15544b", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7ffafc00acd0 con 0x7ffb040cd1c0 2024-04-25T16:50:31.571 INFO:teuthology.orchestra.run.smithi138.stdout:7 2024-04-25T16:50:31.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.575+0000 7ffb0af95640 1 -- 172.21.15.138:0/3449912240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb040cd1c0 msgr2=0x7ffb040b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.575+0000 7ffb0af95640 1 --2- 172.21.15.138:0/3449912240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb040cd1c0 0x7ffb040b5740 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00c6d0 tx=0x7ffafc00cb90 comp rx=0 tx=0).stop 2024-04-25T16:50:31.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.575+0000 7ffb0af95640 1 -- 172.21.15.138:0/3449912240 shutdown_connections 2024-04-25T16:50:31.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.575+0000 7ffb0af95640 1 --2- 172.21.15.138:0/3449912240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb040cd1c0 0x7ffb040b5740 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.575+0000 7ffb0af95640 1 --2- 172.21.15.138:0/3449912240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb040b7060 0x7ffb0415cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.575+0000 7ffb0af95640 1 --2- 172.21.15.138:0/3449912240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb0406dfa0 0x7ffb040c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.575+0000 7ffb0af95640 1 -- 172.21.15.138:0/3449912240 >> 172.21.15.138:0/3449912240 conn(0x7ffb040ce490 msgr2=0x7ffb041551c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:31.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.575+0000 7ffb0af95640 1 -- 172.21.15.138:0/3449912240 shutdown_connections 2024-04-25T16:50:31.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:50:31.575+0000 7ffb0af95640 1 -- 172.21.15.138:0/3449912240 wait complete. 2024-04-25T16:50:31.604 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T16:50:31.604 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T16:50:31.606 INFO:tasks.ceph.osd.0:Started 2024-04-25T16:50:31.606 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T16:50:31.607 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T16:50:31.609 INFO:tasks.ceph.osd.1:Started 2024-04-25T16:50:31.609 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T16:50:31.609 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T16:50:31.611 INFO:tasks.ceph.osd.2:Started 2024-04-25T16:50:31.611 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T16:50:31.611 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T16:50:31.613 INFO:tasks.ceph.osd.3:Started 2024-04-25T16:50:31.614 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T16:50:31.614 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T16:50:31.616 INFO:tasks.ceph.osd.4:Started 2024-04-25T16:50:31.616 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T16:50:31.616 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T16:50:31.618 INFO:tasks.ceph.osd.5:Started 2024-04-25T16:50:31.618 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T16:50:31.618 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T16:50:31.620 INFO:tasks.ceph.osd.6:Started 2024-04-25T16:50:31.620 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T16:50:31.621 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T16:50:31.623 INFO:tasks.ceph.osd.7:Started 2024-04-25T16:50:31.623 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:50:31.771 INFO:tasks.ceph.mgr.y.smithi094.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T16:50:31.771 INFO:tasks.ceph.mgr.y.smithi094.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T16:50:31.771 INFO:tasks.ceph.mgr.y.smithi094.stderr: from numpy import show_config as show_numpy_config 2024-04-25T16:50:31.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.804+0000 7fdc5626d640 1 Processor -- start 2024-04-25T16:50:31.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.804+0000 7fdc5626d640 1 -- start start 2024-04-25T16:50:31.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.804+0000 7fdc5626d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc500cdaa0 0x7fdc500d2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.804+0000 7fdc5626d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdc500d2570 0x7fdc500b7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.804+0000 7fdc5626d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc500d2980 con 0x7fdc500cdeb0 2024-04-25T16:50:31.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.804+0000 7fdc5626d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdc500cec90 con 0x7fdc500d2570 2024-04-25T16:50:31.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.804+0000 7fdc5626d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdc500ce3f0 con 0x7fdc500cdaa0 2024-04-25T16:50:31.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.804+0000 7fdc4f7fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fdc500cdeb0 0x7fdc500b6910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:35560/0 (socket says 172.21.15.94:35560) 2024-04-25T16:50:31.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.804+0000 7fdc4f7fe640 1 -- 172.21.15.94:0/1385538465 learned_addr learned my addr 172.21.15.94:0/1385538465 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:50:31.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.804+0000 7fdc4ffff640 1 --2- 172.21.15.94:0/1385538465 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc500cdaa0 0x7fdc500d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.804+0000 7fdc5480a640 1 --2- 172.21.15.94:0/1385538465 >> v2:172.21.15.138:3300/0 conn(0x7fdc500d2570 0x7fdc500b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.804+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1903483996 0 0) 0x7fdc44001120 con 0x7fdc500cdeb0 2024-04-25T16:50:31.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.804+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc38003530 con 0x7fdc500cdeb0 2024-04-25T16:50:31.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2031135603 0 0) 0x7fdc38003530 con 0x7fdc500cdeb0 2024-04-25T16:50:31.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc500cdaa0 msgr2=0x7fdc500d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:31.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 --2- 172.21.15.94:0/1385538465 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc500cdaa0 0x7fdc500d2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 >> v2:172.21.15.138:3300/0 conn(0x7fdc500d2570 msgr2=0x7fdc500b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 --2- 172.21.15.94:0/1385538465 >> v2:172.21.15.138:3300/0 conn(0x7fdc500d2570 0x7fdc500b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc500ce6d0 con 0x7fdc500cdeb0 2024-04-25T16:50:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fdc44001850 con 0x7fdc500cdeb0 2024-04-25T16:50:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 >> v1:172.21.15.94:6789/0 conn(0x7fdc500cdeb0 legacy=0x7fdc500b6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 --2- 172.21.15.94:0/1385538465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc380039c0 0x7fdc38003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 --2- 172.21.15.94:0/1385538465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc38004810 0x7fdc38004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 --2- 172.21.15.94:0/1385538465 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc500cdaa0 0x7fdc38005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc500cec90 con 0x7fdc38004810 2024-04-25T16:50:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc500ce3f0 con 0x7fdc380039c0 2024-04-25T16:50:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdc3800ba80 con 0x7fdc500cdaa0 2024-04-25T16:50:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4f7fe640 1 --2- 172.21.15.94:0/1385538465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc38004810 0x7fdc38004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc5480a640 1 --2- 172.21.15.94:0/1385538465 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc500cdaa0 0x7fdc38005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4ffff640 1 --2- 172.21.15.94:0/1385538465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc380039c0 0x7fdc38003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4f7fe640 1 -- 172.21.15.94:0/1385538465 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc500cdaa0 msgr2=0x7fdc38005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:31.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4f7fe640 1 --2- 172.21.15.94:0/1385538465 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc500cdaa0 0x7fdc38005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4f7fe640 1 -- 172.21.15.94:0/1385538465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc380039c0 msgr2=0x7fdc38003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:31.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4f7fe640 1 --2- 172.21.15.94:0/1385538465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc380039c0 0x7fdc38003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4f7fe640 1 -- 172.21.15.94:0/1385538465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc38003530 con 0x7fdc38004810 2024-04-25T16:50:31.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4ffff640 1 --2- 172.21.15.94:0/1385538465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc380039c0 0x7fdc38003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:50:31.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4f7fe640 1 --2- 172.21.15.94:0/1385538465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc38004810 0x7fdc38004c20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdc5006d570 tx=0x7fdc4401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efe1d4cfe7340b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:31.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4400c0e0 con 0x7fdc38004810 2024-04-25T16:50:31.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc440a9020 con 0x7fdc38004810 2024-04-25T16:50:31.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc4effd640 1 -- 172.21.15.94:0/1385538465 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44005b00 con 0x7fdc38004810 2024-04-25T16:50:31.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc5626d640 1 -- 172.21.15.94:0/1385538465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc38004810 msgr2=0x7fdc38004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc5626d640 1 --2- 172.21.15.94:0/1385538465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc38004810 0x7fdc38004c20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdc5006d570 tx=0x7fdc4401cc40 comp rx=0 tx=0).stop 2024-04-25T16:50:31.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc5480a640 1 -- 172.21.15.94:0/1385538465 reap_dead start 2024-04-25T16:50:31.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc5626d640 1 -- 172.21.15.94:0/1385538465 shutdown_connections 2024-04-25T16:50:31.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc5626d640 1 -- 172.21.15.94:0/1385538465 >> 172.21.15.94:0/1385538465 conn(0x7fdc500cfed0 msgr2=0x7fdc500d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:31.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc5626d640 1 -- 172.21.15.94:0/1385538465 shutdown_connections 2024-04-25T16:50:31.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.808+0000 7fdc5626d640 1 -- 172.21.15.94:0/1385538465 wait complete. 2024-04-25T16:50:31.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc5626d640 1 Processor -- start 2024-04-25T16:50:31.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc5626d640 1 -- start start 2024-04-25T16:50:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc5626d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc38004810 0x7fdc50070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc5626d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc501556d0 0x7fdc500c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc5626d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc50155ae0 0x7fdc500ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc5626d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc500b76e0 con 0x7fdc38004810 2024-04-25T16:50:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc5626d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc500b7560 con 0x7fdc50155ae0 2024-04-25T16:50:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc5626d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdc500cf7e0 con 0x7fdc501556d0 2024-04-25T16:50:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc4ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc38004810 0x7fdc50070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc4ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc38004810 0x7fdc50070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48132/0 (socket says 172.21.15.94:48132) 2024-04-25T16:50:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc4ffff640 1 -- 172.21.15.94:0/1034894184 learned_addr learned my addr 172.21.15.94:0/1034894184 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc4f7fe640 1 --2- 172.21.15.94:0/1034894184 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc501556d0 0x7fdc500c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc5480a640 1 --2- 172.21.15.94:0/1034894184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc50155ae0 0x7fdc500ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc4ffff640 1 -- 172.21.15.94:0/1034894184 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc501556d0 msgr2=0x7fdc500c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc4ffff640 1 --2- 172.21.15.94:0/1034894184 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc501556d0 0x7fdc500c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc4ffff640 1 -- 172.21.15.94:0/1034894184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc50155ae0 msgr2=0x7fdc500ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc4ffff640 1 --2- 172.21.15.94:0/1034894184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc50155ae0 0x7fdc500ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc4ffff640 1 -- 172.21.15.94:0/1034894184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc50070c40 con 0x7fdc38004810 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc4ffff640 1 --2- 172.21.15.94:0/1034894184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc38004810 0x7fdc50070700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdc40007ea0 tx=0x7fdc40007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc4cff9640 1 -- 172.21.15.94:0/1034894184 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4001a020 con 0x7fdc38004810 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc4cff9640 1 -- 172.21.15.94:0/1034894184 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc40016470 con 0x7fdc38004810 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc5626d640 1 -- 172.21.15.94:0/1034894184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc50156550 con 0x7fdc38004810 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc4cff9640 1 -- 172.21.15.94:0/1034894184 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40016840 con 0x7fdc38004810 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.812+0000 7fdc5626d640 1 -- 172.21.15.94:0/1034894184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc500c7df0 con 0x7fdc38004810 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.816+0000 7fdc4cff9640 1 -- 172.21.15.94:0/1034894184 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdc4001d020 con 0x7fdc38004810 2024-04-25T16:50:31.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.816+0000 7fdc4cff9640 1 -- 172.21.15.94:0/1034894184 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fdc40015aa0 con 0x7fdc38004810 2024-04-25T16:50:31.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.816+0000 7fdc5626d640 1 -- 172.21.15.94:0/1034894184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5006a730 con 0x7fdc38004810 2024-04-25T16:50:31.815 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:31.816+0000 7ff5e660f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T16:50:31.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.816+0000 7fdc4cff9640 1 -- 172.21.15.94:0/1034894184 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdc40024030 con 0x7fdc38004810 2024-04-25T16:50:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.876+0000 7fdc5626d640 1 -- 172.21.15.94:0/1034894184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc500c6c30 con 0x7fdc38004810 2024-04-25T16:50:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.876+0000 7fdc4cff9640 1 -- 172.21.15.94:0/1034894184 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fdc400153d0 con 0x7fdc38004810 2024-04-25T16:50:31.874 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:50:31.874 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":10,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:31.570598+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:50:31.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.876+0000 7fdc5626d640 1 -- 172.21.15.94:0/1034894184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc38004810 msgr2=0x7fdc50070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:31.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.876+0000 7fdc5626d640 1 --2- 172.21.15.94:0/1034894184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc38004810 0x7fdc50070700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdc40007ea0 tx=0x7fdc40007f90 comp rx=0 tx=0).stop 2024-04-25T16:50:31.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.876+0000 7fdc5626d640 1 -- 172.21.15.94:0/1034894184 shutdown_connections 2024-04-25T16:50:31.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.876+0000 7fdc5626d640 1 --2- 172.21.15.94:0/1034894184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc50155ae0 0x7fdc500ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.876+0000 7fdc5626d640 1 --2- 172.21.15.94:0/1034894184 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc501556d0 0x7fdc500c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.876+0000 7fdc5626d640 1 --2- 172.21.15.94:0/1034894184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc38004810 0x7fdc50070700 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:31.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.876+0000 7fdc5626d640 1 -- 172.21.15.94:0/1034894184 >> 172.21.15.94:0/1034894184 conn(0x7fdc500cfed0 msgr2=0x7fdc500b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:31.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.876+0000 7fdc5626d640 1 -- 172.21.15.94:0/1034894184 shutdown_connections 2024-04-25T16:50:31.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:31.876+0000 7fdc5626d640 1 -- 172.21.15.94:0/1034894184 wait complete. 2024-04-25T16:50:31.892 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T16:50:31.893 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T16:50:31.903 INFO:tasks.ceph.mgr.x.smithi138.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T16:50:31.903 INFO:tasks.ceph.mgr.x.smithi138.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T16:50:31.903 INFO:tasks.ceph.mgr.x.smithi138.stderr: from numpy import show_config as show_numpy_config 2024-04-25T16:50:31.944 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:31.943+0000 7f4e3b4e8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T16:50:31.948 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:31.952+0000 7ff5e660f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:50:32.078 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:32.079+0000 7f4e3b4e8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:50:32.102 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:32.104+0000 7ff5e660f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T16:50:32.165 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:32.168+0000 7ff5e660f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T16:50:32.247 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:32.247+0000 7f4e3b4e8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T16:50:32.253 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T16:50:32.256+0000 7f9a1a7f47c0 -1 Falling back to public interface 2024-04-25T16:50:32.257 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T16:50:32.260+0000 7f83779b07c0 -1 Falling back to public interface 2024-04-25T16:50:32.258 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T16:50:32.259+0000 7f2c331817c0 -1 Falling back to public interface 2024-04-25T16:50:32.258 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T16:50:32.259+0000 7f55eec4b7c0 -1 Falling back to public interface 2024-04-25T16:50:32.258 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T16:50:32.259+0000 7f2c32141640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2024-04-25T16:50:32.265 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T16:50:32.268+0000 7efca8cc77c0 -1 Falling back to public interface 2024-04-25T16:50:32.277 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T16:50:32.279+0000 7f7fdac7b7c0 -1 Falling back to public interface 2024-04-25T16:50:32.280 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T16:50:32.283+0000 7f0f17ed27c0 -1 Falling back to public interface 2024-04-25T16:50:32.289 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T16:50:32.292+0000 7f380404a7c0 -1 Falling back to public interface 2024-04-25T16:50:32.301 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:32.304+0000 7ff5e660f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:50:32.318 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:32.319+0000 7f4e3b4e8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T16:50:32.451 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:32.451+0000 7f4e3b4e8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:50:32.698 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T16:50:32.838 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.840+0000 7f49b1d41640 1 Processor -- start 2024-04-25T16:50:32.839 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.840+0000 7f49b1d41640 1 -- start start 2024-04-25T16:50:32.839 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.840+0000 7f49b1d41640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f49ac0b1710 0x7f49ac15caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:32.840 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.840+0000 7f49b1d41640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49ac06f760 0x7f49ac06e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:32.840 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.840+0000 7f49b1d41640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49ac0d06f0 con 0x7f49ac06fb70 2024-04-25T16:50:32.840 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.840+0000 7f49b1d41640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f49ac0b1b20 con 0x7f49ac0b1710 2024-04-25T16:50:32.840 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.840+0000 7f49b1d41640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f49ac0d0150 con 0x7f49ac06f760 2024-04-25T16:50:32.841 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.844+0000 7f49abfff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f49ac06fb70 0x7f49ac0d21e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:42380/0 (socket says 172.21.15.94:42380) 2024-04-25T16:50:32.841 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.844+0000 7f49abfff640 1 -- 172.21.15.94:0/2022364577 learned_addr learned my addr 172.21.15.94:0/2022364577 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:50:32.841 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.844+0000 7f49aaffd640 1 --2- 172.21.15.94:0/2022364577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49ac06f760 0x7f49ac06e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:32.841 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.844+0000 7f49ab7fe640 1 --2- 172.21.15.94:0/2022364577 >> v2:172.21.15.138:3300/0 conn(0x7f49ac0b1710 0x7f49ac15caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:32.841 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.844+0000 7f49ab7fe640 1 -- 172.21.15.94:0/2022364577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49ac06f760 msgr2=0x7f49ac06e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:32.841 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.844+0000 7f49ab7fe640 1 --2- 172.21.15.94:0/2022364577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49ac06f760 0x7f49ac06e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:32.841 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.844+0000 7f49ab7fe640 1 -- 172.21.15.94:0/2022364577 >> v1:172.21.15.138:6789/0 conn(0x7f49ac06fb70 legacy=0x7f49ac0d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:32.841 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.844+0000 7f49ab7fe640 1 -- 172.21.15.94:0/2022364577 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac0d3080 con 0x7f49ac0b1710 2024-04-25T16:50:32.841 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.844+0000 7f49aaffd640 1 --2- 172.21.15.94:0/2022364577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49ac06f760 0x7f49ac06e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:32.843 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.844+0000 7f49ab7fe640 1 --2- 172.21.15.94:0/2022364577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49ac0b1710 0x7f49ac15caa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f499c0097a0 tx=0x7f499c01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bcc443feffad571 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:32.843 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.844+0000 7f49aa7fc640 1 -- 172.21.15.94:0/2022364577 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0ab5f0 con 0x7f49ac0b1710 2024-04-25T16:50:32.843 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.844+0000 7f49aa7fc640 1 -- 172.21.15.94:0/2022364577 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f499c0b0070 con 0x7f49ac0b1710 2024-04-25T16:50:32.843 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.844+0000 7f49aa7fc640 1 -- 172.21.15.94:0/2022364577 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0ab9b0 con 0x7f49ac0b1710 2024-04-25T16:50:32.844 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 -- 172.21.15.94:0/2022364577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49ac0b1710 msgr2=0x7f49ac15caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:32.844 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 --2- 172.21.15.94:0/2022364577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49ac0b1710 0x7f49ac15caa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f499c0097a0 tx=0x7f499c01c3f0 comp rx=0 tx=0).stop 2024-04-25T16:50:32.844 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 -- 172.21.15.94:0/2022364577 shutdown_connections 2024-04-25T16:50:32.844 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 --2- 172.21.15.94:0/2022364577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49ac06f760 0x7f49ac06e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:32.844 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 --2- 172.21.15.94:0/2022364577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49ac0b1710 0x7f49ac15caa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:32.844 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 -- 172.21.15.94:0/2022364577 >> 172.21.15.94:0/2022364577 conn(0x7f49ac09ba70 msgr2=0x7f49ac0b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:32.845 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 -- 172.21.15.94:0/2022364577 shutdown_connections 2024-04-25T16:50:32.845 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 -- 172.21.15.94:0/2022364577 wait complete. 2024-04-25T16:50:32.846 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 Processor -- start 2024-04-25T16:50:32.846 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 -- start start 2024-04-25T16:50:32.849 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49ac06f760 0x7f49ac14f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:32.849 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49ac06fb70 0x7f49ac0b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:32.849 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49ac0b1710 0x7f49ac150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:32.849 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac169e60 con 0x7f49ac06fb70 2024-04-25T16:50:32.849 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac0d2d00 con 0x7f49ac06f760 2024-04-25T16:50:32.849 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49b1d41640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f49ac169fe0 con 0x7f49ac0b1710 2024-04-25T16:50:32.849 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49ac06fb70 0x7f49ac0b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:32.849 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49ac06fb70 0x7f49ac0b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48138/0 (socket says 172.21.15.94:48138) 2024-04-25T16:50:32.849 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49aaffd640 1 -- 172.21.15.94:0/579345696 learned_addr learned my addr 172.21.15.94:0/579345696 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:32.849 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49ab7fe640 1 --2- 172.21.15.94:0/579345696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49ac06f760 0x7f49ac14f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49abfff640 1 --2- 172.21.15.94:0/579345696 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49ac0b1710 0x7f49ac150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49aaffd640 1 -- 172.21.15.94:0/579345696 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49ac0b1710 msgr2=0x7f49ac150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49aaffd640 1 --2- 172.21.15.94:0/579345696 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49ac0b1710 0x7f49ac150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49aaffd640 1 -- 172.21.15.94:0/579345696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49ac06f760 msgr2=0x7f49ac14f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49aaffd640 1 --2- 172.21.15.94:0/579345696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49ac06f760 0x7f49ac14f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49aaffd640 1 -- 172.21.15.94:0/579345696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac157900 con 0x7f49ac06fb70 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.848+0000 7f49ab7fe640 1 --2- 172.21.15.94:0/579345696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49ac06f760 0x7f49ac14f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.852+0000 7f49aaffd640 1 --2- 172.21.15.94:0/579345696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49ac06fb70 0x7f49ac0b2730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f499400e980 tx=0x7f499400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.852+0000 7f49a3fff640 1 -- 172.21.15.94:0/579345696 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994017070 con 0x7f49ac06fb70 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.852+0000 7f49a3fff640 1 -- 172.21.15.94:0/579345696 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f499400ca00 con 0x7f49ac06fb70 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.852+0000 7f49b1d41640 1 -- 172.21.15.94:0/579345696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ac157be0 con 0x7f49ac06fb70 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.852+0000 7f49a3fff640 1 -- 172.21.15.94:0/579345696 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499400cdd0 con 0x7f49ac06fb70 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.852+0000 7f49b1d41640 1 -- 172.21.15.94:0/579345696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ac1581a0 con 0x7f49ac06fb70 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.852+0000 7f49a3fff640 1 -- 172.21.15.94:0/579345696 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f499401a070 con 0x7f49ac06fb70 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.852+0000 7f49a3fff640 1 -- 172.21.15.94:0/579345696 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f4994009420 con 0x7f49ac06fb70 2024-04-25T16:50:32.850 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.852+0000 7f49b1d41640 1 -- 172.21.15.94:0/579345696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ac0768c0 con 0x7f49ac06fb70 2024-04-25T16:50:32.853 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.856+0000 7f49a3fff640 1 -- 172.21.15.94:0/579345696 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4994013470 con 0x7f49ac06fb70 2024-04-25T16:50:32.912 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.912+0000 7f49b1d41640 1 -- 172.21.15.94:0/579345696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49ac153e40 con 0x7f49ac06fb70 2024-04-25T16:50:32.913 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.916+0000 7f49a3fff640 1 -- 172.21.15.94:0/579345696 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f4994013650 con 0x7f49ac06fb70 2024-04-25T16:50:32.913 INFO:teuthology.misc.health.smithi094.stdout: 2024-04-25T16:50:32.913 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":10,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:31.570598+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:50:32.914 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.916+0000 7f49b1d41640 1 -- 172.21.15.94:0/579345696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49ac06fb70 msgr2=0x7f49ac0b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:32.914 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.916+0000 7f49b1d41640 1 --2- 172.21.15.94:0/579345696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49ac06fb70 0x7f49ac0b2730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f499400e980 tx=0x7f499400ee40 comp rx=0 tx=0).stop 2024-04-25T16:50:32.914 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.916+0000 7f49b1d41640 1 -- 172.21.15.94:0/579345696 shutdown_connections 2024-04-25T16:50:32.914 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.916+0000 7f49b1d41640 1 --2- 172.21.15.94:0/579345696 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49ac0b1710 0x7f49ac150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:32.914 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.916+0000 7f49b1d41640 1 --2- 172.21.15.94:0/579345696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49ac06fb70 0x7f49ac0b2730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:32.914 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.916+0000 7f49b1d41640 1 --2- 172.21.15.94:0/579345696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49ac06f760 0x7f49ac14f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:32.914 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.916+0000 7f49b1d41640 1 -- 172.21.15.94:0/579345696 >> 172.21.15.94:0/579345696 conn(0x7f49ac09ba70 msgr2=0x7f49ac1530b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:32.914 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.916+0000 7f49b1d41640 1 -- 172.21.15.94:0/579345696 shutdown_connections 2024-04-25T16:50:32.914 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:32.916+0000 7f49b1d41640 1 -- 172.21.15.94:0/579345696 wait complete. 2024-04-25T16:50:32.930 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T16:50:32.932 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:32.936+0000 7ff5e660f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T16:50:33.009 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:33.012+0000 7ff5e660f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T16:50:33.067 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:33.067+0000 7f4e3b4e8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T16:50:33.142 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:33.143+0000 7f4e3b4e8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T16:50:33.190 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:33.192+0000 7ff5e660f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T16:50:33.264 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:33.268+0000 7ff5e660f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T16:50:33.323 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:33.323+0000 7f4e3b4e8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T16:50:33.396 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:33.399+0000 7f4e3b4e8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T16:50:33.554 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:33.556+0000 7ff5e660f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T16:50:33.615 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:33.616+0000 7ff5e660f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T16:50:33.689 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:33.691+0000 7f4e3b4e8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T16:50:33.753 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T16:50:33.755+0000 7f4e3b4e8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T16:50:34.668 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:34.672+0000 7ff58cfc0640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T16:50:34.669 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-25T16:50:34.672+0000 7ff58cfc0640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T16:50:36.021 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T16:50:36.024+0000 7f9a1a7f47c0 -1 osd.0 0 log_to_monitors true 2024-04-25T16:50:36.088 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T16:50:36.091+0000 7f7fdac7b7c0 -1 osd.4 0 log_to_monitors true 2024-04-25T16:50:36.093 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T16:50:36.096+0000 7efca8cc77c0 -1 osd.3 0 log_to_monitors true 2024-04-25T16:50:36.094 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T16:50:36.095+0000 7f0f17ed27c0 -1 osd.5 0 log_to_monitors true 2024-04-25T16:50:36.136 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T16:50:36.140+0000 7f83779b07c0 -1 osd.1 0 log_to_monitors true 2024-04-25T16:50:36.198 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T16:50:36.199+0000 7f2c331817c0 -1 osd.6 0 log_to_monitors true 2024-04-25T16:50:36.198 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T16:50:36.199+0000 7f55eec4b7c0 -1 osd.7 0 log_to_monitors true 2024-04-25T16:50:36.208 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T16:50:36.208+0000 7f380404a7c0 -1 osd.2 0 log_to_monitors true 2024-04-25T16:50:39.322 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T16:50:39.324+0000 7f37ffff1640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T16:50:39.338 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T16:50:39.340+0000 7f8373957640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T16:50:39.344 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T16:50:39.348+0000 7f37f7df2640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:50:39.349 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T16:50:39.352+0000 7f9a1679b640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T16:50:39.365 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T16:50:39.368+0000 7f836b758640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:50:39.369 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T16:50:39.372+0000 7efca4c6e640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T16:50:39.393 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T16:50:39.396+0000 7efc9ca6f640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:50:39.393 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T16:50:39.396+0000 7f9a0e59c640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:50:39.737 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T16:50:39.881 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26b4c56640 1 Processor -- start 2024-04-25T16:50:39.881 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26b4c56640 1 -- start start 2024-04-25T16:50:39.885 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26b4c56640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26b015dc20 0x7f26b0161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:39.885 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26b4c56640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f26b00d1650 0x7f26b00d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:39.885 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26b4c56640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26b01627b0 con 0x7f26b00ced10 2024-04-25T16:50:39.885 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26b4c56640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f26b006ca10 con 0x7f26b00d1650 2024-04-25T16:50:39.885 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26b4c56640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f26b01621f0 con 0x7f26b015dc20 2024-04-25T16:50:39.885 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26aed9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f26b00ced10 0x7f26b00cf120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:35148/0 (socket says 172.21.15.94:35148) 2024-04-25T16:50:39.885 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26aed9d640 1 -- 172.21.15.94:0/3502615569 learned_addr learned my addr 172.21.15.94:0/3502615569 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:50:39.885 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ae59c640 1 --2- 172.21.15.94:0/3502615569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26b015dc20 0x7f26b0161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:39.885 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26add9b640 1 --2- 172.21.15.94:0/3502615569 >> v2:172.21.15.138:3300/0 conn(0x7f26b00d1650 0x7f26b00d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:39.885 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3066580505 0 0) 0x7f26b01627b0 con 0x7f26b00ced10 2024-04-25T16:50:39.885 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2698003530 con 0x7f26b00ced10 2024-04-25T16:50:39.885 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2541080989 0 0) 0x7f2698003530 con 0x7f26b00ced10 2024-04-25T16:50:39.885 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26b015dc20 msgr2=0x7f26b0161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:39.886 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 --2- 172.21.15.94:0/3502615569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26b015dc20 0x7f26b0161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:39.886 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 >> v2:172.21.15.138:3300/0 conn(0x7f26b00d1650 msgr2=0x7f26b00d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:39.886 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 --2- 172.21.15.94:0/3502615569 >> v2:172.21.15.138:3300/0 conn(0x7f26b00d1650 0x7f26b00d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:39.886 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b01624d0 con 0x7f26b00ced10 2024-04-25T16:50:39.886 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f26a8001a20 con 0x7f26b00ced10 2024-04-25T16:50:39.886 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 >> v1:172.21.15.94:6789/0 conn(0x7f26b00ced10 legacy=0x7f26b00cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:39.886 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 --2- 172.21.15.94:0/3502615569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26980039c0 0x7f2698003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:39.886 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 --2- 172.21.15.94:0/3502615569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2698004810 0x7f2698004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:39.886 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 --2- 172.21.15.94:0/3502615569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26b015dc20 0x7f2698005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:39.886 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26b006ca10 con 0x7f2698004810 2024-04-25T16:50:39.886 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01621f0 con 0x7f26980039c0 2024-04-25T16:50:39.886 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f269800ba80 con 0x7f26b015dc20 2024-04-25T16:50:39.886 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26add9b640 1 --2- 172.21.15.94:0/3502615569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2698004810 0x7f2698004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:39.887 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26ae59c640 1 --2- 172.21.15.94:0/3502615569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26980039c0 0x7f2698003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:39.887 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.884+0000 7f26aed9d640 1 --2- 172.21.15.94:0/3502615569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26b015dc20 0x7f2698005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:39.887 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26add9b640 1 -- 172.21.15.94:0/3502615569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26b015dc20 msgr2=0x7f2698005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:39.887 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26add9b640 1 --2- 172.21.15.94:0/3502615569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26b015dc20 0x7f2698005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:39.887 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26add9b640 1 -- 172.21.15.94:0/3502615569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26980039c0 msgr2=0x7f2698003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:39.887 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26add9b640 1 --2- 172.21.15.94:0/3502615569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26980039c0 0x7f2698003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:39.887 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26add9b640 1 -- 172.21.15.94:0/3502615569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2698003530 con 0x7f2698004810 2024-04-25T16:50:39.887 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26ae59c640 1 --2- 172.21.15.94:0/3502615569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26980039c0 0x7f2698003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:50:39.888 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26add9b640 1 --2- 172.21.15.94:0/3502615569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2698004810 0x7f2698004c20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f26b00cc950 tx=0x7f26a401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed278a19c922619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:39.888 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a40096b0 con 0x7f2698004810 2024-04-25T16:50:39.888 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a40a6500 con 0x7f2698004810 2024-04-25T16:50:39.888 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26ad59a640 1 -- 172.21.15.94:0/3502615569 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a40a68f0 con 0x7f2698004810 2024-04-25T16:50:39.888 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26b4c56640 1 -- 172.21.15.94:0/3502615569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2698004810 msgr2=0x7f2698004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:39.888 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26b4c56640 1 --2- 172.21.15.94:0/3502615569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2698004810 0x7f2698004c20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f26b00cc950 tx=0x7f26a401ca00 comp rx=0 tx=0).stop 2024-04-25T16:50:39.888 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26aed9d640 1 -- 172.21.15.94:0/3502615569 reap_dead start 2024-04-25T16:50:39.889 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26b4c56640 1 -- 172.21.15.94:0/3502615569 shutdown_connections 2024-04-25T16:50:39.889 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26b4c56640 1 -- 172.21.15.94:0/3502615569 >> 172.21.15.94:0/3502615569 conn(0x7f26b009ba70 msgr2=0x7f26b00b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:39.889 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26b4c56640 1 -- 172.21.15.94:0/3502615569 shutdown_connections 2024-04-25T16:50:39.889 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.888+0000 7f26b4c56640 1 -- 172.21.15.94:0/3502615569 wait complete. 2024-04-25T16:50:39.889 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26b4c56640 1 Processor -- start 2024-04-25T16:50:39.889 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26b4c56640 1 -- start start 2024-04-25T16:50:39.891 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26b4c56640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2698004810 0x7f26b01503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26b4c56640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26b0166990 0x7f26b00c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26b4c56640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b015a8b0 0x7f26b0070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26b4c56640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0168d90 con 0x7f26b015a8b0 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26b4c56640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0168c10 con 0x7f2698004810 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26b4c56640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f26b0168f10 con 0x7f26b0166990 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b015a8b0 0x7f26b0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b015a8b0 0x7f26b0070450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48022/0 (socket says 172.21.15.94:48022) 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26aed9d640 1 -- 172.21.15.94:0/3113635441 learned_addr learned my addr 172.21.15.94:0/3113635441 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26add9b640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26b0166990 0x7f26b00c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26ae59c640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2698004810 0x7f26b01503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26aed9d640 1 -- 172.21.15.94:0/3113635441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26b0166990 msgr2=0x7f26b00c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26aed9d640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26b0166990 0x7f26b00c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26aed9d640 1 -- 172.21.15.94:0/3113635441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2698004810 msgr2=0x7f26b01503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:39.892 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26aed9d640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2698004810 0x7f26b01503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:39.893 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26aed9d640 1 -- 172.21.15.94:0/3113635441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b0150ea0 con 0x7f26b015a8b0 2024-04-25T16:50:39.893 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26aed9d640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b015a8b0 0x7f26b0070450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f26b0074150 tx=0x7f26a800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:39.893 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f269effd640 1 -- 172.21.15.94:0/3113635441 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a800eb10 con 0x7f26b015a8b0 2024-04-25T16:50:39.893 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f269effd640 1 -- 172.21.15.94:0/3113635441 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a8018070 con 0x7f26b015a8b0 2024-04-25T16:50:39.893 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f269effd640 1 -- 172.21.15.94:0/3113635441 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8013420 con 0x7f26b015a8b0 2024-04-25T16:50:39.893 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26b4c56640 1 -- 172.21.15.94:0/3113635441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b00c7480 con 0x7f26b015a8b0 2024-04-25T16:50:39.893 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.892+0000 7f26b4c56640 1 -- 172.21.15.94:0/3113635441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b0165eb0 con 0x7f26b015a8b0 2024-04-25T16:50:39.894 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.896+0000 7f26b4c56640 1 -- 172.21.15.94:0/3113635441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b00768c0 con 0x7f26b015a8b0 2024-04-25T16:50:39.894 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.896+0000 7f269effd640 1 -- 172.21.15.94:0/3113635441 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 93151+0+0 (secure 0 0 0) 0x7f26a80135c0 con 0x7f26b015a8b0 2024-04-25T16:50:39.894 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.896+0000 7f269effd640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f269006f370 0x7f2690071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:39.894 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.896+0000 7f269effd640 1 -- 172.21.15.94:0/3113635441 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f26a808e5a0 con 0x7f26b015a8b0 2024-04-25T16:50:39.894 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.896+0000 7f26ae59c640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f269006f370 0x7f2690071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:39.895 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.896+0000 7f26ae59c640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f269006f370 0x7f2690071880 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f26b0072e90 tx=0x7f26a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:39.899 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:39.900+0000 7f269effd640 1 -- 172.21.15.94:0/3113635441 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f26a805c300 con 0x7f26b015a8b0 2024-04-25T16:50:39.922 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T16:50:39.922+0000 7f55eabf2640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T16:50:39.949 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T16:50:39.950+0000 7f55e29f3640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:50:40.023 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T16:50:40.022+0000 7f0f13e79640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T16:50:40.043 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T16:50:40.042+0000 7f7fd6c22640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T16:50:40.051 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.052+0000 7f26b4c56640 1 -- 172.21.15.94:0/3113635441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26b0074220 con 0x7f26b015a8b0 2024-04-25T16:50:40.053 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.056+0000 7f269effd640 1 -- 172.21.15.94:0/3113635441 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f26a8064380 con 0x7f26b015a8b0 2024-04-25T16:50:40.053 INFO:teuthology.misc.health.smithi094.stdout: 2024-04-25T16:50:40.053 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":12,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:37.669002+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:50:40.054 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T16:50:40.054+0000 7f0f0bc7a640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:50:40.055 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.056+0000 7f26b4c56640 1 -- 172.21.15.94:0/3113635441 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f269006f370 msgr2=0x7f2690071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:40.056 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.056+0000 7f26b4c56640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f269006f370 0x7f2690071880 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f26b0072e90 tx=0x7f26a0008040 comp rx=0 tx=0).stop 2024-04-25T16:50:40.056 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.056+0000 7f26b4c56640 1 -- 172.21.15.94:0/3113635441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b015a8b0 msgr2=0x7f26b0070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:40.056 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.056+0000 7f26b4c56640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b015a8b0 0x7f26b0070450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f26b0074150 tx=0x7f26a800e530 comp rx=0 tx=0).stop 2024-04-25T16:50:40.056 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.056+0000 7f26b4c56640 1 -- 172.21.15.94:0/3113635441 shutdown_connections 2024-04-25T16:50:40.056 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.056+0000 7f26b4c56640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b015a8b0 0x7f26b0070450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:40.056 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.056+0000 7f26b4c56640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f269006f370 0x7f2690071880 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:40.056 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.056+0000 7f26b4c56640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26b0166990 0x7f26b00c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:40.056 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.056+0000 7f26b4c56640 1 --2- 172.21.15.94:0/3113635441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2698004810 0x7f26b01503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:40.056 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.056+0000 7f26b4c56640 1 -- 172.21.15.94:0/3113635441 >> 172.21.15.94:0/3113635441 conn(0x7f26b009ba70 msgr2=0x7f26b006f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:40.056 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.056+0000 7f26b4c56640 1 -- 172.21.15.94:0/3113635441 shutdown_connections 2024-04-25T16:50:40.056 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:40.056+0000 7f26b4c56640 1 -- 172.21.15.94:0/3113635441 wait complete. 2024-04-25T16:50:40.065 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T16:50:40.066+0000 7f7fcea23640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:50:40.071 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T16:50:40.072 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T16:50:40.074+0000 7f2c2f128640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T16:50:40.113 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T16:50:40.114+0000 7f2c26f29640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:50:44.062 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:50:44.067 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:50:46.878 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T16:50:47.029 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06b4ace640 1 Processor -- start 2024-04-25T16:50:47.029 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06b4ace640 1 -- start start 2024-04-25T16:50:47.033 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06b4ace640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f06b00b8df0 0x7f06b015da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.033 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06b4ace640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06b00cde30 0x7f06b00ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.033 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06b4ace640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b00ce780 con 0x7f06b01615c0 2024-04-25T16:50:47.033 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06b4ace640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f06b00b4790 con 0x7f06b00b8df0 2024-04-25T16:50:47.033 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06b4ace640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f06b00b3b80 con 0x7f06b00cde30 2024-04-25T16:50:47.033 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06aed9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f06b01615c0 0x7f06b01619d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:35162/0 (socket says 172.21.15.94:35162) 2024-04-25T16:50:47.033 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06aed9d640 1 -- 172.21.15.94:0/2175408469 learned_addr learned my addr 172.21.15.94:0/2175408469 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:50:47.033 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06add9b640 1 --2- 172.21.15.94:0/2175408469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06b00cde30 0x7f06b00ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.034 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ae59c640 1 --2- 172.21.15.94:0/2175408469 >> v2:172.21.15.138:3300/0 conn(0x7f06b00b8df0 0x7f06b015da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.034 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3517210118 0 0) 0x7f06a80014a0 con 0x7f06b01615c0 2024-04-25T16:50:47.034 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0698003530 con 0x7f06b01615c0 2024-04-25T16:50:47.034 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2900713751 0 0) 0x7f0698003530 con 0x7f06b01615c0 2024-04-25T16:50:47.034 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06b00cde30 msgr2=0x7f06b00ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:47.034 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 --2- 172.21.15.94:0/2175408469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06b00cde30 0x7f06b00ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.034 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 >> v2:172.21.15.138:3300/0 conn(0x7f06b00b8df0 msgr2=0x7f06b015da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:47.034 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 --2- 172.21.15.94:0/2175408469 >> v2:172.21.15.138:3300/0 conn(0x7f06b00b8df0 0x7f06b015da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.034 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b00b3e60 con 0x7f06b01615c0 2024-04-25T16:50:47.034 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f06a8001bd0 con 0x7f06b01615c0 2024-04-25T16:50:47.034 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 >> v1:172.21.15.94:6789/0 conn(0x7f06b01615c0 legacy=0x7f06b01619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.034 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 --2- 172.21.15.94:0/2175408469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06b00cde30 0x7f06980039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.034 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 --2- 172.21.15.94:0/2175408469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0698003f40 0x7f0698004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.035 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 --2- 172.21.15.94:0/2175408469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0698004d50 0x7f0698005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.035 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06b00b4790 con 0x7f0698004d50 2024-04-25T16:50:47.035 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06b00b3b80 con 0x7f0698003f40 2024-04-25T16:50:47.035 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.032+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f069800ba20 con 0x7f06b00cde30 2024-04-25T16:50:47.035 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06aed9d640 1 --2- 172.21.15.94:0/2175408469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0698004d50 0x7f0698005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.035 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06ae59c640 1 --2- 172.21.15.94:0/2175408469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06b00cde30 0x7f06980039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.035 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06add9b640 1 --2- 172.21.15.94:0/2175408469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0698003f40 0x7f0698004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.035 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06aed9d640 1 -- 172.21.15.94:0/2175408469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06b00cde30 msgr2=0x7f06980039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:47.035 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06aed9d640 1 --2- 172.21.15.94:0/2175408469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06b00cde30 0x7f06980039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.035 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06aed9d640 1 -- 172.21.15.94:0/2175408469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0698003f40 msgr2=0x7f0698004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:47.035 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06aed9d640 1 --2- 172.21.15.94:0/2175408469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0698003f40 0x7f0698004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.035 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06aed9d640 1 -- 172.21.15.94:0/2175408469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0698003530 con 0x7f0698004d50 2024-04-25T16:50:47.036 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06aed9d640 1 --2- 172.21.15.94:0/2175408469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0698004d50 0x7f0698005160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06b00b9510 tx=0x7f06a801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872cd2308285e9b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:47.037 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a800a690 con 0x7f0698004d50 2024-04-25T16:50:47.037 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a80072f0 con 0x7f0698004d50 2024-04-25T16:50:47.037 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06ad59a640 1 -- 172.21.15.94:0/2175408469 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a8007720 con 0x7f0698004d50 2024-04-25T16:50:47.037 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06b4ace640 1 -- 172.21.15.94:0/2175408469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0698004d50 msgr2=0x7f0698005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.037 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06b4ace640 1 --2- 172.21.15.94:0/2175408469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0698004d50 0x7f0698005160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06b00b9510 tx=0x7f06a801d360 comp rx=0 tx=0).stop 2024-04-25T16:50:47.037 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06aed9d640 1 -- 172.21.15.94:0/2175408469 reap_dead start 2024-04-25T16:50:47.037 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06b4ace640 1 -- 172.21.15.94:0/2175408469 shutdown_connections 2024-04-25T16:50:47.037 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06b4ace640 1 -- 172.21.15.94:0/2175408469 >> 172.21.15.94:0/2175408469 conn(0x7f06b00d3420 msgr2=0x7f06b00d3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:47.037 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06b4ace640 1 -- 172.21.15.94:0/2175408469 shutdown_connections 2024-04-25T16:50:47.037 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.036+0000 7f06b4ace640 1 -- 172.21.15.94:0/2175408469 wait complete. 2024-04-25T16:50:47.040 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06b4ace640 1 Processor -- start 2024-04-25T16:50:47.040 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06b4ace640 1 -- start start 2024-04-25T16:50:47.040 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06b4ace640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06b0153630 0x7f06b01575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.040 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06b4ace640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06b0152df0 0x7f06b00cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.040 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06b4ace640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b0153200 0x7f06b00cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.040 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06b4ace640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06b00b36f0 con 0x7f06b0153200 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06b4ace640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06b00b4420 con 0x7f06b0152df0 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06b4ace640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f06b00b3870 con 0x7f06b0153630 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b0153200 0x7f06b00cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b0153200 0x7f06b00cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48056/0 (socket says 172.21.15.94:48056) 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06aed9d640 1 -- 172.21.15.94:0/3416468289 learned_addr learned my addr 172.21.15.94:0/3416468289 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06ae59c640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06b0153630 0x7f06b01575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06add9b640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06b0152df0 0x7f06b00cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06aed9d640 1 -- 172.21.15.94:0/3416468289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06b0153630 msgr2=0x7f06b01575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06aed9d640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06b0153630 0x7f06b01575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06aed9d640 1 -- 172.21.15.94:0/3416468289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06b0152df0 msgr2=0x7f06b00cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06aed9d640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06b0152df0 0x7f06b00cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06aed9d640 1 -- 172.21.15.94:0/3416468289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b0157c80 con 0x7f06b0153200 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06add9b640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06b0152df0 0x7f06b00cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:50:47.041 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06aed9d640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b0153200 0x7f06b00cbf40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f06a801dc00 tx=0x7f0698005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:47.042 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f069effd640 1 -- 172.21.15.94:0/3416468289 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a80ba020 con 0x7f06b0153200 2024-04-25T16:50:47.042 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f069effd640 1 -- 172.21.15.94:0/3416468289 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a80011d0 con 0x7f06b0153200 2024-04-25T16:50:47.042 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06b4ace640 1 -- 172.21.15.94:0/3416468289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b0157f60 con 0x7f06b0153200 2024-04-25T16:50:47.042 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f069effd640 1 -- 172.21.15.94:0/3416468289 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a8001b90 con 0x7f06b0153200 2024-04-25T16:50:47.042 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.040+0000 7f06b4ace640 1 -- 172.21.15.94:0/3416468289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b0158520 con 0x7f06b0153200 2024-04-25T16:50:47.043 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.044+0000 7f06b4ace640 1 -- 172.21.15.94:0/3416468289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b0158720 con 0x7f06b0153200 2024-04-25T16:50:47.043 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.044+0000 7f069effd640 1 -- 172.21.15.94:0/3416468289 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f06a80072f0 con 0x7f06b0153200 2024-04-25T16:50:47.043 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.044+0000 7f069effd640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f069006f400 0x7f0690071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.043 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.044+0000 7f069effd640 1 -- 172.21.15.94:0/3416468289 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f06a8128780 con 0x7f06b0153200 2024-04-25T16:50:47.043 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.044+0000 7f06ae59c640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f069006f400 0x7f0690071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.043 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.044+0000 7f06ae59c640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f069006f400 0x7f0690071910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009ac0 tx=0x7f06a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:47.046 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.048+0000 7f069effd640 1 -- 172.21.15.94:0/3416468289 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f06b00ce780 con 0x7f06b0153200 2024-04-25T16:50:47.197 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.200+0000 7f06b4ace640 1 -- 172.21.15.94:0/3416468289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06b0161c20 con 0x7f06b0153200 2024-04-25T16:50:47.198 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.200+0000 7f069effd640 1 -- 172.21.15.94:0/3416468289 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f06a80f5a50 con 0x7f06b0153200 2024-04-25T16:50:47.199 INFO:teuthology.misc.health.smithi094.stdout: 2024-04-25T16:50:47.199 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":16,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:43.953393+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:50:47.201 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.204+0000 7f06b4ace640 1 -- 172.21.15.94:0/3416468289 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f069006f400 msgr2=0x7f0690071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.201 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.204+0000 7f06b4ace640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f069006f400 0x7f0690071910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009ac0 tx=0x7f06a0008040 comp rx=0 tx=0).stop 2024-04-25T16:50:47.201 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.204+0000 7f06b4ace640 1 -- 172.21.15.94:0/3416468289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b0153200 msgr2=0x7f06b00cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.201 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.204+0000 7f06b4ace640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b0153200 0x7f06b00cbf40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f06a801dc00 tx=0x7f0698005840 comp rx=0 tx=0).stop 2024-04-25T16:50:47.201 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.204+0000 7f06b4ace640 1 -- 172.21.15.94:0/3416468289 shutdown_connections 2024-04-25T16:50:47.201 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.204+0000 7f06b4ace640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b0153200 0x7f06b00cbf40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.201 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.204+0000 7f06b4ace640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06b0152df0 0x7f06b00cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.201 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.204+0000 7f06b4ace640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f069006f400 0x7f0690071910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.202 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.204+0000 7f06b4ace640 1 --2- 172.21.15.94:0/3416468289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06b0153630 0x7f06b01575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.202 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.204+0000 7f06b4ace640 1 -- 172.21.15.94:0/3416468289 >> 172.21.15.94:0/3416468289 conn(0x7f06b00d3420 msgr2=0x7f06b0071050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:47.202 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.204+0000 7f06b4ace640 1 -- 172.21.15.94:0/3416468289 shutdown_connections 2024-04-25T16:50:47.202 INFO:teuthology.misc.health.smithi094.stderr:2024-04-25T16:50:47.204+0000 7f06b4ace640 1 -- 172.21.15.94:0/3416468289 wait complete. 2024-04-25T16:50:47.217 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T16:50:47.217 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T16:50:47.217 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T16:50:47.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T16:50:47.220 INFO:tasks.ceph.mds.a:Started 2024-04-25T16:50:47.220 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T16:50:47.220 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T16:50:47.222 INFO:tasks.ceph.mds.c:Started 2024-04-25T16:50:47.222 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T16:50:47.222 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T16:50:47.225 INFO:tasks.ceph.mds.b:Started 2024-04-25T16:50:47.225 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T16:50:47.225 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T16:50:47.227 INFO:tasks.ceph.mds.d:Started 2024-04-25T16:50:47.228 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T16:50:47.230 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:50:47.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.372+0000 7f9599c1c640 1 Processor -- start 2024-04-25T16:50:47.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.372+0000 7f9599c1c640 1 -- start start 2024-04-25T16:50:47.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f9599c1c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d1a50 0x7f95940cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f9599c1c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f959406d850 con 0x7f959406ce90 2024-04-25T16:50:47.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f9599c1c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f959406db30 con 0x7f959406d2a0 2024-04-25T16:50:47.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f9599c1c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f959406de10 con 0x7f95940d1a50 2024-04-25T16:50:47.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95937fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f959406ce90 0x7f95940d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:35170/0 (socket says 172.21.15.94:35170) 2024-04-25T16:50:47.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95937fe640 1 -- 172.21.15.94:0/2006651738 learned_addr learned my addr 172.21.15.94:0/2006651738 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:50:47.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f9592ffd640 1 --2- 172.21.15.94:0/2006651738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d1a50 0x7f95940cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 924162010 0 0) 0x7f959406d850 con 0x7f959406ce90 2024-04-25T16:50:47.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f957c003530 con 0x7f959406ce90 2024-04-25T16:50:47.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 15363943 0 0) 0x7f957c003530 con 0x7f959406ce90 2024-04-25T16:50:47.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d1a50 msgr2=0x7f95940cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:47.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 --2- 172.21.15.94:0/2006651738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d1a50 0x7f95940cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 >> v1:172.21.15.138:6789/0 conn(0x7f959406d2a0 legacy=0x7f95940cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959406e0f0 con 0x7f959406ce90 2024-04-25T16:50:47.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f9584001680 con 0x7f959406ce90 2024-04-25T16:50:47.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 >> v1:172.21.15.94:6789/0 conn(0x7f959406ce90 legacy=0x7f95940d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 --2- 172.21.15.94:0/2006651738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f957c0039f0 0x7f957c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 --2- 172.21.15.94:0/2006651738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f957c004810 0x7f957c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 --2- 172.21.15.94:0/2006651738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d1a50 0x7f957c0052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f959406de10 con 0x7f957c0039f0 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f957c00bb90 con 0x7f957c004810 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f957c00bd90 con 0x7f95940d1a50 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95937fe640 1 --2- 172.21.15.94:0/2006651738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f957c0039f0 0x7f957c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f9592ffd640 1 --2- 172.21.15.94:0/2006651738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f957c004810 0x7f957c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f9593fff640 1 --2- 172.21.15.94:0/2006651738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d1a50 0x7f957c0052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95937fe640 1 -- 172.21.15.94:0/2006651738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d1a50 msgr2=0x7f957c0052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95937fe640 1 --2- 172.21.15.94:0/2006651738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d1a50 0x7f957c0052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95937fe640 1 -- 172.21.15.94:0/2006651738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f957c004810 msgr2=0x7f957c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95937fe640 1 --2- 172.21.15.94:0/2006651738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f957c004810 0x7f957c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f95937fe640 1 -- 172.21.15.94:0/2006651738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c003530 con 0x7f957c0039f0 2024-04-25T16:50:47.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.376+0000 7f9593fff640 1 --2- 172.21.15.94:0/2006651738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d1a50 0x7f957c0052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:50:47.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.380+0000 7f95937fe640 1 --2- 172.21.15.94:0/2006651738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f957c0039f0 0x7f957c003e60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f958400c240 tx=0x7f958401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285b15a8c66a28e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:47.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.380+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584007d20 con 0x7f957c0039f0 2024-04-25T16:50:47.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.380+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95840a9070 con 0x7f957c0039f0 2024-04-25T16:50:47.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.380+0000 7f95927fc640 1 -- 172.21.15.94:0/2006651738 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584005d60 con 0x7f957c0039f0 2024-04-25T16:50:47.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.380+0000 7f9599c1c640 1 -- 172.21.15.94:0/2006651738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f957c0039f0 msgr2=0x7f957c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.380+0000 7f9599c1c640 1 --2- 172.21.15.94:0/2006651738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f957c0039f0 0x7f957c003e60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f958400c240 tx=0x7f958401cc80 comp rx=0 tx=0).stop 2024-04-25T16:50:47.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.380+0000 7f9593fff640 1 -- 172.21.15.94:0/2006651738 reap_dead start 2024-04-25T16:50:47.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.384+0000 7f9599c1c640 1 -- 172.21.15.94:0/2006651738 shutdown_connections 2024-04-25T16:50:47.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.384+0000 7f9599c1c640 1 -- 172.21.15.94:0/2006651738 >> 172.21.15.94:0/2006651738 conn(0x7f9594071150 msgr2=0x7f9594071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:47.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.384+0000 7f9599c1c640 1 -- 172.21.15.94:0/2006651738 shutdown_connections 2024-04-25T16:50:47.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.384+0000 7f9599c1c640 1 -- 172.21.15.94:0/2006651738 wait complete. 2024-04-25T16:50:47.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.388+0000 7f9599c1c640 1 Processor -- start 2024-04-25T16:50:47.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.388+0000 7f9599c1c640 1 -- start start 2024-04-25T16:50:47.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.388+0000 7f9599c1c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f957c0039f0 0x7f9594168870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.388+0000 7f9599c1c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f959415d0d0 0x7f959415dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.388+0000 7f9599c1c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9594139060 0x7f9594142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.388+0000 7f9599c1c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9594147520 con 0x7f957c0039f0 2024-04-25T16:50:47.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.388+0000 7f9599c1c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95940d0090 con 0x7f959415d0d0 2024-04-25T16:50:47.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.388+0000 7f9599c1c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f959416a150 con 0x7f9594139060 2024-04-25T16:50:47.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.388+0000 7f95937fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f957c0039f0 0x7f9594168870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.388+0000 7f95937fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f957c0039f0 0x7f9594168870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48084/0 (socket says 172.21.15.94:48084) 2024-04-25T16:50:47.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.388+0000 7f95937fe640 1 -- 172.21.15.94:0/3945042396 learned_addr learned my addr 172.21.15.94:0/3945042396 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:47.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.388+0000 7f9592ffd640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f959415d0d0 0x7f959415dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.388+0000 7f9593fff640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9594139060 0x7f9594142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f95937fe640 1 -- 172.21.15.94:0/3945042396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9594139060 msgr2=0x7f9594142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f95937fe640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9594139060 0x7f9594142570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f95937fe640 1 -- 172.21.15.94:0/3945042396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f959415d0d0 msgr2=0x7f959415dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f95937fe640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f959415d0d0 0x7f959415dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f95937fe640 1 -- 172.21.15.94:0/3945042396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9594167640 con 0x7f957c0039f0 2024-04-25T16:50:47.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f95937fe640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f957c0039f0 0x7f9594168870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f958401cab0 tx=0x7f957c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:47.390 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-04-25T16:50:47.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f957bfff640 1 -- 172.21.15.94:0/3945042396 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584005d40 con 0x7f957c0039f0 2024-04-25T16:50:47.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f957bfff640 1 -- 172.21.15.94:0/3945042396 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95840047f0 con 0x7f957c0039f0 2024-04-25T16:50:47.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f9599c1c640 1 -- 172.21.15.94:0/3945042396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9594139470 con 0x7f957c0039f0 2024-04-25T16:50:47.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f957bfff640 1 -- 172.21.15.94:0/3945042396 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584004bc0 con 0x7f957c0039f0 2024-04-25T16:50:47.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f9599c1c640 1 -- 172.21.15.94:0/3945042396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9594168070 con 0x7f957c0039f0 2024-04-25T16:50:47.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f9599c1c640 1 -- 172.21.15.94:0/3945042396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f959406a730 con 0x7f957c0039f0 2024-04-25T16:50:47.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f957bfff640 1 -- 172.21.15.94:0/3945042396 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9584007070 con 0x7f957c0039f0 2024-04-25T16:50:47.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f957bfff640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f958006f400 0x7f9580071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f9592ffd640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f958006f400 0x7f9580071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.392+0000 7f957bfff640 1 -- 172.21.15.94:0/3945042396 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f95841285a0 con 0x7f957c0039f0 2024-04-25T16:50:47.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.396+0000 7f9592ffd640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f958006f400 0x7f9580071910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9588009160 tx=0x7f9588008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.400+0000 7f957bfff640 1 -- 172.21.15.94:0/3945042396 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9584016990 con 0x7f957c0039f0 2024-04-25T16:50:47.410 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-04-25T16:50:47.442 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-04-25T16:50:47.443 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-04-25T16:50:47.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.552+0000 7f9599c1c640 1 -- 172.21.15.94:0/3945042396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95940d0c10 con 0x7f957c0039f0 2024-04-25T16:50:47.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.552+0000 7f957bfff640 1 -- 172.21.15.94:0/3945042396 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f95840f57e0 con 0x7f957c0039f0 2024-04-25T16:50:47.552 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:50:47.552 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":16,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:43.953393+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:50:47.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.556+0000 7f9599c1c640 1 -- 172.21.15.94:0/3945042396 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f958006f400 msgr2=0x7f9580071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.556+0000 7f9599c1c640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f958006f400 0x7f9580071910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9588009160 tx=0x7f9588008040 comp rx=0 tx=0).stop 2024-04-25T16:50:47.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.556+0000 7f9599c1c640 1 -- 172.21.15.94:0/3945042396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f957c0039f0 msgr2=0x7f9594168870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.556+0000 7f9599c1c640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f957c0039f0 0x7f9594168870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f958401cab0 tx=0x7f957c005840 comp rx=0 tx=0).stop 2024-04-25T16:50:47.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.556+0000 7f9599c1c640 1 -- 172.21.15.94:0/3945042396 shutdown_connections 2024-04-25T16:50:47.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.556+0000 7f9599c1c640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f958006f400 0x7f9580071910 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.556+0000 7f9599c1c640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9594139060 0x7f9594142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.556+0000 7f9599c1c640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f959415d0d0 0x7f959415dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.556+0000 7f9599c1c640 1 --2- 172.21.15.94:0/3945042396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f957c0039f0 0x7f9594168870 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.556+0000 7f9599c1c640 1 -- 172.21.15.94:0/3945042396 >> 172.21.15.94:0/3945042396 conn(0x7f9594071150 msgr2=0x7f95940d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:47.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.556+0000 7f9599c1c640 1 -- 172.21.15.94:0/3945042396 shutdown_connections 2024-04-25T16:50:47.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.556+0000 7f9599c1c640 1 -- 172.21.15.94:0/3945042396 wait complete. 2024-04-25T16:50:47.572 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:50:47.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:50:47.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.724+0000 7fb9be121640 1 Processor -- start 2024-04-25T16:50:47.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.724+0000 7fb9be121640 1 -- start start 2024-04-25T16:50:47.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9be121640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb9b8153900 0x7fb9b80ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9be121640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9b80ce6f0 0x7fb9b80cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9be121640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9b80ceb00 con 0x7fb9b80cfe80 2024-04-25T16:50:47.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9be121640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fb9b8153d10 con 0x7fb9b8153900 2024-04-25T16:50:47.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9be121640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b80ccb10 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9b6ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb9b8153900 0x7fb9b80ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9b6ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb9b8153900 0x7fb9b80ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48116/0 (socket says 172.21.15.94:48116) 2024-04-25T16:50:47.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9b6ffd640 1 -- 172.21.15.94:0/738052932 learned_addr learned my addr 172.21.15.94:0/738052932 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:47.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9b7fff640 1 --2- 172.21.15.94:0/738052932 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9b80ce6f0 0x7fb9b80cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9b6ffd640 1 -- 172.21.15.94:0/738052932 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9b80ce6f0 msgr2=0x7fb9b80cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:47.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9b6ffd640 1 --2- 172.21.15.94:0/738052932 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9b80ce6f0 0x7fb9b80cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9b6ffd640 1 -- 172.21.15.94:0/738052932 >> v1:172.21.15.138:6789/0 conn(0x7fb9b80cfe80 legacy=0x7fb9b806e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:47.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9b6ffd640 1 -- 172.21.15.94:0/738052932 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b815d3f0 con 0x7fb9b8153900 2024-04-25T16:50:47.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9b6ffd640 1 --2- 172.21.15.94:0/738052932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b8153900 0x7fb9b80ce1b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009870 tx=0x7fb9ac01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f16b5e83bcc716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:47.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9b67fc640 1 -- 172.21.15.94:0/738052932 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac0b0070 con 0x7fb9b8153900 2024-04-25T16:50:47.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9b67fc640 1 -- 172.21.15.94:0/738052932 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9ac0ab600 con 0x7fb9b8153900 2024-04-25T16:50:47.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.728+0000 7fb9b67fc640 1 -- 172.21.15.94:0/738052932 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac0ab9d0 con 0x7fb9b8153900 2024-04-25T16:50:47.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 -- 172.21.15.94:0/738052932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b8153900 msgr2=0x7fb9b80ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 --2- 172.21.15.94:0/738052932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b8153900 0x7fb9b80ce1b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009870 tx=0x7fb9ac01c340 comp rx=0 tx=0).stop 2024-04-25T16:50:47.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 -- 172.21.15.94:0/738052932 shutdown_connections 2024-04-25T16:50:47.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 --2- 172.21.15.94:0/738052932 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9b80ce6f0 0x7fb9b80cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 --2- 172.21.15.94:0/738052932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b8153900 0x7fb9b80ce1b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 -- 172.21.15.94:0/738052932 >> 172.21.15.94:0/738052932 conn(0x7fb9b80ced30 msgr2=0x7fb9b80cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:47.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 -- 172.21.15.94:0/738052932 shutdown_connections 2024-04-25T16:50:47.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 -- 172.21.15.94:0/738052932 wait complete. 2024-04-25T16:50:47.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 Processor -- start 2024-04-25T16:50:47.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 -- start start 2024-04-25T16:50:47.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b80ce6f0 0x7fb9b815fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b80cfe80 0x7fb9b8149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9b8153900 0x7fb9b8143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b806fd40 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b806fbc0 con 0x7fb9b80cfe80 2024-04-25T16:50:47.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9be121640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b806fec0 con 0x7fb9b8153900 2024-04-25T16:50:47.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b80ce6f0 0x7fb9b815fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b80ce6f0 0x7fb9b815fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48122/0 (socket says 172.21.15.94:48122) 2024-04-25T16:50:47.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9b77fe640 1 -- 172.21.15.94:0/1509299480 learned_addr learned my addr 172.21.15.94:0/1509299480 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:47.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.732+0000 7fb9b7fff640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9b8153900 0x7fb9b8143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb9b6ffd640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b80cfe80 0x7fb9b8149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb9b77fe640 1 -- 172.21.15.94:0/1509299480 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9b8153900 msgr2=0x7fb9b8143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb9b77fe640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9b8153900 0x7fb9b8143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb9b77fe640 1 -- 172.21.15.94:0/1509299480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b80cfe80 msgr2=0x7fb9b8149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb9b77fe640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b80cfe80 0x7fb9b8149f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb9b77fe640 1 -- 172.21.15.94:0/1509299480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b8147010 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb9b7fff640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9b8153900 0x7fb9b8143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:47.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb9b77fe640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b80ce6f0 0x7fb9b815fa20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800b410 tx=0x7fb9a800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:47.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb99ffff640 1 -- 172.21.15.94:0/1509299480 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8015030 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb99ffff640 1 -- 172.21.15.94:0/1509299480 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a8004020 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb99ffff640 1 -- 172.21.15.94:0/1509299480 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8004410 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb9be121640 1 -- 172.21.15.94:0/1509299480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b814d110 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb9be121640 1 -- 172.21.15.94:0/1509299480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b814d920 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb9be121640 1 -- 172.21.15.94:0/1509299480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b806a730 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb99ffff640 1 -- 172.21.15.94:0/1509299480 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb9a8018070 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb99ffff640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb9a406f400 0x7fb9a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:47.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb99ffff640 1 -- 172.21.15.94:0/1509299480 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb9a808eb20 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb9b6ffd640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb9a406f400 0x7fb9a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:47.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.736+0000 7fb9b6ffd640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb9a406f400 0x7fb9a4071910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0ad000 tx=0x7fb9ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:47.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.740+0000 7fb99ffff640 1 -- 172.21.15.94:0/1509299480 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb9a805bd60 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.900+0000 7fb9be121640 1 -- 172.21.15.94:0/1509299480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb9b80d1cd0 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.904+0000 7fb99ffff640 1 -- 172.21.15.94:0/1509299480 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fb9a805b660 con 0x7fb9b80ce6f0 2024-04-25T16:50:47.900 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:50:47.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.904+0000 7fb9be121640 1 -- 172.21.15.94:0/1509299480 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb9a406f400 msgr2=0x7fb9a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.904+0000 7fb9be121640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb9a406f400 0x7fb9a4071910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0ad000 tx=0x7fb9ac008040 comp rx=0 tx=0).stop 2024-04-25T16:50:47.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.904+0000 7fb9be121640 1 -- 172.21.15.94:0/1509299480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b80ce6f0 msgr2=0x7fb9b815fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:47.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.904+0000 7fb9be121640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b80ce6f0 0x7fb9b815fa20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800b410 tx=0x7fb9a800b8d0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.904+0000 7fb9be121640 1 -- 172.21.15.94:0/1509299480 shutdown_connections 2024-04-25T16:50:47.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.904+0000 7fb9be121640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb9a406f400 0x7fb9a4071910 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.904+0000 7fb9be121640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9b8153900 0x7fb9b8143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.904+0000 7fb9be121640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b80cfe80 0x7fb9b8149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.904+0000 7fb9be121640 1 --2- 172.21.15.94:0/1509299480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b80ce6f0 0x7fb9b815fa20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:47.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.904+0000 7fb9be121640 1 -- 172.21.15.94:0/1509299480 >> 172.21.15.94:0/1509299480 conn(0x7fb9b80ced30 msgr2=0x7fb9b80702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:47.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.904+0000 7fb9be121640 1 -- 172.21.15.94:0/1509299480 shutdown_connections 2024-04-25T16:50:47.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:47.904+0000 7fb9be121640 1 -- 172.21.15.94:0/1509299480 wait complete. 2024-04-25T16:50:47.921 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:50:48.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.060+0000 7f5566495640 1 Processor -- start 2024-04-25T16:50:48.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.060+0000 7f5566495640 1 -- start start 2024-04-25T16:50:48.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.064+0000 7f5566495640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f556015e990 0x7f5560165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.064+0000 7f5566495640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5560165b90 0x7f5560167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.064+0000 7f5566495640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f556006d610 con 0x7f5560070080 2024-04-25T16:50:48.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.064+0000 7f5566495640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f5560158550 con 0x7f5560165b90 2024-04-25T16:50:48.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.064+0000 7f5566495640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f556006f7d0 con 0x7f556015e990 2024-04-25T16:50:48.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.064+0000 7f5564a32640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5560165b90 0x7f5560167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.064+0000 7f5564a32640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5560165b90 0x7f5560167fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43346/0 (socket says 172.21.15.94:43346) 2024-04-25T16:50:48.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.064+0000 7f5564a32640 1 -- 172.21.15.94:0/3337455854 learned_addr learned my addr 172.21.15.94:0/3337455854 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:48.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.064+0000 7f555f7fe640 1 --2- 172.21.15.94:0/3337455854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f556015e990 0x7f5560165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.064+0000 7f5564a32640 1 -- 172.21.15.94:0/3337455854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f556015e990 msgr2=0x7f5560165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:48.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.064+0000 7f5564a32640 1 --2- 172.21.15.94:0/3337455854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f556015e990 0x7f5560165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.064+0000 7f5564a32640 1 -- 172.21.15.94:0/3337455854 >> v1:172.21.15.138:6789/0 conn(0x7f5560070080 legacy=0x7f556015e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:48.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.064+0000 7f5564a32640 1 -- 172.21.15.94:0/3337455854 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556006fc60 con 0x7f5560165b90 2024-04-25T16:50:48.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f5564a32640 1 --2- 172.21.15.94:0/3337455854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5560165b90 0x7f5560167fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f55601586d0 tx=0x7f555001c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc755714057d3648 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:48.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f555effd640 1 -- 172.21.15.94:0/3337455854 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500b0070 con 0x7f5560165b90 2024-04-25T16:50:48.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f555effd640 1 -- 172.21.15.94:0/3337455854 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55500ab600 con 0x7f5560165b90 2024-04-25T16:50:48.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f555effd640 1 -- 172.21.15.94:0/3337455854 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500ab9d0 con 0x7f5560165b90 2024-04-25T16:50:48.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f5566495640 1 -- 172.21.15.94:0/3337455854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5560165b90 msgr2=0x7f5560167fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f5566495640 1 --2- 172.21.15.94:0/3337455854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5560165b90 0x7f5560167fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f55601586d0 tx=0x7f555001c300 comp rx=0 tx=0).stop 2024-04-25T16:50:48.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f5566495640 1 -- 172.21.15.94:0/3337455854 shutdown_connections 2024-04-25T16:50:48.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f5566495640 1 --2- 172.21.15.94:0/3337455854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5560165b90 0x7f5560167fb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f5566495640 1 --2- 172.21.15.94:0/3337455854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f556015e990 0x7f5560165650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f5566495640 1 -- 172.21.15.94:0/3337455854 >> 172.21.15.94:0/3337455854 conn(0x7f55600b47f0 msgr2=0x7f55600b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:48.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f5566495640 1 -- 172.21.15.94:0/3337455854 shutdown_connections 2024-04-25T16:50:48.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f5566495640 1 -- 172.21.15.94:0/3337455854 wait complete. 2024-04-25T16:50:48.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f5566495640 1 Processor -- start 2024-04-25T16:50:48.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.068+0000 7f5566495640 1 -- start start 2024-04-25T16:50:48.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f5566495640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5560070080 0x7f556014ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f5566495640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f556015e990 0x7f556015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f5566495640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5560165b90 0x7f556013d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f5566495640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55600ae6c0 con 0x7f556015e990 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f5566495640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f556006f460 con 0x7f5560070080 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f5566495640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5560070490 con 0x7f5560165b90 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f556015e990 0x7f556015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f556015e990 0x7f556015fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43352/0 (socket says 172.21.15.94:43352) 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555f7fe640 1 -- 172.21.15.94:0/1158116338 learned_addr learned my addr 172.21.15.94:0/1158116338 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555ffff640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5560070080 0x7f556014ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f5564a32640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5560165b90 0x7f556013d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555f7fe640 1 -- 172.21.15.94:0/1158116338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5560165b90 msgr2=0x7f556013d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555f7fe640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5560165b90 0x7f556013d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555f7fe640 1 -- 172.21.15.94:0/1158116338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5560070080 msgr2=0x7f556014ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555f7fe640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5560070080 0x7f556014ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555f7fe640 1 -- 172.21.15.94:0/1158116338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55600ce340 con 0x7f556015e990 2024-04-25T16:50:48.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555f7fe640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f556015e990 0x7f556015fc30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f554800d5f0 tx=0x7f554800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:48.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555cff9640 1 -- 172.21.15.94:0/1158116338 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55480125f0 con 0x7f556015e990 2024-04-25T16:50:48.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555cff9640 1 -- 172.21.15.94:0/1158116338 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5548019070 con 0x7f556015e990 2024-04-25T16:50:48.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f5566495640 1 -- 172.21.15.94:0/1158116338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55600d04b0 con 0x7f556015e990 2024-04-25T16:50:48.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555cff9640 1 -- 172.21.15.94:0/1158116338 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55480129b0 con 0x7f556015e990 2024-04-25T16:50:48.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f5566495640 1 -- 172.21.15.94:0/1158116338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556015f430 con 0x7f556015e990 2024-04-25T16:50:48.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f5566495640 1 -- 172.21.15.94:0/1158116338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5560076830 con 0x7f556015e990 2024-04-25T16:50:48.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.072+0000 7f555cff9640 1 -- 172.21.15.94:0/1158116338 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5548012b50 con 0x7f556015e990 2024-04-25T16:50:48.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.076+0000 7f555cff9640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f554006f430 0x7f5540071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.076+0000 7f555cff9640 1 -- 172.21.15.94:0/1158116338 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f554808f370 con 0x7f556015e990 2024-04-25T16:50:48.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.076+0000 7f555ffff640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f554006f430 0x7f5540071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.076+0000 7f555ffff640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f554006f430 0x7f5540071940 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f55600b50c0 tx=0x7f5558007240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:48.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.080+0000 7f555cff9640 1 -- 172.21.15.94:0/1158116338 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f554805c5b0 con 0x7f556015e990 2024-04-25T16:50:48.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.260+0000 7f5566495640 1 -- 172.21.15.94:0/1158116338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55601508f0 con 0x7f556015e990 2024-04-25T16:50:48.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.260+0000 7f555cff9640 1 -- 172.21.15.94:0/1158116338 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f554805beb0 con 0x7f556015e990 2024-04-25T16:50:48.259 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:50:48.259 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2237413225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2237413225},{"type":"v1","addr":"172.21.15.94:6837","nonce":2237413225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T16:50:48.259 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 2 2024-04-25T16:50:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.264+0000 7f5566495640 1 -- 172.21.15.94:0/1158116338 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f554006f430 msgr2=0x7f5540071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.264+0000 7f5566495640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f554006f430 0x7f5540071940 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f55600b50c0 tx=0x7f5558007240 comp rx=0 tx=0).stop 2024-04-25T16:50:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.264+0000 7f5566495640 1 -- 172.21.15.94:0/1158116338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f556015e990 msgr2=0x7f556015fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.264+0000 7f5566495640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f556015e990 0x7f556015fc30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f554800d5f0 tx=0x7f554800dab0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.264+0000 7f5566495640 1 -- 172.21.15.94:0/1158116338 shutdown_connections 2024-04-25T16:50:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.264+0000 7f5566495640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f554006f430 0x7f5540071940 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.264+0000 7f5566495640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5560165b90 0x7f556013d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.264+0000 7f5566495640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f556015e990 0x7f556015fc30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.264+0000 7f5566495640 1 --2- 172.21.15.94:0/1158116338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5560070080 0x7f556014ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.264+0000 7f5566495640 1 -- 172.21.15.94:0/1158116338 >> 172.21.15.94:0/1158116338 conn(0x7f55600b47f0 msgr2=0x7f556006d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.264+0000 7f5566495640 1 -- 172.21.15.94:0/1158116338 shutdown_connections 2024-04-25T16:50:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:48.264+0000 7f5566495640 1 -- 172.21.15.94:0/1158116338 wait complete. 2024-04-25T16:50:50.278 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:50:50.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.440+0000 7f3f444c4640 1 Processor -- start 2024-04-25T16:50:50.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.440+0000 7f3f444c4640 1 -- start start 2024-04-25T16:50:50.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.440+0000 7f3f444c4640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3f3c06e960 0x7f3f3c072670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.440+0000 7f3f444c4640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3c072fc0 con 0x7f3f3c072bb0 2024-04-25T16:50:50.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.440+0000 7f3f444c4640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3c0d3230 con 0x7f3f3c160ae0 2024-04-25T16:50:50.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.440+0000 7f3f444c4640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3f3c0d2990 con 0x7f3f3c06e960 2024-04-25T16:50:50.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.440+0000 7f3f41a5f640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f3f3c072bb0 0x7f3f3c0ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:36672/0 (socket says 172.21.15.94:36672) 2024-04-25T16:50:50.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.440+0000 7f3f41a5f640 1 -- 172.21.15.94:0/3109555499 learned_addr learned my addr 172.21.15.94:0/3109555499 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:50:50.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.440+0000 7f3f42260640 1 --2- 172.21.15.94:0/3109555499 >> v2:172.21.15.138:3300/0 conn(0x7f3f3c06e960 0x7f3f3c072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:50.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.440+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3966888043 0 0) 0x7f3f30001120 con 0x7f3f3c072bb0 2024-04-25T16:50:50.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.440+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f28003530 con 0x7f3f3c072bb0 2024-04-25T16:50:50.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4144720929 0 0) 0x7f3f28003530 con 0x7f3f3c072bb0 2024-04-25T16:50:50.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 >> v2:172.21.15.138:3300/0 conn(0x7f3f3c06e960 msgr2=0x7f3f3c072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:50.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 --2- 172.21.15.94:0/3109555499 >> v2:172.21.15.138:3300/0 conn(0x7f3f3c06e960 0x7f3f3c072670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 >> v1:172.21.15.138:6789/0 conn(0x7f3f3c160ae0 legacy=0x7f3f3c0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:50.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c0d2c70 con 0x7f3f3c072bb0 2024-04-25T16:50:50.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f3f30001850 con 0x7f3f3c072bb0 2024-04-25T16:50:50.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 >> v1:172.21.15.94:6789/0 conn(0x7f3f3c072bb0 legacy=0x7f3f3c0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:50.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 --2- 172.21.15.94:0/3109555499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f280039c0 0x7f3f28003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 --2- 172.21.15.94:0/3109555499 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f28004810 0x7f3f28004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 --2- 172.21.15.94:0/3109555499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f28005260 0x7f3f2800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c0d2990 con 0x7f3f280039c0 2024-04-25T16:50:50.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2800c1f0 con 0x7f3f28005260 2024-04-25T16:50:50.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2800c3f0 con 0x7f3f28004810 2024-04-25T16:50:50.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f42260640 1 --2- 172.21.15.94:0/3109555499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f280039c0 0x7f3f28003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:50.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f42a61640 1 --2- 172.21.15.94:0/3109555499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f28005260 0x7f3f2800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:50.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f42260640 1 -- 172.21.15.94:0/3109555499 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f28004810 msgr2=0x7f3f28004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:50.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f42260640 1 --2- 172.21.15.94:0/3109555499 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f28004810 0x7f3f28004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f42260640 1 -- 172.21.15.94:0/3109555499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f28005260 msgr2=0x7f3f2800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:50.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f42260640 1 --2- 172.21.15.94:0/3109555499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f28005260 0x7f3f2800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f42260640 1 -- 172.21.15.94:0/3109555499 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f28003530 con 0x7f3f280039c0 2024-04-25T16:50:50.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f42a61640 1 --2- 172.21.15.94:0/3109555499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f28005260 0x7f3f2800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:50.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f42a61640 1 -- 172.21.15.94:0/3109555499 reap_dead start 2024-04-25T16:50:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f42260640 1 --2- 172.21.15.94:0/3109555499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f280039c0 0x7f3f28003e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009f90 tx=0x7f3f3401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fd8ee30769ccc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f340b2070 con 0x7f3f280039c0 2024-04-25T16:50:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f340a5b10 con 0x7f3f280039c0 2024-04-25T16:50:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f4125e640 1 -- 172.21.15.94:0/3109555499 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34009c10 con 0x7f3f280039c0 2024-04-25T16:50:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f444c4640 1 -- 172.21.15.94:0/3109555499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f280039c0 msgr2=0x7f3f28003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f444c4640 1 --2- 172.21.15.94:0/3109555499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f280039c0 0x7f3f28003e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009f90 tx=0x7f3f3401cc10 comp rx=0 tx=0).stop 2024-04-25T16:50:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f444c4640 1 -- 172.21.15.94:0/3109555499 shutdown_connections 2024-04-25T16:50:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f444c4640 1 --2- 172.21.15.94:0/3109555499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f280039c0 0x7f3f28003e30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.444+0000 7f3f444c4640 1 -- 172.21.15.94:0/3109555499 >> 172.21.15.94:0/3109555499 conn(0x7f3f3c0d3850 msgr2=0x7f3f3c0d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f444c4640 1 -- 172.21.15.94:0/3109555499 shutdown_connections 2024-04-25T16:50:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f444c4640 1 -- 172.21.15.94:0/3109555499 wait complete. 2024-04-25T16:50:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f444c4640 1 Processor -- start 2024-04-25T16:50:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f444c4640 1 -- start start 2024-04-25T16:50:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f444c4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f280039c0 0x7f3f3c151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f444c4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f3c0cc1f0 0x7f3f3c0cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f444c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f3c162590 0x7f3f3c1629a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f444c4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c06eef0 con 0x7f3f3c162590 2024-04-25T16:50:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f444c4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c06ed70 con 0x7f3f3c0cc1f0 2024-04-25T16:50:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f444c4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3c1602e0 con 0x7f3f280039c0 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f42a61640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f3c162590 0x7f3f3c1629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f42a61640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f3c162590 0x7f3f3c1629a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43360/0 (socket says 172.21.15.94:43360) 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f42a61640 1 -- 172.21.15.94:0/3088975616 learned_addr learned my addr 172.21.15.94:0/3088975616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f41a5f640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f3c0cc1f0 0x7f3f3c0cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f42260640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f280039c0 0x7f3f3c151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f42a61640 1 -- 172.21.15.94:0/3088975616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f280039c0 msgr2=0x7f3f3c151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f42a61640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f280039c0 0x7f3f3c151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f42a61640 1 -- 172.21.15.94:0/3088975616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f3c0cc1f0 msgr2=0x7f3f3c0cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f42a61640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f3c0cc1f0 0x7f3f3c0cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f42a61640 1 -- 172.21.15.94:0/3088975616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c162ee0 con 0x7f3f3c162590 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.448+0000 7f3f42260640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f280039c0 0x7f3f3c151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.452+0000 7f3f42a61640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f3c162590 0x7f3f3c1629a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0735a0 tx=0x7f3f3800d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.452+0000 7f3f2effd640 1 -- 172.21.15.94:0/3088975616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38018070 con 0x7f3f3c162590 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.452+0000 7f3f2effd640 1 -- 172.21.15.94:0/3088975616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f38003600 con 0x7f3f3c162590 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.452+0000 7f3f2effd640 1 -- 172.21.15.94:0/3088975616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f380039f0 con 0x7f3f3c162590 2024-04-25T16:50:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.452+0000 7f3f444c4640 1 -- 172.21.15.94:0/3088975616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3c0c7140 con 0x7f3f3c162590 2024-04-25T16:50:50.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.452+0000 7f3f444c4640 1 -- 172.21.15.94:0/3088975616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3c0c7700 con 0x7f3f3c162590 2024-04-25T16:50:50.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.452+0000 7f3f444c4640 1 -- 172.21.15.94:0/3088975616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f3c06a730 con 0x7f3f3c162590 2024-04-25T16:50:50.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.452+0000 7f3f2effd640 1 -- 172.21.15.94:0/3088975616 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3f3801b070 con 0x7f3f3c162590 2024-04-25T16:50:50.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.452+0000 7f3f2effd640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3f2006f430 0x7f3f20071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.452+0000 7f3f2effd640 1 -- 172.21.15.94:0/3088975616 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3f3808e460 con 0x7f3f3c162590 2024-04-25T16:50:50.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.452+0000 7f3f42260640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3f2006f430 0x7f3f20071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:50.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.452+0000 7f3f42260640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3f2006f430 0x7f3f20071940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f340af000 tx=0x7f3f340a7040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.460+0000 7f3f2effd640 1 -- 172.21.15.94:0/3088975616 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3f3c162ee0 con 0x7f3f3c162590 2024-04-25T16:50:50.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.652+0000 7f3f444c4640 1 -- 172.21.15.94:0/3088975616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f3c0710b0 con 0x7f3f3c162590 2024-04-25T16:50:50.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.652+0000 7f3f2effd640 1 -- 172.21.15.94:0/3088975616 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7f3f3805b730 con 0x7f3f3c162590 2024-04-25T16:50:50.652 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:50:50.652 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2352186041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2352186041},{"type":"v1","addr":"172.21.15.138:6833","nonce":2352186041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3210927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3210927873},{"type":"v1","addr":"172.21.15.94:6835","nonce":3210927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2237413225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2237413225},{"type":"v1","addr":"172.21.15.94:6837","nonce":2237413225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3652545132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3652545132},{"type":"v1","addr":"172.21.15.138:6835","nonce":3652545132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T16:50:50.652 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 3 2024-04-25T16:50:50.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.656+0000 7f3f444c4640 1 -- 172.21.15.94:0/3088975616 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3f2006f430 msgr2=0x7f3f20071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:50.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.656+0000 7f3f444c4640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3f2006f430 0x7f3f20071940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f340af000 tx=0x7f3f340a7040 comp rx=0 tx=0).stop 2024-04-25T16:50:50.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.656+0000 7f3f444c4640 1 -- 172.21.15.94:0/3088975616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f3c162590 msgr2=0x7f3f3c1629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:50.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.656+0000 7f3f444c4640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f3c162590 0x7f3f3c1629a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0735a0 tx=0x7f3f3800d810 comp rx=0 tx=0).stop 2024-04-25T16:50:50.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.656+0000 7f3f444c4640 1 -- 172.21.15.94:0/3088975616 shutdown_connections 2024-04-25T16:50:50.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.656+0000 7f3f444c4640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f3c162590 0x7f3f3c1629a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.656+0000 7f3f444c4640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f3c0cc1f0 0x7f3f3c0cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.656+0000 7f3f444c4640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3f2006f430 0x7f3f20071940 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.656+0000 7f3f444c4640 1 --2- 172.21.15.94:0/3088975616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f280039c0 0x7f3f3c151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.656+0000 7f3f444c4640 1 -- 172.21.15.94:0/3088975616 >> 172.21.15.94:0/3088975616 conn(0x7f3f3c0d3850 msgr2=0x7f3f3c071920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:50.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.656+0000 7f3f444c4640 1 -- 172.21.15.94:0/3088975616 shutdown_connections 2024-04-25T16:50:50.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.656+0000 7f3f444c4640 1 -- 172.21.15.94:0/3088975616 wait complete. 2024-04-25T16:50:50.673 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:50:50.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.824+0000 7f7c42f4d640 1 Processor -- start 2024-04-25T16:50:50.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c42f4d640 1 -- start start 2024-04-25T16:50:50.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c42f4d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7c3c168350 0x7f7c3c06ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c42f4d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7c3c073560 0x7f7c3c06f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c42f4d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c3c06d1b0 con 0x7f7c3c073150 2024-04-25T16:50:50.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c42f4d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7c3c0ae170 con 0x7f7c3c073560 2024-04-25T16:50:50.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c42f4d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7c3c06ddc0 con 0x7f7c3c168350 2024-04-25T16:50:50.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c414ea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7c3c073560 0x7f7c3c06f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:50.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c414ea640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7c3c073560 0x7f7c3c06f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43368/0 (socket says 172.21.15.94:43368) 2024-04-25T16:50:50.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c414ea640 1 -- 172.21.15.94:0/3978617765 learned_addr learned my addr 172.21.15.94:0/3978617765 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:50.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c3bfff640 1 --2- 172.21.15.94:0/3978617765 >> v2:172.21.15.138:3300/0 conn(0x7f7c3c168350 0x7f7c3c06ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:50.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c414ea640 1 -- 172.21.15.94:0/3978617765 >> v2:172.21.15.138:3300/0 conn(0x7f7c3c168350 msgr2=0x7f7c3c06ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:50.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c414ea640 1 --2- 172.21.15.94:0/3978617765 >> v2:172.21.15.138:3300/0 conn(0x7f7c3c168350 0x7f7c3c06ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c414ea640 1 -- 172.21.15.94:0/3978617765 >> v1:172.21.15.138:6789/0 conn(0x7f7c3c073150 legacy=0x7f7c3c167c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:50.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.828+0000 7f7c414ea640 1 -- 172.21.15.94:0/3978617765 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c06d490 con 0x7f7c3c073560 2024-04-25T16:50:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.832+0000 7f7c414ea640 1 --2- 172.21.15.94:0/3978617765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c3c073560 0x7f7c3c06f220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0b41a0 tx=0x7f7c3401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7151c7139b1c13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.832+0000 7f7c3b7fe640 1 -- 172.21.15.94:0/3978617765 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c340b4070 con 0x7f7c3c073560 2024-04-25T16:50:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.832+0000 7f7c3b7fe640 1 -- 172.21.15.94:0/3978617765 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c340af400 con 0x7f7c3c073560 2024-04-25T16:50:50.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.832+0000 7f7c3b7fe640 1 -- 172.21.15.94:0/3978617765 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c340af7d0 con 0x7f7c3c073560 2024-04-25T16:50:50.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.832+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3978617765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c3c073560 msgr2=0x7f7c3c06f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:50.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.832+0000 7f7c42f4d640 1 --2- 172.21.15.94:0/3978617765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c3c073560 0x7f7c3c06f220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0b41a0 tx=0x7f7c3401f1e0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.832+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3978617765 shutdown_connections 2024-04-25T16:50:50.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.832+0000 7f7c42f4d640 1 --2- 172.21.15.94:0/3978617765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c3c073560 0x7f7c3c06f220 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.832+0000 7f7c42f4d640 1 --2- 172.21.15.94:0/3978617765 >> v2:172.21.15.138:3300/0 conn(0x7f7c3c168350 0x7f7c3c06ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.832+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3978617765 >> 172.21.15.94:0/3978617765 conn(0x7f7c3c0b47f0 msgr2=0x7f7c3c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:50.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.832+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3978617765 shutdown_connections 2024-04-25T16:50:50.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.832+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3978617765 wait complete. 2024-04-25T16:50:50.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c42f4d640 1 Processor -- start 2024-04-25T16:50:50.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c42f4d640 1 -- start start 2024-04-25T16:50:50.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c42f4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c3c073150 0x7f7c3c155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c42f4d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c3c073560 0x7f7c3c15a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c42f4d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c3c1438f0 0x7f7c3c143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c42f4d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c1689a0 con 0x7f7c3c073150 2024-04-25T16:50:50.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c42f4d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c06da50 con 0x7f7c3c1438f0 2024-04-25T16:50:50.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c42f4d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3c06f760 con 0x7f7c3c073560 2024-04-25T16:50:50.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c40ce9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c3c073150 0x7f7c3c155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:50.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c40ce9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c3c073150 0x7f7c3c155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43384/0 (socket says 172.21.15.94:43384) 2024-04-25T16:50:50.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c40ce9640 1 -- 172.21.15.94:0/3540467013 learned_addr learned my addr 172.21.15.94:0/3540467013 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:50.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c3bfff640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c3c073560 0x7f7c3c15a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:50.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c40ce9640 1 -- 172.21.15.94:0/3540467013 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c3c073560 msgr2=0x7f7c3c15a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:50.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c414ea640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c3c1438f0 0x7f7c3c143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:50.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c40ce9640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c3c073560 0x7f7c3c15a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c40ce9640 1 -- 172.21.15.94:0/3540467013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c3c1438f0 msgr2=0x7f7c3c143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:50.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c40ce9640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c3c1438f0 0x7f7c3c143d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:50.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c40ce9640 1 -- 172.21.15.94:0/3540467013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c154bf0 con 0x7f7c3c073150 2024-04-25T16:50:50.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c40ce9640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c3c073150 0x7f7c3c155b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00c3c0 tx=0x7f7c2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:50.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c397fa640 1 -- 172.21.15.94:0/3540467013 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c017070 con 0x7f7c3c073150 2024-04-25T16:50:50.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c397fa640 1 -- 172.21.15.94:0/3540467013 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2c0052d0 con 0x7f7c3c073150 2024-04-25T16:50:50.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3540467013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3c142850 con 0x7f7c3c073150 2024-04-25T16:50:50.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c397fa640 1 -- 172.21.15.94:0/3540467013 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c003840 con 0x7f7c3c073150 2024-04-25T16:50:50.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3540467013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3c142e10 con 0x7f7c3c073150 2024-04-25T16:50:50.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.836+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3540467013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c3c076830 con 0x7f7c3c073150 2024-04-25T16:50:50.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.840+0000 7f7c397fa640 1 -- 172.21.15.94:0/3540467013 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7c2c01a070 con 0x7f7c3c073150 2024-04-25T16:50:50.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.840+0000 7f7c397fa640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7c2806f400 0x7f7c28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:50.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.840+0000 7f7c3bfff640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7c2806f400 0x7f7c28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:50.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.840+0000 7f7c397fa640 1 -- 172.21.15.94:0/3540467013 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7c2c08e650 con 0x7f7c3c073150 2024-04-25T16:50:50.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.840+0000 7f7c3bfff640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7c2806f400 0x7f7c28071910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c15de10 tx=0x7f7c30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:50.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:50.844+0000 7f7c397fa640 1 -- 172.21.15.94:0/3540467013 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7c2c021020 con 0x7f7c3c073150 2024-04-25T16:50:51.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.004+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3540467013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c3c06e9c0 con 0x7f7c3c073150 2024-04-25T16:50:51.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.004+0000 7f7c397fa640 1 -- 172.21.15.94:0/3540467013 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f7c2c005480 con 0x7f7c3c073150 2024-04-25T16:50:51.001 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:50:51.002 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":16,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:43.953393+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:50:51.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.004+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3540467013 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7c2806f400 msgr2=0x7f7c28071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.004+0000 7f7c42f4d640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7c2806f400 0x7f7c28071910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c15de10 tx=0x7f7c30008040 comp rx=0 tx=0).stop 2024-04-25T16:50:51.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.004+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3540467013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c3c073150 msgr2=0x7f7c3c155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.004+0000 7f7c42f4d640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c3c073150 0x7f7c3c155b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00c3c0 tx=0x7f7c2c00c880 comp rx=0 tx=0).stop 2024-04-25T16:50:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.008+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3540467013 shutdown_connections 2024-04-25T16:50:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.008+0000 7f7c42f4d640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c3c1438f0 0x7f7c3c143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.008+0000 7f7c42f4d640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7c2806f400 0x7f7c28071910 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.008+0000 7f7c42f4d640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c3c073560 0x7f7c3c15a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.008+0000 7f7c42f4d640 1 --2- 172.21.15.94:0/3540467013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c3c073150 0x7f7c3c155b40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.008+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3540467013 >> 172.21.15.94:0/3540467013 conn(0x7f7c3c0b47f0 msgr2=0x7f7c3c167bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.008+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3540467013 shutdown_connections 2024-04-25T16:50:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.008+0000 7f7c42f4d640 1 -- 172.21.15.94:0/3540467013 wait complete. 2024-04-25T16:50:51.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:50:51.021 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:50:51.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.176+0000 7f2aee9ff640 1 Processor -- start 2024-04-25T16:50:51.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.176+0000 7f2aee9ff640 1 -- start start 2024-04-25T16:50:51.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.176+0000 7f2aee9ff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2ae806f470 0x7f2ae815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.176+0000 7f2aee9ff640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae8070630 con 0x7f2ae80b0420 2024-04-25T16:50:51.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.176+0000 7f2aee9ff640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae80ae170 con 0x7f2ae815e160 2024-04-25T16:50:51.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.176+0000 7f2aee9ff640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae80aeaa0 con 0x7f2ae806f470 2024-04-25T16:50:51.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.176+0000 7f2ae77fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2ae806f470 0x7f2ae815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.176+0000 7f2ae77fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2ae806f470 0x7f2ae815dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43396/0 (socket says 172.21.15.94:43396) 2024-04-25T16:50:51.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.176+0000 7f2ae77fe640 1 -- 172.21.15.94:0/1456477506 learned_addr learned my addr 172.21.15.94:0/1456477506 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:51.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.176+0000 7f2ae77fe640 1 -- 172.21.15.94:0/1456477506 >> v1:172.21.15.138:6789/0 conn(0x7f2ae815e160 legacy=0x7f2ae815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:51.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.176+0000 7f2ae77fe640 1 -- 172.21.15.94:0/1456477506 >> v1:172.21.15.94:6789/0 conn(0x7f2ae80b0420 legacy=0x7f2ae806ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.176+0000 7f2ae77fe640 1 -- 172.21.15.94:0/1456477506 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae80ae450 con 0x7f2ae806f470 2024-04-25T16:50:51.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.180+0000 7f2ae77fe640 1 --2- 172.21.15.94:0/1456477506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae806f470 0x7f2ae815dc20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009870 tx=0x7f2ad401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce600a5ecd27776 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:51.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.180+0000 7f2ae6ffd640 1 -- 172.21.15.94:0/1456477506 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40b0070 con 0x7f2ae806f470 2024-04-25T16:50:51.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.180+0000 7f2ae6ffd640 1 -- 172.21.15.94:0/1456477506 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad40ab600 con 0x7f2ae806f470 2024-04-25T16:50:51.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.180+0000 7f2ae6ffd640 1 -- 172.21.15.94:0/1456477506 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40ab9d0 con 0x7f2ae806f470 2024-04-25T16:50:51.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.180+0000 7f2aee9ff640 1 -- 172.21.15.94:0/1456477506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae806f470 msgr2=0x7f2ae815dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.180+0000 7f2aee9ff640 1 --2- 172.21.15.94:0/1456477506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae806f470 0x7f2ae815dc20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009870 tx=0x7f2ad401c4c0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.180+0000 7f2aee9ff640 1 -- 172.21.15.94:0/1456477506 shutdown_connections 2024-04-25T16:50:51.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.180+0000 7f2aee9ff640 1 --2- 172.21.15.94:0/1456477506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae806f470 0x7f2ae815dc20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.180+0000 7f2aee9ff640 1 -- 172.21.15.94:0/1456477506 >> 172.21.15.94:0/1456477506 conn(0x7f2ae80b47f0 msgr2=0x7f2ae80b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:51.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.180+0000 7f2aee9ff640 1 -- 172.21.15.94:0/1456477506 shutdown_connections 2024-04-25T16:50:51.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.180+0000 7f2aee9ff640 1 -- 172.21.15.94:0/1456477506 wait complete. 2024-04-25T16:50:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aee9ff640 1 Processor -- start 2024-04-25T16:50:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aee9ff640 1 -- start start 2024-04-25T16:50:51.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aee9ff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ae80b0420 0x7f2ae80cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aee9ff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ae815e160 0x7f2ae8150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aee9ff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae81fcf40 0x7f2ae81fd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aee9ff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae816b880 con 0x7f2ae81fcf40 2024-04-25T16:50:51.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aee9ff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae816b700 con 0x7f2ae80b0420 2024-04-25T16:50:51.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aee9ff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae816ba00 con 0x7f2ae815e160 2024-04-25T16:50:51.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aecf9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae81fcf40 0x7f2ae81fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aecf9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae81fcf40 0x7f2ae81fd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43412/0 (socket says 172.21.15.94:43412) 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aecf9c640 1 -- 172.21.15.94:0/3559060107 learned_addr learned my addr 172.21.15.94:0/3559060107 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2ae7fff640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ae80b0420 0x7f2ae80cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2ae77fe640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ae815e160 0x7f2ae8150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aecf9c640 1 -- 172.21.15.94:0/3559060107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ae815e160 msgr2=0x7f2ae8150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aecf9c640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ae815e160 0x7f2ae8150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aecf9c640 1 -- 172.21.15.94:0/3559060107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ae80b0420 msgr2=0x7f2ae80cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aecf9c640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ae80b0420 0x7f2ae80cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aecf9c640 1 -- 172.21.15.94:0/3559060107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae81f99d0 con 0x7f2ae81fcf40 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2ae7fff640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ae80b0420 0x7f2ae80cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aecf9c640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae81fcf40 0x7f2ae81fd350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000b970 tx=0x7f2ae000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2ae4ff9640 1 -- 172.21.15.94:0/3559060107 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0019070 con 0x7f2ae81fcf40 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2ae4ff9640 1 -- 172.21.15.94:0/3559060107 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae000cb40 con 0x7f2ae81fcf40 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aee9ff640 1 -- 172.21.15.94:0/3559060107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae81f9cb0 con 0x7f2ae81fcf40 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2ae4ff9640 1 -- 172.21.15.94:0/3559060107 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0014440 con 0x7f2ae81fcf40 2024-04-25T16:50:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aee9ff640 1 -- 172.21.15.94:0/3559060107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae81fca40 con 0x7f2ae81fcf40 2024-04-25T16:50:51.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.184+0000 7f2aee9ff640 1 -- 172.21.15.94:0/3559060107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae8076830 con 0x7f2ae81fcf40 2024-04-25T16:50:51.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.188+0000 7f2ae4ff9640 1 -- 172.21.15.94:0/3559060107 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2ae001c070 con 0x7f2ae81fcf40 2024-04-25T16:50:51.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.188+0000 7f2ae4ff9640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2ad006f400 0x7f2ad0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.188+0000 7f2ae4ff9640 1 -- 172.21.15.94:0/3559060107 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2ae008ebe0 con 0x7f2ae81fcf40 2024-04-25T16:50:51.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.188+0000 7f2ae7fff640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2ad006f400 0x7f2ad0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.188+0000 7f2ae7fff640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2ad006f400 0x7f2ad0071910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80cd2f0 tx=0x7f2ad8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:51.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.192+0000 7f2ae4ff9640 1 -- 172.21.15.94:0/3559060107 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ae005be20 con 0x7f2ae81fcf40 2024-04-25T16:50:51.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2aee9ff640 1 -- 172.21.15.94:0/3559060107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2ae81f9760 con 0x7f2ae81fcf40 2024-04-25T16:50:51.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2ae4ff9640 1 -- 172.21.15.94:0/3559060107 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f2ae005b720 con 0x7f2ae81fcf40 2024-04-25T16:50:51.345 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:50:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2aee9ff640 1 -- 172.21.15.94:0/3559060107 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2ad006f400 msgr2=0x7f2ad0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2aee9ff640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2ad006f400 0x7f2ad0071910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80cd2f0 tx=0x7f2ad8009040 comp rx=0 tx=0).stop 2024-04-25T16:50:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2aee9ff640 1 -- 172.21.15.94:0/3559060107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae81fcf40 msgr2=0x7f2ae81fd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2aee9ff640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae81fcf40 0x7f2ae81fd350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000b970 tx=0x7f2ae000be30 comp rx=0 tx=0).stop 2024-04-25T16:50:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2aee9ff640 1 -- 172.21.15.94:0/3559060107 shutdown_connections 2024-04-25T16:50:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2aee9ff640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae81fcf40 0x7f2ae81fd350 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2aee9ff640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2ad006f400 0x7f2ad0071910 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2aee9ff640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ae815e160 0x7f2ae8150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2aee9ff640 1 --2- 172.21.15.94:0/3559060107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ae80b0420 0x7f2ae80cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2aee9ff640 1 -- 172.21.15.94:0/3559060107 >> 172.21.15.94:0/3559060107 conn(0x7f2ae80b47f0 msgr2=0x7f2ae80b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:51.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2aee9ff640 1 -- 172.21.15.94:0/3559060107 shutdown_connections 2024-04-25T16:50:51.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.348+0000 7f2aee9ff640 1 -- 172.21.15.94:0/3559060107 wait complete. 2024-04-25T16:50:51.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T16:50:51.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.500+0000 7fe53d8af640 1 Processor -- start 2024-04-25T16:50:51.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.500+0000 7fe53d8af640 1 -- start start 2024-04-25T16:50:51.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe53d8af640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe53815e990 0x7fe538165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe53d8af640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe538165b90 0x7fe538167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe53d8af640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe53806d610 con 0x7fe538070080 2024-04-25T16:50:51.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe53d8af640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe538158550 con 0x7fe538165b90 2024-04-25T16:50:51.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe53d8af640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe53806f7d0 con 0x7fe53815e990 2024-04-25T16:50:51.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe5367fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe53815e990 0x7fe538165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe536ffd640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe538070080 0x7fe53815e280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:57300/0 (socket says 172.21.15.94:57300) 2024-04-25T16:50:51.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe536ffd640 1 -- 172.21.15.94:0/3576056623 learned_addr learned my addr 172.21.15.94:0/3576056623 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:50:51.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe5377fe640 1 --2- 172.21.15.94:0/3576056623 >> v2:172.21.15.138:3300/0 conn(0x7fe538165b90 0x7fe538167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe535ffb640 1 -- 172.21.15.94:0/3576056623 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 471399971 0 0) 0x7fe52c001120 con 0x7fe538070080 2024-04-25T16:50:51.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe535ffb640 1 -- 172.21.15.94:0/3576056623 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe520003530 con 0x7fe538070080 2024-04-25T16:50:51.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe5367fc640 1 -- 172.21.15.94:0/3576056623 >> v2:172.21.15.138:3300/0 conn(0x7fe538165b90 msgr2=0x7fe538167fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:51.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe5367fc640 1 --2- 172.21.15.94:0/3576056623 >> v2:172.21.15.138:3300/0 conn(0x7fe538165b90 0x7fe538167fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe5367fc640 1 -- 172.21.15.94:0/3576056623 >> v1:172.21.15.138:6789/0 conn(0x7fe538070080 legacy=0x7fe53815e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.504+0000 7fe5367fc640 1 -- 172.21.15.94:0/3576056623 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53806fc60 con 0x7fe53815e990 2024-04-25T16:50:51.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.508+0000 7fe5367fc640 1 --2- 172.21.15.94:0/3576056623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe53815e990 0x7fe538165650 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe5381586d0 tx=0x7fe52801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ab6f74a3970b338 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:51.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.508+0000 7fe535ffb640 1 -- 172.21.15.94:0/3576056623 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280ab680 con 0x7fe53815e990 2024-04-25T16:50:51.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.508+0000 7fe535ffb640 1 -- 172.21.15.94:0/3576056623 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5280b0070 con 0x7fe53815e990 2024-04-25T16:50:51.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.508+0000 7fe535ffb640 1 -- 172.21.15.94:0/3576056623 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280aba20 con 0x7fe53815e990 2024-04-25T16:50:51.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.508+0000 7fe53d8af640 1 -- 172.21.15.94:0/3576056623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe53815e990 msgr2=0x7fe538165650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.508+0000 7fe53d8af640 1 --2- 172.21.15.94:0/3576056623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe53815e990 0x7fe538165650 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe5381586d0 tx=0x7fe52801c480 comp rx=0 tx=0).stop 2024-04-25T16:50:51.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.508+0000 7fe53d8af640 1 -- 172.21.15.94:0/3576056623 shutdown_connections 2024-04-25T16:50:51.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.508+0000 7fe53d8af640 1 --2- 172.21.15.94:0/3576056623 >> v2:172.21.15.138:3300/0 conn(0x7fe538165b90 0x7fe538167fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.508+0000 7fe53d8af640 1 --2- 172.21.15.94:0/3576056623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe53815e990 0x7fe538165650 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.508+0000 7fe53d8af640 1 -- 172.21.15.94:0/3576056623 >> 172.21.15.94:0/3576056623 conn(0x7fe5380b47f0 msgr2=0x7fe5380b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:51.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.508+0000 7fe53d8af640 1 -- 172.21.15.94:0/3576056623 shutdown_connections 2024-04-25T16:50:51.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.508+0000 7fe53d8af640 1 -- 172.21.15.94:0/3576056623 wait complete. 2024-04-25T16:50:51.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe53d8af640 1 Processor -- start 2024-04-25T16:50:51.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe53d8af640 1 -- start start 2024-04-25T16:50:51.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe53d8af640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe538070080 0x7fe53809aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe53d8af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe53815e990 0x7fe5380cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe53d8af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe538165b90 0x7fe53815f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe53d8af640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5380ae6c0 con 0x7fe538165b90 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe53d8af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe53806f460 con 0x7fe53815e990 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe53d8af640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe538070490 con 0x7fe538070080 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe5377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe538165b90 0x7fe53815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe5377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe538165b90 0x7fe53815f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43428/0 (socket says 172.21.15.94:43428) 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe5377fe640 1 -- 172.21.15.94:0/2788264482 learned_addr learned my addr 172.21.15.94:0/2788264482 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe5367fc640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe53815e990 0x7fe5380cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe536ffd640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe538070080 0x7fe53809aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe5377fe640 1 -- 172.21.15.94:0/2788264482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe538070080 msgr2=0x7fe53809aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe5377fe640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe538070080 0x7fe53809aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe5377fe640 1 -- 172.21.15.94:0/2788264482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe53815e990 msgr2=0x7fe5380cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe5377fe640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe53815e990 0x7fe5380cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe5377fe640 1 -- 172.21.15.94:0/2788264482 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5380d0fe0 con 0x7fe538165b90 2024-04-25T16:50:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe5367fc640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe53815e990 0x7fe5380cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:50:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe5377fe640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe538165b90 0x7fe53815f780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe530009cb0 tx=0x7fe53000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe51ffff640 1 -- 172.21.15.94:0/2788264482 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530018020 con 0x7fe538165b90 2024-04-25T16:50:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe51ffff640 1 -- 172.21.15.94:0/2788264482 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe530014480 con 0x7fe538165b90 2024-04-25T16:50:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe53d8af640 1 -- 172.21.15.94:0/2788264482 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe538151cc0 con 0x7fe538165b90 2024-04-25T16:50:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe51ffff640 1 -- 172.21.15.94:0/2788264482 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530014870 con 0x7fe538165b90 2024-04-25T16:50:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe53d8af640 1 -- 172.21.15.94:0/2788264482 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5380d0d00 con 0x7fe538165b90 2024-04-25T16:50:51.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.512+0000 7fe53d8af640 1 -- 172.21.15.94:0/2788264482 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe538076830 con 0x7fe538165b90 2024-04-25T16:50:51.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.516+0000 7fe51ffff640 1 -- 172.21.15.94:0/2788264482 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe53001b020 con 0x7fe538165b90 2024-04-25T16:50:51.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.516+0000 7fe51ffff640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe52406f400 0x7fe524071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.516+0000 7fe51ffff640 1 -- 172.21.15.94:0/2788264482 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe53008fcc0 con 0x7fe538165b90 2024-04-25T16:50:51.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.516+0000 7fe536ffd640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe52406f400 0x7fe524071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.516+0000 7fe536ffd640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe52406f400 0x7fe524071910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0012c0 tx=0x7fe52c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:51.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.520+0000 7fe51ffff640 1 -- 172.21.15.94:0/2788264482 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe530022030 con 0x7fe538165b90 2024-04-25T16:50:51.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.668+0000 7fe53d8af640 1 -- 172.21.15.94:0/2788264482 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe538074320 con 0x7fe538165b90 2024-04-25T16:50:51.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.668+0000 7fe51ffff640 1 -- 172.21.15.94:0/2788264482 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fe530010040 con 0x7fe538165b90 2024-04-25T16:50:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:50:51.668 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-04-25T16:50:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:50:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-04-25T16:50:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-04-25T16:50:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:50:51.668 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-04-25T16:50:51.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.672+0000 7fe53d8af640 1 -- 172.21.15.94:0/2788264482 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe52406f400 msgr2=0x7fe524071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.672+0000 7fe53d8af640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe52406f400 0x7fe524071910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0012c0 tx=0x7fe52c009040 comp rx=0 tx=0).stop 2024-04-25T16:50:51.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.672+0000 7fe53d8af640 1 -- 172.21.15.94:0/2788264482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe538165b90 msgr2=0x7fe53815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.672+0000 7fe53d8af640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe538165b90 0x7fe53815f780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe530009cb0 tx=0x7fe53000c490 comp rx=0 tx=0).stop 2024-04-25T16:50:51.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.672+0000 7fe53d8af640 1 -- 172.21.15.94:0/2788264482 shutdown_connections 2024-04-25T16:50:51.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.672+0000 7fe53d8af640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe538165b90 0x7fe53815f780 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.672+0000 7fe53d8af640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe53815e990 0x7fe5380cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.672+0000 7fe53d8af640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe52406f400 0x7fe524071910 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.672+0000 7fe53d8af640 1 --2- 172.21.15.94:0/2788264482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe538070080 0x7fe53809aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.672+0000 7fe53d8af640 1 -- 172.21.15.94:0/2788264482 >> 172.21.15.94:0/2788264482 conn(0x7fe5380b47f0 msgr2=0x7fe538160960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:51.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.672+0000 7fe53d8af640 1 -- 172.21.15.94:0/2788264482 shutdown_connections 2024-04-25T16:50:51.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.672+0000 7fe53d8af640 1 -- 172.21.15.94:0/2788264482 wait complete. 2024-04-25T16:50:51.687 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T16:50:51.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T16:50:51.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.844+0000 7fc38aae0640 1 Processor -- start 2024-04-25T16:50:51.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.844+0000 7fc38aae0640 1 -- start start 2024-04-25T16:50:51.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.848+0000 7fc38aae0640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3840d2760 0x7fc3840b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.848+0000 7fc38aae0640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc3840702f0 0x7fc384070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.848+0000 7fc38aae0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3840b6510 con 0x7fc3840cf700 2024-04-25T16:50:51.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.848+0000 7fc38aae0640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fc3840b62c0 con 0x7fc3840702f0 2024-04-25T16:50:51.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.848+0000 7fc38aae0640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc3840b67f0 con 0x7fc3840d2760 2024-04-25T16:50:51.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.848+0000 7fc38907d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc3840702f0 0x7fc384070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.848+0000 7fc38907d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc3840702f0 0x7fc384070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43436/0 (socket says 172.21.15.94:43436) 2024-04-25T16:50:51.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.848+0000 7fc38907d640 1 -- 172.21.15.94:0/1870826095 learned_addr learned my addr 172.21.15.94:0/1870826095 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:51.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.848+0000 7fc383fff640 1 --2- 172.21.15.94:0/1870826095 >> v2:172.21.15.138:3300/0 conn(0x7fc3840d2760 0x7fc3840b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.848+0000 7fc38907d640 1 -- 172.21.15.94:0/1870826095 >> v2:172.21.15.138:3300/0 conn(0x7fc3840d2760 msgr2=0x7fc3840b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:51.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.848+0000 7fc38907d640 1 --2- 172.21.15.94:0/1870826095 >> v2:172.21.15.138:3300/0 conn(0x7fc3840d2760 0x7fc3840b72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.848+0000 7fc38907d640 1 -- 172.21.15.94:0/1870826095 >> v1:172.21.15.138:6789/0 conn(0x7fc3840cf700 legacy=0x7fc3840d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:51.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.848+0000 7fc38907d640 1 -- 172.21.15.94:0/1870826095 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3840b6ad0 con 0x7fc3840702f0 2024-04-25T16:50:51.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc38907d640 1 --2- 172.21.15.94:0/1870826095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3840702f0 0x7fc384070700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc384161550 tx=0x7fc37c01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fbf60fbdc26c9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:51.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc3837fe640 1 -- 172.21.15.94:0/1870826095 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0a5650 con 0x7fc3840702f0 2024-04-25T16:50:51.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc3837fe640 1 -- 172.21.15.94:0/1870826095 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37c0a57f0 con 0x7fc3840702f0 2024-04-25T16:50:51.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc3837fe640 1 -- 172.21.15.94:0/1870826095 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0a5bc0 con 0x7fc3840702f0 2024-04-25T16:50:51.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc38aae0640 1 -- 172.21.15.94:0/1870826095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3840702f0 msgr2=0x7fc384070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc38aae0640 1 --2- 172.21.15.94:0/1870826095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3840702f0 0x7fc384070700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc384161550 tx=0x7fc37c01c990 comp rx=0 tx=0).stop 2024-04-25T16:50:51.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc38aae0640 1 -- 172.21.15.94:0/1870826095 shutdown_connections 2024-04-25T16:50:51.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc38aae0640 1 --2- 172.21.15.94:0/1870826095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3840702f0 0x7fc384070700 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc38aae0640 1 --2- 172.21.15.94:0/1870826095 >> v2:172.21.15.138:3300/0 conn(0x7fc3840d2760 0x7fc3840b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc38aae0640 1 -- 172.21.15.94:0/1870826095 >> 172.21.15.94:0/1870826095 conn(0x7fc3840cfef0 msgr2=0x7fc3840d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:51.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc38aae0640 1 -- 172.21.15.94:0/1870826095 shutdown_connections 2024-04-25T16:50:51.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc38aae0640 1 -- 172.21.15.94:0/1870826095 wait complete. 2024-04-25T16:50:51.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc38aae0640 1 Processor -- start 2024-04-25T16:50:51.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.852+0000 7fc38aae0640 1 -- start start 2024-04-25T16:50:51.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38aae0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3840702f0 0x7fc38414d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38aae0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3840cf700 0x7fc38414b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38aae0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc384071fd0 0x7fc3840723e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38aae0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc38415deb0 con 0x7fc3840702f0 2024-04-25T16:50:51.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38aae0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc38415dd30 con 0x7fc3840cf700 2024-04-25T16:50:51.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38aae0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc38416ae70 con 0x7fc384071fd0 2024-04-25T16:50:51.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38887c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3840702f0 0x7fc38414d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38887c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3840702f0 0x7fc38414d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43448/0 (socket says 172.21.15.94:43448) 2024-04-25T16:50:51.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38887c640 1 -- 172.21.15.94:0/1658158452 learned_addr learned my addr 172.21.15.94:0/1658158452 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:51.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc383fff640 1 --2- 172.21.15.94:0/1658158452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3840cf700 0x7fc38414b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38887c640 1 -- 172.21.15.94:0/1658158452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc384071fd0 msgr2=0x7fc3840723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38887c640 1 --2- 172.21.15.94:0/1658158452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc384071fd0 0x7fc3840723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38887c640 1 -- 172.21.15.94:0/1658158452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3840cf700 msgr2=0x7fc38414b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:51.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38887c640 1 --2- 172.21.15.94:0/1658158452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3840cf700 0x7fc38414b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:51.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38887c640 1 -- 172.21.15.94:0/1658158452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc384072920 con 0x7fc3840702f0 2024-04-25T16:50:51.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38887c640 1 --2- 172.21.15.94:0/1658158452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3840702f0 0x7fc38414d410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc37800b410 tx=0x7fc37800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:51.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc3817fa640 1 -- 172.21.15.94:0/1658158452 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378015030 con 0x7fc3840702f0 2024-04-25T16:50:51.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc3817fa640 1 -- 172.21.15.94:0/1658158452 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc378004020 con 0x7fc3840702f0 2024-04-25T16:50:51.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38aae0640 1 -- 172.21.15.94:0/1658158452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38414ddf0 con 0x7fc3840702f0 2024-04-25T16:50:51.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc3817fa640 1 -- 172.21.15.94:0/1658158452 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378004410 con 0x7fc3840702f0 2024-04-25T16:50:51.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38aae0640 1 -- 172.21.15.94:0/1658158452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc38414e3b0 con 0x7fc3840702f0 2024-04-25T16:50:51.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc38aae0640 1 -- 172.21.15.94:0/1658158452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc38414b7c0 con 0x7fc3840702f0 2024-04-25T16:50:51.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc3817fa640 1 -- 172.21.15.94:0/1658158452 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc378018070 con 0x7fc3840702f0 2024-04-25T16:50:51.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc3817fa640 1 --2- 172.21.15.94:0/1658158452 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc36806f430 0x7fc368071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:51.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc3817fa640 1 -- 172.21.15.94:0/1658158452 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fc37808e9d0 con 0x7fc3840702f0 2024-04-25T16:50:51.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.856+0000 7fc383fff640 1 --2- 172.21.15.94:0/1658158452 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc36806f430 0x7fc368071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:51.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.860+0000 7fc383fff640 1 --2- 172.21.15.94:0/1658158452 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc36806f430 0x7fc368071940 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc384168360 tx=0x7fc370008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:51.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:51.864+0000 7fc3817fa640 1 -- 172.21.15.94:0/1658158452 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc384072920 con 0x7fc3840702f0 2024-04-25T16:50:52.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:52.012+0000 7fc38aae0640 1 -- 172.21.15.94:0/1658158452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc3840708a0 con 0x7fc3840702f0 2024-04-25T16:50:52.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.000+0000 7fc3817fa640 1 -- 172.21.15.94:0/1658158452 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fc37805bca0 con 0x7fc3840702f0 2024-04-25T16:50:52.999 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-04-25T16:50:53.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.004+0000 7fc38aae0640 1 -- 172.21.15.94:0/1658158452 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc36806f430 msgr2=0x7fc368071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:53.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.004+0000 7fc38aae0640 1 --2- 172.21.15.94:0/1658158452 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc36806f430 0x7fc368071940 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc384168360 tx=0x7fc370008040 comp rx=0 tx=0).stop 2024-04-25T16:50:53.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.004+0000 7fc38aae0640 1 -- 172.21.15.94:0/1658158452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3840702f0 msgr2=0x7fc38414d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:53.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.004+0000 7fc38aae0640 1 --2- 172.21.15.94:0/1658158452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3840702f0 0x7fc38414d410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc37800b410 tx=0x7fc37800b8d0 comp rx=0 tx=0).stop 2024-04-25T16:50:53.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.004+0000 7fc38aae0640 1 -- 172.21.15.94:0/1658158452 shutdown_connections 2024-04-25T16:50:53.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.004+0000 7fc38aae0640 1 --2- 172.21.15.94:0/1658158452 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc36806f430 0x7fc368071940 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:53.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.004+0000 7fc38aae0640 1 --2- 172.21.15.94:0/1658158452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc384071fd0 0x7fc3840723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:53.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.004+0000 7fc38aae0640 1 --2- 172.21.15.94:0/1658158452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3840cf700 0x7fc38414b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:53.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.004+0000 7fc38aae0640 1 --2- 172.21.15.94:0/1658158452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3840702f0 0x7fc38414d410 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:53.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.004+0000 7fc38aae0640 1 -- 172.21.15.94:0/1658158452 >> 172.21.15.94:0/1658158452 conn(0x7fc3840cfef0 msgr2=0x7fc3840d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:53.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.004+0000 7fc38aae0640 1 -- 172.21.15.94:0/1658158452 shutdown_connections 2024-04-25T16:50:53.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.004+0000 7fc38aae0640 1 -- 172.21.15.94:0/1658158452 wait complete. 2024-04-25T16:50:53.029 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T16:50:53.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.188+0000 7f1805da3640 1 Processor -- start 2024-04-25T16:50:53.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.188+0000 7f1805da3640 1 -- start start 2024-04-25T16:50:53.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f1805da3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1800072b60 0x7f180015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:53.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f1805da3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18000afa50 0x7f18001601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:53.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f1805da3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f180006f360 con 0x7f180015cd60 2024-04-25T16:50:53.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f1805da3640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f18000703e0 con 0x7f1800072b60 2024-04-25T16:50:53.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f1805da3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f180006f620 con 0x7f18000afa50 2024-04-25T16:50:53.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f17ff7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1800072b60 0x7f180015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:53.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f17ff7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1800072b60 0x7f180015c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43452/0 (socket says 172.21.15.94:43452) 2024-04-25T16:50:53.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f17ff7fe640 1 -- 172.21.15.94:0/685292823 learned_addr learned my addr 172.21.15.94:0/685292823 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f17fffff640 1 --2- 172.21.15.94:0/685292823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18000afa50 0x7f18001601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f17ff7fe640 1 -- 172.21.15.94:0/685292823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18000afa50 msgr2=0x7f18001601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f17ff7fe640 1 --2- 172.21.15.94:0/685292823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18000afa50 0x7f18001601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f17ff7fe640 1 -- 172.21.15.94:0/685292823 >> v1:172.21.15.94:6789/0 conn(0x7f180015cd60 legacy=0x7f180015fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f17ff7fe640 1 -- 172.21.15.94:0/685292823 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180006f900 con 0x7f1800072b60 2024-04-25T16:50:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f17fffff640 1 --2- 172.21.15.94:0/685292823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18000afa50 0x7f18001601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:53.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.192+0000 7f17ff7fe640 1 --2- 172.21.15.94:0/685292823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1800072b60 0x7f180015c820 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17f000aa10 tx=0x7f17f001d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba044e71f97d66a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:53.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.196+0000 7f17fe7fc640 1 -- 172.21.15.94:0/685292823 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f00b3070 con 0x7f1800072b60 2024-04-25T16:50:53.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.196+0000 7f17fe7fc640 1 -- 172.21.15.94:0/685292823 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f00a68a0 con 0x7f1800072b60 2024-04-25T16:50:53.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.196+0000 7f17fe7fc640 1 -- 172.21.15.94:0/685292823 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f000b040 con 0x7f1800072b60 2024-04-25T16:50:53.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.196+0000 7f1805da3640 1 -- 172.21.15.94:0/685292823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1800072b60 msgr2=0x7f180015c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:53.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.196+0000 7f1805da3640 1 --2- 172.21.15.94:0/685292823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1800072b60 0x7f180015c820 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17f000aa10 tx=0x7f17f001d870 comp rx=0 tx=0).stop 2024-04-25T16:50:53.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.196+0000 7f1805da3640 1 -- 172.21.15.94:0/685292823 shutdown_connections 2024-04-25T16:50:53.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.196+0000 7f1805da3640 1 --2- 172.21.15.94:0/685292823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18000afa50 0x7f18001601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:53.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.196+0000 7f1805da3640 1 --2- 172.21.15.94:0/685292823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1800072b60 0x7f180015c820 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:53.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.196+0000 7f1805da3640 1 -- 172.21.15.94:0/685292823 >> 172.21.15.94:0/685292823 conn(0x7f18000b47f0 msgr2=0x7f18000b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:53.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.196+0000 7f1805da3640 1 -- 172.21.15.94:0/685292823 shutdown_connections 2024-04-25T16:50:53.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.196+0000 7f1805da3640 1 -- 172.21.15.94:0/685292823 wait complete. 2024-04-25T16:50:53.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.196+0000 7f1805da3640 1 Processor -- start 2024-04-25T16:50:53.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.196+0000 7f1805da3640 1 -- start start 2024-04-25T16:50:53.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f1805da3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1800072b60 0x7f18001663f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:53.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f1805da3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f180015cd60 0x7f18000b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:53.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f1805da3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18001453b0 0x7f18000b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:53.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f1805da3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1800159e40 con 0x7f180015cd60 2024-04-25T16:50:53.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f1805da3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1800159cc0 con 0x7f18001453b0 2024-04-25T16:50:53.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f1805da3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1800160730 con 0x7f1800072b60 2024-04-25T16:50:53.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f180015cd60 0x7f18000b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:53.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f180015cd60 0x7f18000b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43462/0 (socket says 172.21.15.94:43462) 2024-04-25T16:50:53.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17feffd640 1 -- 172.21.15.94:0/1437429645 learned_addr learned my addr 172.21.15.94:0/1437429645 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:53.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17fffff640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18001453b0 0x7f18000b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:53.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17ff7fe640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1800072b60 0x7f18001663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:53.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17feffd640 1 -- 172.21.15.94:0/1437429645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1800072b60 msgr2=0x7f18001663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:53.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17feffd640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1800072b60 0x7f18001663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:53.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17feffd640 1 -- 172.21.15.94:0/1437429645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18001453b0 msgr2=0x7f18000b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:53.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17feffd640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18001453b0 0x7f18000b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:53.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17feffd640 1 -- 172.21.15.94:0/1437429645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1800166930 con 0x7f180015cd60 2024-04-25T16:50:53.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17fffff640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18001453b0 0x7f18000b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:53.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17ff7fe640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1800072b60 0x7f18001663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:53.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17feffd640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f180015cd60 0x7f18000b2070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f17f400c370 tx=0x7f17f400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17e7fff640 1 -- 172.21.15.94:0/1437429645 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f40052d0 con 0x7f180015cd60 2024-04-25T16:50:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17e7fff640 1 -- 172.21.15.94:0/1437429645 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f4005470 con 0x7f180015cd60 2024-04-25T16:50:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f17e7fff640 1 -- 172.21.15.94:0/1437429645 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4007260 con 0x7f180015cd60 2024-04-25T16:50:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f1805da3640 1 -- 172.21.15.94:0/1437429645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1800150eb0 con 0x7f180015cd60 2024-04-25T16:50:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f1805da3640 1 -- 172.21.15.94:0/1437429645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1800151190 con 0x7f180015cd60 2024-04-25T16:50:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.200+0000 7f1805da3640 1 -- 172.21.15.94:0/1437429645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1800151390 con 0x7f180015cd60 2024-04-25T16:50:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.204+0000 7f17e7fff640 1 -- 172.21.15.94:0/1437429645 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f17f4007400 con 0x7f180015cd60 2024-04-25T16:50:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.204+0000 7f17e7fff640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f17ec06f400 0x7f17ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.204+0000 7f17e7fff640 1 -- 172.21.15.94:0/1437429645 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f17f408eca0 con 0x7f180015cd60 2024-04-25T16:50:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.204+0000 7f17ff7fe640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f17ec06f400 0x7f17ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.204+0000 7f17ff7fe640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f17ec06f400 0x7f17ec071910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f17f00b0000 tx=0x7f17f00a8070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:53.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.208+0000 7f17e7fff640 1 -- 172.21.15.94:0/1437429645 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1800151390 con 0x7f180015cd60 2024-04-25T16:50:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:53.356+0000 7f1805da3640 1 -- 172.21.15.94:0/1437429645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f1800076830 con 0x7f180015cd60 2024-04-25T16:50:54.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.004+0000 7f17e7fff640 1 -- 172.21.15.94:0/1437429645 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f17f405bee0 con 0x7f180015cd60 2024-04-25T16:50:54.003 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-04-25T16:50:54.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.008+0000 7f1805da3640 1 -- 172.21.15.94:0/1437429645 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f17ec06f400 msgr2=0x7f17ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:54.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.008+0000 7f1805da3640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f17ec06f400 0x7f17ec071910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f17f00b0000 tx=0x7f17f00a8070 comp rx=0 tx=0).stop 2024-04-25T16:50:54.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.008+0000 7f1805da3640 1 -- 172.21.15.94:0/1437429645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f180015cd60 msgr2=0x7f18000b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:54.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.008+0000 7f1805da3640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f180015cd60 0x7f18000b2070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f17f400c370 tx=0x7f17f400c830 comp rx=0 tx=0).stop 2024-04-25T16:50:54.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.008+0000 7f1805da3640 1 -- 172.21.15.94:0/1437429645 shutdown_connections 2024-04-25T16:50:54.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.008+0000 7f1805da3640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18001453b0 0x7f18000b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:54.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.008+0000 7f1805da3640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f180015cd60 0x7f18000b2070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:54.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.008+0000 7f1805da3640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f17ec06f400 0x7f17ec071910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:54.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.008+0000 7f1805da3640 1 --2- 172.21.15.94:0/1437429645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1800072b60 0x7f18001663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:54.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.008+0000 7f1805da3640 1 -- 172.21.15.94:0/1437429645 >> 172.21.15.94:0/1437429645 conn(0x7f18000b47f0 msgr2=0x7f18000af210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:54.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.008+0000 7f1805da3640 1 -- 172.21.15.94:0/1437429645 shutdown_connections 2024-04-25T16:50:54.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.008+0000 7f1805da3640 1 -- 172.21.15.94:0/1437429645 wait complete. 2024-04-25T16:50:54.048 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T16:50:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec21eda640 1 Processor -- start 2024-04-25T16:50:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec21eda640 1 -- start start 2024-04-25T16:50:54.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec21eda640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fec1c0b0420 0x7fec1c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec21eda640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec1c06f2a0 0x7fec1c06c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec21eda640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fec1c06cb40 0x7fec1c06cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec21eda640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fec1c06d490 con 0x7fec1c0b0420 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec21eda640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fec1c0aeaa0 con 0x7fec1c06cb40 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec21eda640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fec1c0703e0 con 0x7fec1c06f2a0 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec1b7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fec1c0b0420 0x7fec1c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec1b7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fec1c0b0420 0x7fec1c06ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43464/0 (socket says 172.21.15.94:43464) 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec1b7fe640 1 -- 172.21.15.94:0/4141858836 learned_addr learned my addr 172.21.15.94:0/4141858836 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec1bfff640 1 --2- 172.21.15.94:0/4141858836 >> v2:172.21.15.138:3300/0 conn(0x7fec1c06cb40 0x7fec1c06cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec1affd640 1 --2- 172.21.15.94:0/4141858836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec1c06f2a0 0x7fec1c06c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec1b7fe640 1 -- 172.21.15.94:0/4141858836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec1c06f2a0 msgr2=0x7fec1c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec1b7fe640 1 --2- 172.21.15.94:0/4141858836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec1c06f2a0 0x7fec1c06c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec1b7fe640 1 -- 172.21.15.94:0/4141858836 >> v2:172.21.15.138:3300/0 conn(0x7fec1c06cb40 msgr2=0x7fec1c06cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec1b7fe640 1 --2- 172.21.15.94:0/4141858836 >> v2:172.21.15.138:3300/0 conn(0x7fec1c06cb40 0x7fec1c06cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:54.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec1b7fe640 1 -- 172.21.15.94:0/4141858836 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c070630 con 0x7fec1c0b0420 2024-04-25T16:50:54.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.188+0000 7fec1bfff640 1 --2- 172.21.15.94:0/4141858836 >> v2:172.21.15.138:3300/0 conn(0x7fec1c06cb40 0x7fec1c06cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:54.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.192+0000 7fec1b7fe640 1 --2- 172.21.15.94:0/4141858836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1c0b0420 0x7fec1c06ed60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fec1c071c10 tx=0x7fec0c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aad6dbb154b256 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:54.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.192+0000 7fec1a7fc640 1 -- 172.21.15.94:0/4141858836 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c0b1070 con 0x7fec1c0b0420 2024-04-25T16:50:54.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.192+0000 7fec1a7fc640 1 -- 172.21.15.94:0/4141858836 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec0c0ac630 con 0x7fec1c0b0420 2024-04-25T16:50:54.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.192+0000 7fec1a7fc640 1 -- 172.21.15.94:0/4141858836 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c0a8050 con 0x7fec1c0b0420 2024-04-25T16:50:54.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.192+0000 7fec21eda640 1 -- 172.21.15.94:0/4141858836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1c0b0420 msgr2=0x7fec1c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:54.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.192+0000 7fec21eda640 1 --2- 172.21.15.94:0/4141858836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1c0b0420 0x7fec1c06ed60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fec1c071c10 tx=0x7fec0c01c520 comp rx=0 tx=0).stop 2024-04-25T16:50:54.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.192+0000 7fec21eda640 1 -- 172.21.15.94:0/4141858836 shutdown_connections 2024-04-25T16:50:54.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.192+0000 7fec21eda640 1 --2- 172.21.15.94:0/4141858836 >> v2:172.21.15.138:3300/0 conn(0x7fec1c06cb40 0x7fec1c06cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:54.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.192+0000 7fec21eda640 1 --2- 172.21.15.94:0/4141858836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec1c06f2a0 0x7fec1c06c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:54.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.192+0000 7fec21eda640 1 --2- 172.21.15.94:0/4141858836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1c0b0420 0x7fec1c06ed60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:54.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.192+0000 7fec21eda640 1 -- 172.21.15.94:0/4141858836 >> 172.21.15.94:0/4141858836 conn(0x7fec1c0b47f0 msgr2=0x7fec1c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:54.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.192+0000 7fec21eda640 1 -- 172.21.15.94:0/4141858836 shutdown_connections 2024-04-25T16:50:54.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.192+0000 7fec21eda640 1 -- 172.21.15.94:0/4141858836 wait complete. 2024-04-25T16:50:54.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec21eda640 1 Processor -- start 2024-04-25T16:50:54.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec21eda640 1 -- start start 2024-04-25T16:50:54.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec21eda640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c06cb40 0x7fec1c14b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:54.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec21eda640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1c06f2a0 0x7fec1c159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:54.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec21eda640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec1c0b0420 0x7fec1c15a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:54.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec21eda640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c0b0830 con 0x7fec1c06f2a0 2024-04-25T16:50:54.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec21eda640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c0aef30 con 0x7fec1c06cb40 2024-04-25T16:50:54.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec21eda640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fec1c0b09b0 con 0x7fec1c0b0420 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec1affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1c06f2a0 0x7fec1c159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec1affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1c06f2a0 0x7fec1c159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43472/0 (socket says 172.21.15.94:43472) 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec1affd640 1 -- 172.21.15.94:0/3760060578 learned_addr learned my addr 172.21.15.94:0/3760060578 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec1bfff640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec1c0b0420 0x7fec1c15a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec1b7fe640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c06cb40 0x7fec1c14b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec1affd640 1 -- 172.21.15.94:0/3760060578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec1c0b0420 msgr2=0x7fec1c15a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec1affd640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec1c0b0420 0x7fec1c15a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec1affd640 1 -- 172.21.15.94:0/3760060578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c06cb40 msgr2=0x7fec1c14b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec1affd640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c06cb40 0x7fec1c14b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec1affd640 1 -- 172.21.15.94:0/3760060578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c0740b0 con 0x7fec1c06f2a0 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec1b7fe640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c06cb40 0x7fec1c14b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec1affd640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1c06f2a0 0x7fec1c159ad0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fec1000c7f0 tx=0x7fec1000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec03fff640 1 -- 172.21.15.94:0/3760060578 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10015070 con 0x7fec1c06f2a0 2024-04-25T16:50:54.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec03fff640 1 -- 172.21.15.94:0/3760060578 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec10007820 con 0x7fec1c06f2a0 2024-04-25T16:50:54.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec21eda640 1 -- 172.21.15.94:0/3760060578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c073530 con 0x7fec1c06f2a0 2024-04-25T16:50:54.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec03fff640 1 -- 172.21.15.94:0/3760060578 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10007c10 con 0x7fec1c06f2a0 2024-04-25T16:50:54.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.196+0000 7fec21eda640 1 -- 172.21.15.94:0/3760060578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c073af0 con 0x7fec1c06f2a0 2024-04-25T16:50:54.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.200+0000 7fec21eda640 1 -- 172.21.15.94:0/3760060578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1c0732a0 con 0x7fec1c06f2a0 2024-04-25T16:50:54.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.200+0000 7fec03fff640 1 -- 172.21.15.94:0/3760060578 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fec10018070 con 0x7fec1c06f2a0 2024-04-25T16:50:54.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.200+0000 7fec03fff640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fec0806f400 0x7fec08071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:54.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.200+0000 7fec03fff640 1 -- 172.21.15.94:0/3760060578 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fec1008f660 con 0x7fec1c06f2a0 2024-04-25T16:50:54.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.200+0000 7fec1b7fe640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fec0806f400 0x7fec08071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:54.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.200+0000 7fec1b7fe640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fec0806f400 0x7fec08071910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0ae000 tx=0x7fec0c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:54.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.204+0000 7fec03fff640 1 -- 172.21.15.94:0/3760060578 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fec1005c720 con 0x7fec1c06f2a0 2024-04-25T16:50:54.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:54.400+0000 7fec21eda640 1 -- 172.21.15.94:0/3760060578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fec1c076830 con 0x7fec1c06f2a0 2024-04-25T16:50:55.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.072+0000 7fec03fff640 1 -- 172.21.15.94:0/3760060578 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fec1005c020 con 0x7fec1c06f2a0 2024-04-25T16:50:55.070 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T16:50:55.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.076+0000 7fec21eda640 1 -- 172.21.15.94:0/3760060578 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fec0806f400 msgr2=0x7fec08071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:55.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.076+0000 7fec21eda640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fec0806f400 0x7fec08071910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0ae000 tx=0x7fec0c0a7000 comp rx=0 tx=0).stop 2024-04-25T16:50:55.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.076+0000 7fec21eda640 1 -- 172.21.15.94:0/3760060578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1c06f2a0 msgr2=0x7fec1c159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:55.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.076+0000 7fec21eda640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1c06f2a0 0x7fec1c159ad0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fec1000c7f0 tx=0x7fec1000ccb0 comp rx=0 tx=0).stop 2024-04-25T16:50:55.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.076+0000 7fec21eda640 1 -- 172.21.15.94:0/3760060578 shutdown_connections 2024-04-25T16:50:55.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.076+0000 7fec21eda640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fec0806f400 0x7fec08071910 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:55.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.076+0000 7fec21eda640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec1c0b0420 0x7fec1c15a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:55.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.076+0000 7fec21eda640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec1c06f2a0 0x7fec1c159ad0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:55.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.076+0000 7fec21eda640 1 --2- 172.21.15.94:0/3760060578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c06cb40 0x7fec1c14b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:55.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.076+0000 7fec21eda640 1 -- 172.21.15.94:0/3760060578 >> 172.21.15.94:0/3760060578 conn(0x7fec1c0b47f0 msgr2=0x7fec1c154ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:55.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.076+0000 7fec21eda640 1 -- 172.21.15.94:0/3760060578 shutdown_connections 2024-04-25T16:50:55.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.076+0000 7fec21eda640 1 -- 172.21.15.94:0/3760060578 wait complete. 2024-04-25T16:50:55.108 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T16:50:55.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T16:50:55.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.260+0000 7f6e17b7a640 1 Processor -- start 2024-04-25T16:50:55.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.260+0000 7f6e17b7a640 1 -- start start 2024-04-25T16:50:55.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e17b7a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e100b09d0 0x7f6e100b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:55.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e17b7a640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e1015ff10 con 0x7f6e100b6450 2024-04-25T16:50:55.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e17b7a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e100b6a80 con 0x7f6e100d16e0 2024-04-25T16:50:55.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e17b7a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6e100b6860 con 0x7f6e100b09d0 2024-04-25T16:50:55.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e15115640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f6e100b6450 0x7f6e100b02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:36690/0 (socket says 172.21.15.94:36690) 2024-04-25T16:50:55.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e15115640 1 -- 172.21.15.94:0/3160390219 learned_addr learned my addr 172.21.15.94:0/3160390219 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:50:55.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e16117640 1 --2- 172.21.15.94:0/3160390219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e100b09d0 0x7f6e100b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:55.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3098191792 0 0) 0x7f6e1015ff10 con 0x7f6e100b6450 2024-04-25T16:50:55.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dfc003530 con 0x7f6e100b6450 2024-04-25T16:50:55.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1635229431 0 0) 0x7f6dfc003530 con 0x7f6e100b6450 2024-04-25T16:50:55.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e100b09d0 msgr2=0x7f6e100b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:55.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 --2- 172.21.15.94:0/3160390219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e100b09d0 0x7f6e100b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:55.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 >> v1:172.21.15.138:6789/0 conn(0x7f6e100d16e0 legacy=0x7f6e100b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:55.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e100b6d60 con 0x7f6e100b6450 2024-04-25T16:50:55.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f6e00001680 con 0x7f6e100b6450 2024-04-25T16:50:55.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 >> v1:172.21.15.94:6789/0 conn(0x7f6e100b6450 legacy=0x7f6e100b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 --2- 172.21.15.94:0/3160390219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dfc003a60 0x7f6dfc003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 --2- 172.21.15.94:0/3160390219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dfc004810 0x7f6dfc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 --2- 172.21.15.94:0/3160390219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e100b09d0 0x7f6dfc0052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100b6860 con 0x7f6dfc003a60 2024-04-25T16:50:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e15916640 1 --2- 172.21.15.94:0/3160390219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dfc003a60 0x7f6dfc003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc00bb60 con 0x7f6dfc004810 2024-04-25T16:50:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6dfc00bd60 con 0x7f6e100b09d0 2024-04-25T16:50:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e15115640 1 --2- 172.21.15.94:0/3160390219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dfc004810 0x7f6dfc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e16117640 1 --2- 172.21.15.94:0/3160390219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e100b09d0 0x7f6dfc0052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e15916640 1 -- 172.21.15.94:0/3160390219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e100b09d0 msgr2=0x7f6dfc0052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e15916640 1 --2- 172.21.15.94:0/3160390219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e100b09d0 0x7f6dfc0052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e15916640 1 -- 172.21.15.94:0/3160390219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dfc004810 msgr2=0x7f6dfc004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e15916640 1 --2- 172.21.15.94:0/3160390219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dfc004810 0x7f6dfc004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:55.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e15916640 1 -- 172.21.15.94:0/3160390219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc003530 con 0x7f6dfc003a60 2024-04-25T16:50:55.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.264+0000 7f6e16117640 1 --2- 172.21.15.94:0/3160390219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e100b09d0 0x7f6dfc0052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:55.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.268+0000 7f6e15916640 1 --2- 172.21.15.94:0/3160390219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dfc003a60 0x7f6dfc003ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c001e40 tx=0x7f6e0c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407deab31380b97e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:55.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.268+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c001a90 con 0x7f6dfc003a60 2024-04-25T16:50:55.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.268+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0c003b80 con 0x7f6dfc003a60 2024-04-25T16:50:55.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.268+0000 7f6e14914640 1 -- 172.21.15.94:0/3160390219 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e100b15f0 con 0x7f6dfc003a60 2024-04-25T16:50:55.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.268+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3160390219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dfc003a60 msgr2=0x7f6dfc003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:55.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.268+0000 7f6e17b7a640 1 --2- 172.21.15.94:0/3160390219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dfc003a60 0x7f6dfc003ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c001e40 tx=0x7f6e0c01c760 comp rx=0 tx=0).stop 2024-04-25T16:50:55.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.268+0000 7f6e16117640 1 -- 172.21.15.94:0/3160390219 reap_dead start 2024-04-25T16:50:55.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.268+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3160390219 shutdown_connections 2024-04-25T16:50:55.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.268+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3160390219 >> 172.21.15.94:0/3160390219 conn(0x7f6e10070f40 msgr2=0x7f6e10071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:55.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.268+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3160390219 shutdown_connections 2024-04-25T16:50:55.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.268+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3160390219 wait complete. 2024-04-25T16:50:55.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.268+0000 7f6e17b7a640 1 Processor -- start 2024-04-25T16:50:55.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.268+0000 7f6e17b7a640 1 -- start start 2024-04-25T16:50:55.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e17b7a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dfc003a60 0x7f6e100cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:55.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e17b7a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e101fc2e0 0x7f6e101f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:55.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e17b7a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e1015cab0 0x7f6e1015cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:55.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e17b7a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1016b320 con 0x7f6dfc003a60 2024-04-25T16:50:55.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e17b7a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1016b1a0 con 0x7f6e1015cab0 2024-04-25T16:50:55.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e17b7a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6e1016b4a0 con 0x7f6e101fc2e0 2024-04-25T16:50:55.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e15916640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dfc003a60 0x7f6e100cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:55.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e15916640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dfc003a60 0x7f6e100cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43482/0 (socket says 172.21.15.94:43482) 2024-04-25T16:50:55.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e15916640 1 -- 172.21.15.94:0/3055247880 learned_addr learned my addr 172.21.15.94:0/3055247880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:55.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e16117640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e1015cab0 0x7f6e1015cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:55.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e15115640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e101fc2e0 0x7f6e101f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e15916640 1 -- 172.21.15.94:0/3055247880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e101fc2e0 msgr2=0x7f6e101f9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e15916640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e101fc2e0 0x7f6e101f9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e15916640 1 -- 172.21.15.94:0/3055247880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e1015cab0 msgr2=0x7f6e1015cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e15916640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e1015cab0 0x7f6e1015cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e15916640 1 -- 172.21.15.94:0/3055247880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e10071f20 con 0x7f6dfc003a60 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e15115640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e101fc2e0 0x7f6e101f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e16117640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e1015cab0 0x7f6e1015cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e15916640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dfc003a60 0x7f6e100cd600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c01cc40 tx=0x7f6dfc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e067fc640 1 -- 172.21.15.94:0/3055247880 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c001a90 con 0x7f6dfc003a60 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e067fc640 1 -- 172.21.15.94:0/3055247880 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0c0bc070 con 0x7f6dfc003a60 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3055247880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1015d400 con 0x7f6dfc003a60 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e067fc640 1 -- 172.21.15.94:0/3055247880 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c005710 con 0x7f6dfc003a60 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3055247880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e100ce0a0 con 0x7f6dfc003a60 2024-04-25T16:50:55.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.272+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3055247880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e100715f0 con 0x7f6dfc003a60 2024-04-25T16:50:55.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.276+0000 7f6e067fc640 1 -- 172.21.15.94:0/3055247880 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6e0c0b1040 con 0x7f6dfc003a60 2024-04-25T16:50:55.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.276+0000 7f6e067fc640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6df406f430 0x7f6df4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:55.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.276+0000 7f6e067fc640 1 -- 172.21.15.94:0/3055247880 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f6e0c12a270 con 0x7f6dfc003a60 2024-04-25T16:50:55.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.276+0000 7f6e15115640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6df406f430 0x7f6df4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:55.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.276+0000 7f6e15115640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6df406f430 0x7f6df4071940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6e00001930 tx=0x7f6e000018a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:55.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.280+0000 7f6e067fc640 1 -- 172.21.15.94:0/3055247880 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6e1015d400 con 0x7f6dfc003a60 2024-04-25T16:50:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:55.448+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3055247880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f6e10168990 con 0x7f6dfc003a60 2024-04-25T16:50:56.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.032+0000 7f6e067fc640 1 -- 172.21.15.94:0/3055247880 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f6e0c0ad020 con 0x7f6dfc003a60 2024-04-25T16:50:56.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.036+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3055247880 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6df406f430 msgr2=0x7f6df4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.036+0000 7f6e17b7a640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6df406f430 0x7f6df4071940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6e00001930 tx=0x7f6e000018a0 comp rx=0 tx=0).stop 2024-04-25T16:50:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.036+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3055247880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dfc003a60 msgr2=0x7f6e100cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.036+0000 7f6e17b7a640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dfc003a60 0x7f6e100cd600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c01cc40 tx=0x7f6dfc005840 comp rx=0 tx=0).stop 2024-04-25T16:50:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.036+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3055247880 shutdown_connections 2024-04-25T16:50:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.036+0000 7f6e17b7a640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e1015cab0 0x7f6e1015cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.036+0000 7f6e17b7a640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6df406f430 0x7f6df4071940 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.036+0000 7f6e17b7a640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e101fc2e0 0x7f6e101f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.036+0000 7f6e17b7a640 1 --2- 172.21.15.94:0/3055247880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dfc003a60 0x7f6e100cd600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.036+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3055247880 >> 172.21.15.94:0/3055247880 conn(0x7f6e10070f40 msgr2=0x7f6e100700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.036+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3055247880 shutdown_connections 2024-04-25T16:50:56.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.036+0000 7f6e17b7a640 1 -- 172.21.15.94:0/3055247880 wait complete. 2024-04-25T16:50:56.056 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T16:50:56.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac2efb640 1 Processor -- start 2024-04-25T16:50:56.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac2efb640 1 -- start start 2024-04-25T16:50:56.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac2efb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06d710 0x7f7abc06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:56.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac2efb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7abc06e7a0 0x7f7abc06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:56.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac2efb640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7abc074380 con 0x7f7abc15c7e0 2024-04-25T16:50:56.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac2efb640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7abc06e060 con 0x7f7abc06e7a0 2024-04-25T16:50:56.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac2efb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7abc0727e0 con 0x7f7abc06d710 2024-04-25T16:50:56.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac1498640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7abc06e7a0 0x7f7abc06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:56.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac1498640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7abc06e7a0 0x7f7abc06ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43496/0 (socket says 172.21.15.94:43496) 2024-04-25T16:50:56.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac1498640 1 -- 172.21.15.94:0/3916610788 learned_addr learned my addr 172.21.15.94:0/3916610788 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:56.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7abbfff640 1 --2- 172.21.15.94:0/3916610788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06d710 0x7f7abc06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:56.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac1498640 1 -- 172.21.15.94:0/3916610788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06d710 msgr2=0x7f7abc06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:56.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac1498640 1 --2- 172.21.15.94:0/3916610788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06d710 0x7f7abc06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:56.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac1498640 1 -- 172.21.15.94:0/3916610788 >> v1:172.21.15.138:6789/0 conn(0x7f7abc15c7e0 legacy=0x7f7abc0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:56.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.224+0000 7f7ac1498640 1 -- 172.21.15.94:0/3916610788 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc072ac0 con 0x7f7abc06e7a0 2024-04-25T16:50:56.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.228+0000 7f7ac1498640 1 --2- 172.21.15.94:0/3916610788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc06e7a0 0x7f7abc06ebb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400d150 tx=0x7f7ab401eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6521e2219685f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:56.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.228+0000 7f7abb7fe640 1 -- 172.21.15.94:0/3916610788 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab400b2d0 con 0x7f7abc06e7a0 2024-04-25T16:50:56.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.228+0000 7f7abb7fe640 1 -- 172.21.15.94:0/3916610788 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab400b470 con 0x7f7abc06e7a0 2024-04-25T16:50:56.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.228+0000 7f7abb7fe640 1 -- 172.21.15.94:0/3916610788 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40a7b60 con 0x7f7abc06e7a0 2024-04-25T16:50:56.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.228+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3916610788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc06e7a0 msgr2=0x7f7abc06ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:56.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.228+0000 7f7ac2efb640 1 --2- 172.21.15.94:0/3916610788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc06e7a0 0x7f7abc06ebb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400d150 tx=0x7f7ab401eb70 comp rx=0 tx=0).stop 2024-04-25T16:50:56.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.228+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3916610788 shutdown_connections 2024-04-25T16:50:56.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.228+0000 7f7ac2efb640 1 --2- 172.21.15.94:0/3916610788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc06e7a0 0x7f7abc06ebb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:56.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.228+0000 7f7ac2efb640 1 --2- 172.21.15.94:0/3916610788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06d710 0x7f7abc06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:56.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.228+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3916610788 >> 172.21.15.94:0/3916610788 conn(0x7f7abc0b47f0 msgr2=0x7f7abc0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:56.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.228+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3916610788 shutdown_connections 2024-04-25T16:50:56.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.228+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3916610788 wait complete. 2024-04-25T16:50:56.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ac2efb640 1 Processor -- start 2024-04-25T16:50:56.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ac2efb640 1 -- start start 2024-04-25T16:50:56.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ac2efb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc06d710 0x7f7abc14f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:56.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ac2efb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc06e7a0 0x7f7abc149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:56.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ac2efb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc15c7e0 0x7f7abc09a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:56.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ac2efb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc15cbf0 con 0x7f7abc06e7a0 2024-04-25T16:50:56.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ac2efb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc073ca0 con 0x7f7abc06d710 2024-04-25T16:50:56.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ac2efb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7abc0b0500 con 0x7f7abc15c7e0 2024-04-25T16:50:56.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc06e7a0 0x7f7abc149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc06e7a0 0x7f7abc149a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43508/0 (socket says 172.21.15.94:43508) 2024-04-25T16:50:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7abbfff640 1 -- 172.21.15.94:0/3963505452 learned_addr learned my addr 172.21.15.94:0/3963505452 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ac1498640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc15c7e0 0x7f7abc09a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ac0c97640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc06d710 0x7f7abc14f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7abbfff640 1 -- 172.21.15.94:0/3963505452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc15c7e0 msgr2=0x7f7abc09a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7abbfff640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc15c7e0 0x7f7abc09a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7abbfff640 1 -- 172.21.15.94:0/3963505452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc06d710 msgr2=0x7f7abc14f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7abbfff640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc06d710 0x7f7abc14f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7abbfff640 1 -- 172.21.15.94:0/3963505452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc156b40 con 0x7f7abc06e7a0 2024-04-25T16:50:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7abbfff640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc06e7a0 0x7f7abc149a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400d800 tx=0x7f7aa400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ab97fa640 1 -- 172.21.15.94:0/3963505452 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa400b6c0 con 0x7f7abc06e7a0 2024-04-25T16:50:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ab97fa640 1 -- 172.21.15.94:0/3963505452 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa4019070 con 0x7f7abc06e7a0 2024-04-25T16:50:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3963505452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7abc156e20 con 0x7f7abc06e7a0 2024-04-25T16:50:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ab97fa640 1 -- 172.21.15.94:0/3963505452 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa400ba80 con 0x7f7abc06e7a0 2024-04-25T16:50:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.232+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3963505452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7abc1573e0 con 0x7f7abc06e7a0 2024-04-25T16:50:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.236+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3963505452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7abc076830 con 0x7f7abc06e7a0 2024-04-25T16:50:56.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.236+0000 7f7ab97fa640 1 -- 172.21.15.94:0/3963505452 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7aa400bc20 con 0x7f7abc06e7a0 2024-04-25T16:50:56.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.236+0000 7f7ab97fa640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7a9c06f430 0x7f7a9c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:56.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.236+0000 7f7ac0c97640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7a9c06f430 0x7f7a9c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:56.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.236+0000 7f7ab97fa640 1 -- 172.21.15.94:0/3963505452 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f7aa408f800 con 0x7f7abc06e7a0 2024-04-25T16:50:56.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.236+0000 7f7ac0c97640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7a9c06f430 0x7f7a9c071940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0015b0 tx=0x7f7aac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:56.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.240+0000 7f7ab97fa640 1 -- 172.21.15.94:0/3963505452 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7aa405c5c0 con 0x7f7abc06e7a0 2024-04-25T16:50:56.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:56.400+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3963505452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7abc14fa30 con 0x7f7abc06e7a0 2024-04-25T16:50:58.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.100+0000 7f7ab97fa640 1 -- 172.21.15.94:0/3963505452 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f7aa405bec0 con 0x7f7abc06e7a0 2024-04-25T16:50:58.097 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data_ec' created 2024-04-25T16:50:58.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.104+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3963505452 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7a9c06f430 msgr2=0x7f7a9c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:58.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.104+0000 7f7ac2efb640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7a9c06f430 0x7f7a9c071940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0015b0 tx=0x7f7aac009040 comp rx=0 tx=0).stop 2024-04-25T16:50:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.104+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3963505452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc06e7a0 msgr2=0x7f7abc149a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.104+0000 7f7ac2efb640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc06e7a0 0x7f7abc149a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400d800 tx=0x7f7aa400dcc0 comp rx=0 tx=0).stop 2024-04-25T16:50:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.104+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3963505452 shutdown_connections 2024-04-25T16:50:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.104+0000 7f7ac2efb640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7a9c06f430 0x7f7a9c071940 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.104+0000 7f7ac2efb640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc15c7e0 0x7f7abc09a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.104+0000 7f7ac2efb640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc06e7a0 0x7f7abc149a20 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400d800 tx=0x7f7aa400dcc0 comp rx=0 tx=0).stop 2024-04-25T16:50:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.104+0000 7f7ac2efb640 1 --2- 172.21.15.94:0/3963505452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc06d710 0x7f7abc14f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.104+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3963505452 >> 172.21.15.94:0/3963505452 conn(0x7f7abc0b47f0 msgr2=0x7f7abc06f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.104+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3963505452 shutdown_connections 2024-04-25T16:50:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.104+0000 7f7ac2efb640 1 -- 172.21.15.94:0/3963505452 wait complete. 2024-04-25T16:50:58.136 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T16:50:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.292+0000 7f7e60c92640 1 Processor -- start 2024-04-25T16:50:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.292+0000 7f7e60c92640 1 -- start start 2024-04-25T16:50:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e60c92640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7e5c0d1a90 0x7f7e5c163ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e60c92640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7e5c164420 0x7f7e5c0d5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e60c92640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e5c0d1ea0 con 0x7f7e5c0b7150 2024-04-25T16:50:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e60c92640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7e5c0b7fd0 con 0x7f7e5c164420 2024-04-25T16:50:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e60c92640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7e5c164830 con 0x7f7e5c0d1a90 2024-04-25T16:50:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e5ad9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7e5c164420 0x7f7e5c0d5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e5ad9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7e5c164420 0x7f7e5c0d5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48886/0 (socket says 172.21.15.94:48886) 2024-04-25T16:50:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e5ad9d640 1 -- 172.21.15.94:0/3536402260 learned_addr learned my addr 172.21.15.94:0/3536402260 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e59d9b640 1 --2- 172.21.15.94:0/3536402260 >> v2:172.21.15.138:3300/0 conn(0x7f7e5c0d1a90 0x7f7e5c163ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e5ad9d640 1 -- 172.21.15.94:0/3536402260 >> v2:172.21.15.138:3300/0 conn(0x7f7e5c0d1a90 msgr2=0x7f7e5c163ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e5ad9d640 1 --2- 172.21.15.94:0/3536402260 >> v2:172.21.15.138:3300/0 conn(0x7f7e5c0d1a90 0x7f7e5c163ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e5ad9d640 1 -- 172.21.15.94:0/3536402260 >> v1:172.21.15.94:6789/0 conn(0x7f7e5c0b7150 legacy=0x7f7e5c0d44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e5ad9d640 1 -- 172.21.15.94:0/3536402260 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c0b7730 con 0x7f7e5c164420 2024-04-25T16:50:58.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e5ad9d640 1 --2- 172.21.15.94:0/3536402260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5c164420 0x7f7e5c0d5640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c08d710 tx=0x7f7e5401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8878c6d3c0aa0b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:58.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.296+0000 7f7e5959a640 1 -- 172.21.15.94:0/3536402260 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e540a5620 con 0x7f7e5c164420 2024-04-25T16:50:58.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.300+0000 7f7e5959a640 1 -- 172.21.15.94:0/3536402260 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e540a57c0 con 0x7f7e5c164420 2024-04-25T16:50:58.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.300+0000 7f7e5959a640 1 -- 172.21.15.94:0/3536402260 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e540a5b90 con 0x7f7e5c164420 2024-04-25T16:50:58.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.300+0000 7f7e60c92640 1 -- 172.21.15.94:0/3536402260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5c164420 msgr2=0x7f7e5c0d5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:58.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.300+0000 7f7e60c92640 1 --2- 172.21.15.94:0/3536402260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5c164420 0x7f7e5c0d5640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c08d710 tx=0x7f7e5401c960 comp rx=0 tx=0).stop 2024-04-25T16:50:58.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.300+0000 7f7e60c92640 1 -- 172.21.15.94:0/3536402260 shutdown_connections 2024-04-25T16:50:58.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.300+0000 7f7e60c92640 1 --2- 172.21.15.94:0/3536402260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5c164420 0x7f7e5c0d5640 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:58.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.300+0000 7f7e60c92640 1 --2- 172.21.15.94:0/3536402260 >> v2:172.21.15.138:3300/0 conn(0x7f7e5c0d1a90 0x7f7e5c163ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:58.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.300+0000 7f7e60c92640 1 -- 172.21.15.94:0/3536402260 >> 172.21.15.94:0/3536402260 conn(0x7f7e5c09e940 msgr2=0x7f7e5c08d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:58.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.300+0000 7f7e60c92640 1 -- 172.21.15.94:0/3536402260 shutdown_connections 2024-04-25T16:50:58.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.300+0000 7f7e60c92640 1 -- 172.21.15.94:0/3536402260 wait complete. 2024-04-25T16:50:58.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.300+0000 7f7e60c92640 1 Processor -- start 2024-04-25T16:50:58.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.300+0000 7f7e60c92640 1 -- start start 2024-04-25T16:50:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e60c92640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5c0b7150 0x7f7e5c0b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e60c92640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e5c0d1a90 0x7f7e5c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e60c92640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e5c164420 0x7f7e5c156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e60c92640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c0d4bd0 con 0x7f7e5c0b7150 2024-04-25T16:50:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e60c92640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c15fb30 con 0x7f7e5c0d1a90 2024-04-25T16:50:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e60c92640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5c163100 con 0x7f7e5c164420 2024-04-25T16:50:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e5a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5c0b7150 0x7f7e5c0b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e5a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5c0b7150 0x7f7e5c0b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48890/0 (socket says 172.21.15.94:48890) 2024-04-25T16:50:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e5a59c640 1 -- 172.21.15.94:0/1695022618 learned_addr learned my addr 172.21.15.94:0/1695022618 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e5ad9d640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e5c164420 0x7f7e5c156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e59d9b640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e5c0d1a90 0x7f7e5c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e5a59c640 1 -- 172.21.15.94:0/1695022618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e5c164420 msgr2=0x7f7e5c156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e5a59c640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e5c164420 0x7f7e5c156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e5a59c640 1 -- 172.21.15.94:0/1695022618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e5c0d1a90 msgr2=0x7f7e5c156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e5a59c640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e5c0d1a90 0x7f7e5c156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e5a59c640 1 -- 172.21.15.94:0/1695022618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c0b4050 con 0x7f7e5c0b7150 2024-04-25T16:50:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e5a59c640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5c0b7150 0x7f7e5c0b3b10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00c340 tx=0x7f7e4c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e4affd640 1 -- 172.21.15.94:0/1695022618 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0052d0 con 0x7f7e5c0b7150 2024-04-25T16:50:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e4affd640 1 -- 172.21.15.94:0/1695022618 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4c005470 con 0x7f7e5c0b7150 2024-04-25T16:50:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e60c92640 1 -- 172.21.15.94:0/1695022618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5c0b6750 con 0x7f7e5c0b7150 2024-04-25T16:50:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e4affd640 1 -- 172.21.15.94:0/1695022618 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c007260 con 0x7f7e5c0b7150 2024-04-25T16:50:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e60c92640 1 -- 172.21.15.94:0/1695022618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5c0b6d10 con 0x7f7e5c0b7150 2024-04-25T16:50:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e60c92640 1 -- 172.21.15.94:0/1695022618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5c06ff60 con 0x7f7e5c0b7150 2024-04-25T16:50:58.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e4affd640 1 -- 172.21.15.94:0/1695022618 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7e4c007400 con 0x7f7e5c0b7150 2024-04-25T16:50:58.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e4affd640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7e3c06f400 0x7f7e3c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:58.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e4affd640 1 -- 172.21.15.94:0/1695022618 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f7e4c08f200 con 0x7f7e5c0b7150 2024-04-25T16:50:58.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.304+0000 7f7e59d9b640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7e3c06f400 0x7f7e3c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:58.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.308+0000 7f7e59d9b640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7e3c06f400 0x7f7e3c071910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0b5150 tx=0x7f7e50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:58.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.308+0000 7f7e4affd640 1 -- 172.21.15.94:0/1695022618 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7e5c0d1ea0 con 0x7f7e5c0b7150 2024-04-25T16:50:58.434 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-04-25T16:50:58.438+0000 7f7b7a727640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T16:50:58.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:58.476+0000 7f7e60c92640 1 -- 172.21.15.94:0/1695022618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f7e5c0d24d0 con 0x7f7e5c0b7150 2024-04-25T16:50:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.100+0000 7f7e4affd640 1 -- 172.21.15.94:0/1695022618 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f7e4c05beb0 con 0x7f7e5c0b7150 2024-04-25T16:50:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T16:50:59.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.104+0000 7f7e60c92640 1 -- 172.21.15.94:0/1695022618 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7e3c06f400 msgr2=0x7f7e3c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:59.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.104+0000 7f7e60c92640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7e3c06f400 0x7f7e3c071910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0b5150 tx=0x7f7e50008040 comp rx=0 tx=0).stop 2024-04-25T16:50:59.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.104+0000 7f7e60c92640 1 -- 172.21.15.94:0/1695022618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5c0b7150 msgr2=0x7f7e5c0b3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:59.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.104+0000 7f7e60c92640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5c0b7150 0x7f7e5c0b3b10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00c340 tx=0x7f7e4c00c800 comp rx=0 tx=0).stop 2024-04-25T16:50:59.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.104+0000 7f7e60c92640 1 -- 172.21.15.94:0/1695022618 shutdown_connections 2024-04-25T16:50:59.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.104+0000 7f7e60c92640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7e3c06f400 0x7f7e3c071910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:59.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.104+0000 7f7e60c92640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e5c164420 0x7f7e5c156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:59.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.104+0000 7f7e60c92640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e5c0d1a90 0x7f7e5c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:59.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.104+0000 7f7e60c92640 1 --2- 172.21.15.94:0/1695022618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5c0b7150 0x7f7e5c0b3b10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:59.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.104+0000 7f7e60c92640 1 -- 172.21.15.94:0/1695022618 >> 172.21.15.94:0/1695022618 conn(0x7f7e5c09e940 msgr2=0x7f7e5c158ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:59.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.108+0000 7f7e60c92640 1 -- 172.21.15.94:0/1695022618 shutdown_connections 2024-04-25T16:50:59.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.108+0000 7f7e60c92640 1 -- 172.21.15.94:0/1695022618 wait complete. 2024-04-25T16:50:59.148 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T16:50:59.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.308+0000 7fb4cc6cc640 1 Processor -- start 2024-04-25T16:50:59.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.308+0000 7fb4cc6cc640 1 -- start start 2024-04-25T16:50:59.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4cc6cc640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb4c4072b60 0x7fb4c415c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:59.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4cc6cc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb4c40afa50 0x7fb4c41601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:59.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4cc6cc640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4c406f360 con 0x7fb4c415cd60 2024-04-25T16:50:59.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4cc6cc640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fb4c40703e0 con 0x7fb4c40afa50 2024-04-25T16:50:59.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4cc6cc640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb4c406f620 con 0x7fb4c4072b60 2024-04-25T16:50:59.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4cac69640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb4c40afa50 0x7fb4c41601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4cac69640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb4c40afa50 0x7fb4c41601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48892/0 (socket says 172.21.15.94:48892) 2024-04-25T16:50:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4cac69640 1 -- 172.21.15.94:0/2055954183 learned_addr learned my addr 172.21.15.94:0/2055954183 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4ca468640 1 --2- 172.21.15.94:0/2055954183 >> v2:172.21.15.138:3300/0 conn(0x7fb4c4072b60 0x7fb4c415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4cac69640 1 -- 172.21.15.94:0/2055954183 >> v2:172.21.15.138:3300/0 conn(0x7fb4c4072b60 msgr2=0x7fb4c415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4cac69640 1 --2- 172.21.15.94:0/2055954183 >> v2:172.21.15.138:3300/0 conn(0x7fb4c4072b60 0x7fb4c415c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4cac69640 1 -- 172.21.15.94:0/2055954183 >> v1:172.21.15.138:6789/0 conn(0x7fb4c415cd60 legacy=0x7fb4c415fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4cac69640 1 -- 172.21.15.94:0/2055954183 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c406f900 con 0x7fb4c40afa50 2024-04-25T16:50:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.312+0000 7fb4ca468640 1 --2- 172.21.15.94:0/2055954183 >> v2:172.21.15.138:3300/0 conn(0x7fb4c4072b60 0x7fb4c415c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4cac69640 1 --2- 172.21.15.94:0/2055954183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c40afa50 0x7fb4c41601f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb4c406ee70 tx=0x7fb4bc01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a40a6370bbf83a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4c9466640 1 -- 172.21.15.94:0/2055954183 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0a5730 con 0x7fb4c40afa50 2024-04-25T16:50:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4c9466640 1 -- 172.21.15.94:0/2055954183 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4bc0a58d0 con 0x7fb4c40afa50 2024-04-25T16:50:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4c9466640 1 -- 172.21.15.94:0/2055954183 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0a5ca0 con 0x7fb4c40afa50 2024-04-25T16:50:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/2055954183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c40afa50 msgr2=0x7fb4c41601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4cc6cc640 1 --2- 172.21.15.94:0/2055954183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c40afa50 0x7fb4c41601f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb4c406ee70 tx=0x7fb4bc01c8f0 comp rx=0 tx=0).stop 2024-04-25T16:50:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/2055954183 shutdown_connections 2024-04-25T16:50:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4cc6cc640 1 --2- 172.21.15.94:0/2055954183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c40afa50 0x7fb4c41601f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4cc6cc640 1 --2- 172.21.15.94:0/2055954183 >> v2:172.21.15.138:3300/0 conn(0x7fb4c4072b60 0x7fb4c415c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/2055954183 >> 172.21.15.94:0/2055954183 conn(0x7fb4c40b47f0 msgr2=0x7fb4c40b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/2055954183 shutdown_connections 2024-04-25T16:50:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/2055954183 wait complete. 2024-04-25T16:50:59.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4cc6cc640 1 Processor -- start 2024-04-25T16:50:59.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.316+0000 7fb4cc6cc640 1 -- start start 2024-04-25T16:50:59.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cc6cc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4c4072b60 0x7fb4c414c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:59.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cc6cc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c415cd60 0x7fb4c41f9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:59.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cc6cc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c414f3e0 0x7fb4c41fa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:59.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cc6cc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c4159e40 con 0x7fb4c414f3e0 2024-04-25T16:50:59.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cc6cc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c4159cc0 con 0x7fb4c415cd60 2024-04-25T16:50:59.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cc6cc640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4c4160730 con 0x7fb4c4072b60 2024-04-25T16:50:59.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cac69640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c414f3e0 0x7fb4c41fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cac69640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c414f3e0 0x7fb4c41fa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48898/0 (socket says 172.21.15.94:48898) 2024-04-25T16:50:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cac69640 1 -- 172.21.15.94:0/709333984 learned_addr learned my addr 172.21.15.94:0/709333984 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:50:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4ca468640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4c4072b60 0x7fb4c414c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4c9c67640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c415cd60 0x7fb4c41f9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cac69640 1 -- 172.21.15.94:0/709333984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4c4072b60 msgr2=0x7fb4c414c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cac69640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4c4072b60 0x7fb4c414c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cac69640 1 -- 172.21.15.94:0/709333984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c415cd60 msgr2=0x7fb4c41f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cac69640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c415cd60 0x7fb4c41f9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cac69640 1 -- 172.21.15.94:0/709333984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c414fbc0 con 0x7fb4c414f3e0 2024-04-25T16:50:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4ca468640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4c4072b60 0x7fb4c414c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4c9c67640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c415cd60 0x7fb4c41f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:50:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cac69640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c414f3e0 0x7fb4c41fa010 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0af040 tx=0x7fb4bc019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4baffd640 1 -- 172.21.15.94:0/709333984 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0bc020 con 0x7fb4c414f3e0 2024-04-25T16:50:59.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4baffd640 1 -- 172.21.15.94:0/709333984 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4bc0b8670 con 0x7fb4c414f3e0 2024-04-25T16:50:59.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/709333984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c414fea0 con 0x7fb4c414f3e0 2024-04-25T16:50:59.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4baffd640 1 -- 172.21.15.94:0/709333984 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0b8a20 con 0x7fb4c414f3e0 2024-04-25T16:50:59.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/709333984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c4157d50 con 0x7fb4c414f3e0 2024-04-25T16:50:59.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.320+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/709333984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c4157b40 con 0x7fb4c414f3e0 2024-04-25T16:50:59.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.324+0000 7fb4baffd640 1 -- 172.21.15.94:0/709333984 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb4bc0bf020 con 0x7fb4c414f3e0 2024-04-25T16:50:59.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.324+0000 7fb4baffd640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb4a806f400 0x7fb4a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:59.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.324+0000 7fb4baffd640 1 -- 172.21.15.94:0/709333984 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fb4bc12ad40 con 0x7fb4c414f3e0 2024-04-25T16:50:59.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.324+0000 7fb4ca468640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb4a806f400 0x7fb4a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:59.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.324+0000 7fb4ca468640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb4a806f400 0x7fb4a8071910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4070c50 tx=0x7fb4c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:59.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.328+0000 7fb4baffd640 1 -- 172.21.15.94:0/709333984 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb4bc0c6030 con 0x7fb4c414f3e0 2024-04-25T16:50:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:50:59.524+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/709333984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fb4c414f950 con 0x7fb4c414f3e0 2024-04-25T16:51:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.152+0000 7fb4baffd640 1 -- 172.21.15.94:0/709333984 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7fb4bc0165a0 con 0x7fb4c414f3e0 2024-04-25T16:51:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:added data pool 4 to fsmap 2024-04-25T16:51:00.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.156+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/709333984 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb4a806f400 msgr2=0x7fb4a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.156+0000 7fb4cc6cc640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb4a806f400 0x7fb4a8071910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4070c50 tx=0x7fb4c0008040 comp rx=0 tx=0).stop 2024-04-25T16:51:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.156+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/709333984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c414f3e0 msgr2=0x7fb4c41fa010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.156+0000 7fb4cc6cc640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c414f3e0 0x7fb4c41fa010 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0af040 tx=0x7fb4bc019690 comp rx=0 tx=0).stop 2024-04-25T16:51:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.156+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/709333984 shutdown_connections 2024-04-25T16:51:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.156+0000 7fb4cc6cc640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c414f3e0 0x7fb4c41fa010 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.156+0000 7fb4cc6cc640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c415cd60 0x7fb4c41f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.156+0000 7fb4cc6cc640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb4a806f400 0x7fb4a8071910 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.156+0000 7fb4cc6cc640 1 --2- 172.21.15.94:0/709333984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4c4072b60 0x7fb4c414c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.156+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/709333984 >> 172.21.15.94:0/709333984 conn(0x7fb4c40b47f0 msgr2=0x7fb4c415ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.156+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/709333984 shutdown_connections 2024-04-25T16:51:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.156+0000 7fb4cc6cc640 1 -- 172.21.15.94:0/709333984 wait complete. 2024-04-25T16:51:00.174 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:51:00.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f4217260640 1 Processor -- start 2024-04-25T16:51:00.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f4217260640 1 -- start start 2024-04-25T16:51:00.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f4217260640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4210163da0 0x7f42101641b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:00.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f4217260640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f421006d130 0x7f421006d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:00.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f4217260640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42100d0a30 con 0x7f42100d02e0 2024-04-25T16:51:00.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f4217260640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f42100d1640 con 0x7f421006d130 2024-04-25T16:51:00.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f4217260640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f42100707b0 con 0x7f4210163da0 2024-04-25T16:51:00.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f42157fd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f421006d130 0x7f421006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:00.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f42157fd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f421006d130 0x7f421006d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48910/0 (socket says 172.21.15.94:48910) 2024-04-25T16:51:00.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f42157fd640 1 -- 172.21.15.94:0/2859101765 learned_addr learned my addr 172.21.15.94:0/2859101765 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:00.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f420ffff640 1 --2- 172.21.15.94:0/2859101765 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4210163da0 0x7f42101641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:00.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f42157fd640 1 -- 172.21.15.94:0/2859101765 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4210163da0 msgr2=0x7f42101641b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:00.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f42157fd640 1 --2- 172.21.15.94:0/2859101765 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4210163da0 0x7f42101641b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f42157fd640 1 -- 172.21.15.94:0/2859101765 >> v1:172.21.15.138:6789/0 conn(0x7f42100d02e0 legacy=0x7f4210163690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.320+0000 7f42157fd640 1 -- 172.21.15.94:0/2859101765 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42100d0d10 con 0x7f421006d130 2024-04-25T16:51:00.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.324+0000 7f42157fd640 1 --2- 172.21.15.94:0/2859101765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f421006d130 0x7f421006d540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4208009e80 tx=0x7f420801f200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517b75953adcd7e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:00.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.324+0000 7f420f7fe640 1 -- 172.21.15.94:0/2859101765 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42080b4070 con 0x7f421006d130 2024-04-25T16:51:00.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.324+0000 7f420f7fe640 1 -- 172.21.15.94:0/2859101765 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42080af400 con 0x7f421006d130 2024-04-25T16:51:00.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.324+0000 7f420f7fe640 1 -- 172.21.15.94:0/2859101765 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42080af7d0 con 0x7f421006d130 2024-04-25T16:51:00.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.324+0000 7f4217260640 1 -- 172.21.15.94:0/2859101765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f421006d130 msgr2=0x7f421006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.324+0000 7f4217260640 1 --2- 172.21.15.94:0/2859101765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f421006d130 0x7f421006d540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4208009e80 tx=0x7f420801f200 comp rx=0 tx=0).stop 2024-04-25T16:51:00.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.324+0000 7f4217260640 1 -- 172.21.15.94:0/2859101765 shutdown_connections 2024-04-25T16:51:00.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.324+0000 7f4217260640 1 --2- 172.21.15.94:0/2859101765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f421006d130 0x7f421006d540 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.324+0000 7f4217260640 1 --2- 172.21.15.94:0/2859101765 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4210163da0 0x7f42101641b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.324+0000 7f4217260640 1 -- 172.21.15.94:0/2859101765 >> 172.21.15.94:0/2859101765 conn(0x7f42100d2030 msgr2=0x7f42100d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:00.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.324+0000 7f4217260640 1 -- 172.21.15.94:0/2859101765 shutdown_connections 2024-04-25T16:51:00.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.324+0000 7f4217260640 1 -- 172.21.15.94:0/2859101765 wait complete. 2024-04-25T16:51:00.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f4217260640 1 Processor -- start 2024-04-25T16:51:00.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f4217260640 1 -- start start 2024-04-25T16:51:00.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f4217260640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f421006d130 0x7f42100b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:00.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f4217260640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42100d02e0 0x7f421014f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:00.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f4217260640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4210163da0 0x7f42101f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:00.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f4217260640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42100d06f0 con 0x7f42100d02e0 2024-04-25T16:51:00.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f4217260640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42100d12d0 con 0x7f4210163da0 2024-04-25T16:51:00.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f4217260640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f421006da80 con 0x7f421006d130 2024-04-25T16:51:00.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f420ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42100d02e0 0x7f421014f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:00.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f420ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42100d02e0 0x7f421014f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48924/0 (socket says 172.21.15.94:48924) 2024-04-25T16:51:00.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f420ffff640 1 -- 172.21.15.94:0/2810609182 learned_addr learned my addr 172.21.15.94:0/2810609182 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:00.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f4214ffc640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f421006d130 0x7f42100b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:00.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f42157fd640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4210163da0 0x7f42101f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:00.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f420ffff640 1 -- 172.21.15.94:0/2810609182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f421006d130 msgr2=0x7f42100b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f420ffff640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f421006d130 0x7f42100b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f420ffff640 1 -- 172.21.15.94:0/2810609182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4210163da0 msgr2=0x7f42101f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f420ffff640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4210163da0 0x7f42101f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f420ffff640 1 -- 172.21.15.94:0/2810609182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42100b3540 con 0x7f42100d02e0 2024-04-25T16:51:00.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f420ffff640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42100d02e0 0x7f421014f970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f420000d5f0 tx=0x7f420000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:00.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f420d7fa640 1 -- 172.21.15.94:0/2810609182 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42000125f0 con 0x7f42100d02e0 2024-04-25T16:51:00.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f420d7fa640 1 -- 172.21.15.94:0/2810609182 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4200019070 con 0x7f42100d02e0 2024-04-25T16:51:00.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f4217260640 1 -- 172.21.15.94:0/2810609182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42101565b0 con 0x7f42100d02e0 2024-04-25T16:51:00.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f420d7fa640 1 -- 172.21.15.94:0/2810609182 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42000129b0 con 0x7f42100d02e0 2024-04-25T16:51:00.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f4217260640 1 -- 172.21.15.94:0/2810609182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42101f7770 con 0x7f42100d02e0 2024-04-25T16:51:00.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.328+0000 7f4217260640 1 -- 172.21.15.94:0/2810609182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4210076830 con 0x7f42100d02e0 2024-04-25T16:51:00.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.332+0000 7f420d7fa640 1 -- 172.21.15.94:0/2810609182 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4200012b50 con 0x7f42100d02e0 2024-04-25T16:51:00.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.332+0000 7f420d7fa640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f41f006f430 0x7f41f0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:00.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.332+0000 7f420d7fa640 1 -- 172.21.15.94:0/2810609182 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f420008fa20 con 0x7f42100d02e0 2024-04-25T16:51:00.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.332+0000 7f4214ffc640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f41f006f430 0x7f41f0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:00.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.332+0000 7f4214ffc640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f41f006f430 0x7f41f0071940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f41f80015b0 tx=0x7f41f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:00.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.336+0000 7f420d7fa640 1 -- 172.21.15.94:0/2810609182 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f420005c5e0 con 0x7f42100d02e0 2024-04-25T16:51:00.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f4217260640 1 -- 172.21.15.94:0/2810609182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4210156340 con 0x7f42100d02e0 2024-04-25T16:51:00.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f420d7fa640 1 -- 172.21.15.94:0/2810609182 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f420005bee0 con 0x7f42100d02e0 2024-04-25T16:51:00.521 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:00.521 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2352186041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2352186041},{"type":"v1","addr":"172.21.15.138:6833","nonce":2352186041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3210927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3210927873},{"type":"v1","addr":"172.21.15.94:6835","nonce":3210927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2237413225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2237413225},{"type":"v1","addr":"172.21.15.94:6837","nonce":2237413225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:50:54.403299+0000","modified":"2024-04-25T16:50:59.529114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/3652545132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3652545132},{"type":"v1","addr":"172.21.15.138:6835","nonce":3652545132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T16:51:00.522 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 7 2024-04-25T16:51:00.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f4217260640 1 -- 172.21.15.94:0/2810609182 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f41f006f430 msgr2=0x7f41f0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f4217260640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f41f006f430 0x7f41f0071940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f41f80015b0 tx=0x7f41f8009040 comp rx=0 tx=0).stop 2024-04-25T16:51:00.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f4217260640 1 -- 172.21.15.94:0/2810609182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42100d02e0 msgr2=0x7f421014f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f4217260640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42100d02e0 0x7f421014f970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f420000d5f0 tx=0x7f420000dab0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f4217260640 1 -- 172.21.15.94:0/2810609182 shutdown_connections 2024-04-25T16:51:00.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f4217260640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4210163da0 0x7f42101f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f4217260640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42100d02e0 0x7f421014f970 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f4217260640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f41f006f430 0x7f41f0071940 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f4217260640 1 --2- 172.21.15.94:0/2810609182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f421006d130 0x7f42100b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f4217260640 1 -- 172.21.15.94:0/2810609182 >> 172.21.15.94:0/2810609182 conn(0x7f42100d2030 msgr2=0x7f42101618b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:00.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f4217260640 1 -- 172.21.15.94:0/2810609182 shutdown_connections 2024-04-25T16:51:00.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.524+0000 7f4217260640 1 -- 172.21.15.94:0/2810609182 wait complete. 2024-04-25T16:51:00.541 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.676+0000 7f0e64f0a640 1 Processor -- start 2024-04-25T16:51:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.676+0000 7f0e64f0a640 1 -- start start 2024-04-25T16:51:00.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.676+0000 7f0e64f0a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0e600cf9d0 0x7f0e60160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:00.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.676+0000 7f0e64f0a640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e600d0ef0 con 0x7f0e6015d640 2024-04-25T16:51:00.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.676+0000 7f0e64f0a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e600d11d0 con 0x7f0e600b57b0 2024-04-25T16:51:00.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.676+0000 7f0e64f0a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f0e600d14b0 con 0x7f0e600cf9d0 2024-04-25T16:51:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.676+0000 7f0e5dd9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0e600cf9d0 0x7f0e60160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.676+0000 7f0e5dd9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0e600cf9d0 0x7f0e60160bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48934/0 (socket says 172.21.15.94:48934) 2024-04-25T16:51:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.676+0000 7f0e5dd9b640 1 -- 172.21.15.94:0/2448783652 learned_addr learned my addr 172.21.15.94:0/2448783652 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.676+0000 7f0e5dd9b640 1 -- 172.21.15.94:0/2448783652 >> v1:172.21.15.138:6789/0 conn(0x7f0e600b57b0 legacy=0x7f0e600b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.676+0000 7f0e5dd9b640 1 -- 172.21.15.94:0/2448783652 >> v1:172.21.15.94:6789/0 conn(0x7f0e6015d640 legacy=0x7f0e600cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.676+0000 7f0e5dd9b640 1 -- 172.21.15.94:0/2448783652 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e600d1790 con 0x7f0e600cf9d0 2024-04-25T16:51:00.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.680+0000 7f0e5dd9b640 1 --2- 172.21.15.94:0/2448783652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e600cf9d0 0x7f0e60160bc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009870 tx=0x7f0e4801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b6caa15c88a410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:00.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.680+0000 7f0e5d59a640 1 -- 172.21.15.94:0/2448783652 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e480b1070 con 0x7f0e600cf9d0 2024-04-25T16:51:00.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.680+0000 7f0e5d59a640 1 -- 172.21.15.94:0/2448783652 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e480ac640 con 0x7f0e600cf9d0 2024-04-25T16:51:00.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.680+0000 7f0e5d59a640 1 -- 172.21.15.94:0/2448783652 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e600b6ae0 con 0x7f0e600cf9d0 2024-04-25T16:51:00.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.680+0000 7f0e64f0a640 1 -- 172.21.15.94:0/2448783652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e600cf9d0 msgr2=0x7f0e60160bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.680+0000 7f0e64f0a640 1 --2- 172.21.15.94:0/2448783652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e600cf9d0 0x7f0e60160bc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009870 tx=0x7f0e4801c400 comp rx=0 tx=0).stop 2024-04-25T16:51:00.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.680+0000 7f0e64f0a640 1 -- 172.21.15.94:0/2448783652 shutdown_connections 2024-04-25T16:51:00.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.680+0000 7f0e64f0a640 1 --2- 172.21.15.94:0/2448783652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e600cf9d0 0x7f0e60160bc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.680+0000 7f0e64f0a640 1 -- 172.21.15.94:0/2448783652 >> 172.21.15.94:0/2448783652 conn(0x7f0e60071150 msgr2=0x7f0e60071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:00.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.680+0000 7f0e64f0a640 1 -- 172.21.15.94:0/2448783652 shutdown_connections 2024-04-25T16:51:00.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.680+0000 7f0e64f0a640 1 -- 172.21.15.94:0/2448783652 wait complete. 2024-04-25T16:51:00.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e64f0a640 1 Processor -- start 2024-04-25T16:51:00.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e64f0a640 1 -- start start 2024-04-25T16:51:00.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e64f0a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e600b57b0 0x7f0e601449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:00.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e64f0a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e600cf9d0 0x7f0e601590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:00.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e64f0a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e6015d640 0x7f0e601fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:00.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e64f0a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6016de70 con 0x7f0e600b57b0 2024-04-25T16:51:00.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e64f0a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6016dcf0 con 0x7f0e6015d640 2024-04-25T16:51:00.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e64f0a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0e6016dff0 con 0x7f0e600cf9d0 2024-04-25T16:51:00.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e5e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e600b57b0 0x7f0e601449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:00.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e5e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e600b57b0 0x7f0e601449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48948/0 (socket says 172.21.15.94:48948) 2024-04-25T16:51:00.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e5e59c640 1 -- 172.21.15.94:0/3912195931 learned_addr learned my addr 172.21.15.94:0/3912195931 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:00.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e5ed9d640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e6015d640 0x7f0e601fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:00.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e5dd9b640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e600cf9d0 0x7f0e601590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:00.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e5e59c640 1 -- 172.21.15.94:0/3912195931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e600cf9d0 msgr2=0x7f0e601590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e5e59c640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e600cf9d0 0x7f0e601590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e5e59c640 1 -- 172.21.15.94:0/3912195931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e6015d640 msgr2=0x7f0e601fcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e5e59c640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e6015d640 0x7f0e601fcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e5e59c640 1 -- 172.21.15.94:0/3912195931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e60144ef0 con 0x7f0e600b57b0 2024-04-25T16:51:00.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e5dd9b640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e600cf9d0 0x7f0e601590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:00.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e5ed9d640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e6015d640 0x7f0e601fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:00.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e5e59c640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e600b57b0 0x7f0e601449b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800c730 tx=0x7f0e5800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:00.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e56ffd640 1 -- 172.21.15.94:0/3912195931 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58016020 con 0x7f0e600b57b0 2024-04-25T16:51:00.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e56ffd640 1 -- 172.21.15.94:0/3912195931 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e58006d80 con 0x7f0e600b57b0 2024-04-25T16:51:00.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e64f0a640 1 -- 172.21.15.94:0/3912195931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e60157980 con 0x7f0e600b57b0 2024-04-25T16:51:00.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e56ffd640 1 -- 172.21.15.94:0/3912195931 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58003bc0 con 0x7f0e600b57b0 2024-04-25T16:51:00.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e64f0a640 1 -- 172.21.15.94:0/3912195931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e60157f40 con 0x7f0e600b57b0 2024-04-25T16:51:00.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.684+0000 7f0e64f0a640 1 -- 172.21.15.94:0/3912195931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e6006a730 con 0x7f0e600b57b0 2024-04-25T16:51:00.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.688+0000 7f0e56ffd640 1 -- 172.21.15.94:0/3912195931 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0e5800a7f0 con 0x7f0e600b57b0 2024-04-25T16:51:00.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.688+0000 7f0e56ffd640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0e4006f400 0x7f0e40071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:00.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.688+0000 7f0e5dd9b640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0e4006f400 0x7f0e40071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:00.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.688+0000 7f0e56ffd640 1 -- 172.21.15.94:0/3912195931 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0e5808f180 con 0x7f0e600b57b0 2024-04-25T16:51:00.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.688+0000 7f0e5dd9b640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0e4006f400 0x7f0e40071910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0e480ae000 tx=0x7f0e48008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:00.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.692+0000 7f0e56ffd640 1 -- 172.21.15.94:0/3912195931 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0e58020020 con 0x7f0e600b57b0 2024-04-25T16:51:00.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.852+0000 7f0e64f0a640 1 -- 172.21.15.94:0/3912195931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e6015b9a0 con 0x7f0e600b57b0 2024-04-25T16:51:00.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.852+0000 7f0e56ffd640 1 -- 172.21.15.94:0/3912195931 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f0e5805be50 con 0x7f0e600b57b0 2024-04-25T16:51:00.852 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:00.852 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:59.528743+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:00.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.856+0000 7f0e64f0a640 1 -- 172.21.15.94:0/3912195931 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0e4006f400 msgr2=0x7f0e40071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.856+0000 7f0e64f0a640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0e4006f400 0x7f0e40071910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0e480ae000 tx=0x7f0e48008040 comp rx=0 tx=0).stop 2024-04-25T16:51:00.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.856+0000 7f0e64f0a640 1 -- 172.21.15.94:0/3912195931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e600b57b0 msgr2=0x7f0e601449b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:00.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.856+0000 7f0e64f0a640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e600b57b0 0x7f0e601449b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800c730 tx=0x7f0e5800cbf0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.856+0000 7f0e64f0a640 1 -- 172.21.15.94:0/3912195931 shutdown_connections 2024-04-25T16:51:00.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.856+0000 7f0e64f0a640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e6015d640 0x7f0e601fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.856+0000 7f0e64f0a640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0e4006f400 0x7f0e40071910 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.856+0000 7f0e64f0a640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e600cf9d0 0x7f0e601590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.856+0000 7f0e64f0a640 1 --2- 172.21.15.94:0/3912195931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e600b57b0 0x7f0e601449b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:00.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.856+0000 7f0e64f0a640 1 -- 172.21.15.94:0/3912195931 >> 172.21.15.94:0/3912195931 conn(0x7f0e60071150 msgr2=0x7f0e600b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:00.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.856+0000 7f0e64f0a640 1 -- 172.21.15.94:0/3912195931 shutdown_connections 2024-04-25T16:51:00.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:00.856+0000 7f0e64f0a640 1 -- 172.21.15.94:0/3912195931 wait complete. 2024-04-25T16:51:00.873 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:01.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd493c88640 1 Processor -- start 2024-04-25T16:51:01.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd493c88640 1 -- start start 2024-04-25T16:51:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd493c88640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd48c0cdcb0 0x7fd48c072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd493c88640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c072c80 0x7fd48c161580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd493c88640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd48c073090 con 0x7fd48c161ac0 2024-04-25T16:51:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd493c88640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd48c0d2120 con 0x7fd48c0cdcb0 2024-04-25T16:51:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd493c88640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd48c0d2400 con 0x7fd48c072c80 2024-04-25T16:51:01.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd491a24640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd48c0cdcb0 0x7fd48c072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd491223640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c072c80 0x7fd48c161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd491223640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c072c80 0x7fd48c161580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:60692/0 (socket says 172.21.15.94:60692) 2024-04-25T16:51:01.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd491223640 1 -- 172.21.15.94:0/2550954234 learned_addr learned my addr 172.21.15.94:0/2550954234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:01.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd491223640 1 -- 172.21.15.94:0/2550954234 >> v2:172.21.15.138:3300/0 conn(0x7fd48c0cdcb0 msgr2=0x7fd48c072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:01.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd491223640 1 --2- 172.21.15.94:0/2550954234 >> v2:172.21.15.138:3300/0 conn(0x7fd48c0cdcb0 0x7fd48c072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd491223640 1 -- 172.21.15.94:0/2550954234 >> v1:172.21.15.138:6789/0 conn(0x7fd48c161ac0 legacy=0x7fd48c161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd491223640 1 -- 172.21.15.94:0/2550954234 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c0d26e0 con 0x7fd48c072c80 2024-04-25T16:51:01.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.024+0000 7fd491a24640 1 --2- 172.21.15.94:0/2550954234 >> v2:172.21.15.138:3300/0 conn(0x7fd48c0cdcb0 0x7fd48c072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.028+0000 7fd491223640 1 --2- 172.21.15.94:0/2550954234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c072c80 0x7fd48c161580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009870 tx=0x7fd47c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50f4baf9226cd09d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.028+0000 7fd490a22640 1 -- 172.21.15.94:0/2550954234 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c0ab680 con 0x7fd48c072c80 2024-04-25T16:51:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.028+0000 7fd490a22640 1 -- 172.21.15.94:0/2550954234 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47c0b0070 con 0x7fd48c072c80 2024-04-25T16:51:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.028+0000 7fd490a22640 1 -- 172.21.15.94:0/2550954234 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c0aba40 con 0x7fd48c072c80 2024-04-25T16:51:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.028+0000 7fd493c88640 1 -- 172.21.15.94:0/2550954234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c072c80 msgr2=0x7fd48c161580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.028+0000 7fd493c88640 1 --2- 172.21.15.94:0/2550954234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c072c80 0x7fd48c161580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009870 tx=0x7fd47c01c4a0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.028+0000 7fd493c88640 1 -- 172.21.15.94:0/2550954234 shutdown_connections 2024-04-25T16:51:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.028+0000 7fd493c88640 1 --2- 172.21.15.94:0/2550954234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c072c80 0x7fd48c161580 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.028+0000 7fd493c88640 1 --2- 172.21.15.94:0/2550954234 >> v2:172.21.15.138:3300/0 conn(0x7fd48c0cdcb0 0x7fd48c072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.028+0000 7fd493c88640 1 -- 172.21.15.94:0/2550954234 >> 172.21.15.94:0/2550954234 conn(0x7fd48c0d3850 msgr2=0x7fd48c0d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.028+0000 7fd493c88640 1 -- 172.21.15.94:0/2550954234 shutdown_connections 2024-04-25T16:51:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.028+0000 7fd493c88640 1 -- 172.21.15.94:0/2550954234 wait complete. 2024-04-25T16:51:01.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd493c88640 1 Processor -- start 2024-04-25T16:51:01.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd493c88640 1 -- start start 2024-04-25T16:51:01.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd493c88640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c072c80 0x7fd48c142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd493c88640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c0cdcb0 0x7fd48c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd493c88640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c161ac0 0x7fd48c141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd493c88640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c0b6680 con 0x7fd48c072c80 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd493c88640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c0b6500 con 0x7fd48c161ac0 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd493c88640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd48c0ce0c0 con 0x7fd48c0cdcb0 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd491a24640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c072c80 0x7fd48c142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd491a24640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c072c80 0x7fd48c142b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48962/0 (socket says 172.21.15.94:48962) 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd491a24640 1 -- 172.21.15.94:0/2808650218 learned_addr learned my addr 172.21.15.94:0/2808650218 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd491223640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c0cdcb0 0x7fd48c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd492225640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c161ac0 0x7fd48c141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd491a24640 1 -- 172.21.15.94:0/2808650218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c0cdcb0 msgr2=0x7fd48c0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd491a24640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c0cdcb0 0x7fd48c0cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd491a24640 1 -- 172.21.15.94:0/2808650218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c161ac0 msgr2=0x7fd48c141f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd491a24640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c161ac0 0x7fd48c141f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd491a24640 1 -- 172.21.15.94:0/2808650218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c15fd00 con 0x7fd48c072c80 2024-04-25T16:51:01.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd492225640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c161ac0 0x7fd48c141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:01.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd491a24640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c072c80 0x7fd48c142b00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd48800e9b0 tx=0x7fd48800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd4827fc640 1 -- 172.21.15.94:0/2808650218 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488017070 con 0x7fd48c072c80 2024-04-25T16:51:01.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd4827fc640 1 -- 172.21.15.94:0/2808650218 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd48800ca00 con 0x7fd48c072c80 2024-04-25T16:51:01.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd493c88640 1 -- 172.21.15.94:0/2808650218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c15ffe0 con 0x7fd48c072c80 2024-04-25T16:51:01.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd4827fc640 1 -- 172.21.15.94:0/2808650218 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48800cdd0 con 0x7fd48c072c80 2024-04-25T16:51:01.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.032+0000 7fd493c88640 1 -- 172.21.15.94:0/2808650218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c0cd8f0 con 0x7fd48c072c80 2024-04-25T16:51:01.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.036+0000 7fd493c88640 1 -- 172.21.15.94:0/2808650218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c06a730 con 0x7fd48c072c80 2024-04-25T16:51:01.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.036+0000 7fd4827fc640 1 -- 172.21.15.94:0/2808650218 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd48801a070 con 0x7fd48c072c80 2024-04-25T16:51:01.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.036+0000 7fd4827fc640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd47006f430 0x7fd470071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.036+0000 7fd4827fc640 1 -- 172.21.15.94:0/2808650218 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd48808f7f0 con 0x7fd48c072c80 2024-04-25T16:51:01.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.036+0000 7fd491223640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd47006f430 0x7fd470071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.036+0000 7fd491223640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd47006f430 0x7fd470071940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0af000 tx=0x7fd47c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.040+0000 7fd4827fc640 1 -- 172.21.15.94:0/2808650218 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd48805c430 con 0x7fd48c072c80 2024-04-25T16:51:01.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.200+0000 7fd493c88640 1 -- 172.21.15.94:0/2808650218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd48c0c9a40 con 0x7fd48c072c80 2024-04-25T16:51:01.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.204+0000 7fd4827fc640 1 -- 172.21.15.94:0/2808650218 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7fd48805bd30 con 0x7fd48c072c80 2024-04-25T16:51:01.200 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:01.200 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:59.528743+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:01.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.204+0000 7fd493c88640 1 -- 172.21.15.94:0/2808650218 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd47006f430 msgr2=0x7fd470071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.204+0000 7fd493c88640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd47006f430 0x7fd470071940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0af000 tx=0x7fd47c0a7000 comp rx=0 tx=0).stop 2024-04-25T16:51:01.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.204+0000 7fd493c88640 1 -- 172.21.15.94:0/2808650218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c072c80 msgr2=0x7fd48c142b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.204+0000 7fd493c88640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c072c80 0x7fd48c142b00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd48800e9b0 tx=0x7fd48800ee70 comp rx=0 tx=0).stop 2024-04-25T16:51:01.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.204+0000 7fd493c88640 1 -- 172.21.15.94:0/2808650218 shutdown_connections 2024-04-25T16:51:01.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.204+0000 7fd493c88640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c161ac0 0x7fd48c141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.204+0000 7fd493c88640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd47006f430 0x7fd470071940 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.204+0000 7fd493c88640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c0cdcb0 0x7fd48c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.204+0000 7fd493c88640 1 --2- 172.21.15.94:0/2808650218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48c072c80 0x7fd48c142b00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.204+0000 7fd493c88640 1 -- 172.21.15.94:0/2808650218 >> 172.21.15.94:0/2808650218 conn(0x7fd48c0d3850 msgr2=0x7fd48c0d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:01.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.204+0000 7fd493c88640 1 -- 172.21.15.94:0/2808650218 shutdown_connections 2024-04-25T16:51:01.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.204+0000 7fd493c88640 1 -- 172.21.15.94:0/2808650218 wait complete. 2024-04-25T16:51:01.234 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:51:01.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f8308b640 1 Processor -- start 2024-04-25T16:51:01.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f8308b640 1 -- start start 2024-04-25T16:51:01.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f8308b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f7c156790 0x7f2f7c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f8308b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f7c06fbe0 0x7f2f7c06fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f8308b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f7c070530 con 0x7f2f7c072740 2024-04-25T16:51:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f8308b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f2f7c156ba0 con 0x7f2f7c06fbe0 2024-04-25T16:51:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f8308b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2f7c15fd80 con 0x7f2f7c156790 2024-04-25T16:51:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f7bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f7c06fbe0 0x7f2f7c06fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f7bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f7c06fbe0 0x7f2f7c06fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48968/0 (socket says 172.21.15.94:48968) 2024-04-25T16:51:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f7bfff640 1 -- 172.21.15.94:0/2159686148 learned_addr learned my addr 172.21.15.94:0/2159686148 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f80e27640 1 --2- 172.21.15.94:0/2159686148 >> v2:172.21.15.138:3300/0 conn(0x7f2f7c156790 0x7f2f7c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f7bfff640 1 -- 172.21.15.94:0/2159686148 >> v2:172.21.15.138:3300/0 conn(0x7f2f7c156790 msgr2=0x7f2f7c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f7bfff640 1 --2- 172.21.15.94:0/2159686148 >> v2:172.21.15.138:3300/0 conn(0x7f2f7c156790 0x7f2f7c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f7bfff640 1 -- 172.21.15.94:0/2159686148 >> v1:172.21.15.94:6789/0 conn(0x7f2f7c072740 legacy=0x7f2f7c072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.372+0000 7f2f7bfff640 1 -- 172.21.15.94:0/2159686148 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7c160060 con 0x7f2f7c06fbe0 2024-04-25T16:51:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f7bfff640 1 --2- 172.21.15.94:0/2159686148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c06fbe0 0x7f2f7c06fff0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c071f10 tx=0x7f2f6801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e045dc0c968c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f7b7fe640 1 -- 172.21.15.94:0/2159686148 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f680b0070 con 0x7f2f7c06fbe0 2024-04-25T16:51:01.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f7b7fe640 1 -- 172.21.15.94:0/2159686148 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f680ab600 con 0x7f2f7c06fbe0 2024-04-25T16:51:01.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f7b7fe640 1 -- 172.21.15.94:0/2159686148 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f680ab9d0 con 0x7f2f7c06fbe0 2024-04-25T16:51:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f8308b640 1 -- 172.21.15.94:0/2159686148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c06fbe0 msgr2=0x7f2f7c06fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f8308b640 1 --2- 172.21.15.94:0/2159686148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c06fbe0 0x7f2f7c06fff0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c071f10 tx=0x7f2f6801c490 comp rx=0 tx=0).stop 2024-04-25T16:51:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f8308b640 1 -- 172.21.15.94:0/2159686148 shutdown_connections 2024-04-25T16:51:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f8308b640 1 --2- 172.21.15.94:0/2159686148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c06fbe0 0x7f2f7c06fff0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f8308b640 1 --2- 172.21.15.94:0/2159686148 >> v2:172.21.15.138:3300/0 conn(0x7f2f7c156790 0x7f2f7c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f8308b640 1 -- 172.21.15.94:0/2159686148 >> 172.21.15.94:0/2159686148 conn(0x7f2f7c0b47f0 msgr2=0x7f2f7c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f8308b640 1 -- 172.21.15.94:0/2159686148 shutdown_connections 2024-04-25T16:51:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f8308b640 1 -- 172.21.15.94:0/2159686148 wait complete. 2024-04-25T16:51:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f8308b640 1 Processor -- start 2024-04-25T16:51:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.376+0000 7f2f8308b640 1 -- start start 2024-04-25T16:51:01.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f8308b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c06fbe0 0x7f2f7c13e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f8308b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c072740 0x7f2f7c15a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f8308b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f7c156790 0x7f2f7c1f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f8308b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c0afaf0 con 0x7f2f7c072740 2024-04-25T16:51:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f8308b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c074100 con 0x7f2f7c156790 2024-04-25T16:51:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f8308b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7c0afc70 con 0x7f2f7c06fbe0 2024-04-25T16:51:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c072740 0x7f2f7c15a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c072740 0x7f2f7c15a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48984/0 (socket says 172.21.15.94:48984) 2024-04-25T16:51:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f7bfff640 1 -- 172.21.15.94:0/2504091094 learned_addr learned my addr 172.21.15.94:0/2504091094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f7bfff640 1 -- 172.21.15.94:0/2504091094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c06fbe0 msgr2=0x7f2f7c13e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f7bfff640 1 --2- 172.21.15.94:0/2504091094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c06fbe0 0x7f2f7c13e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f7bfff640 1 -- 172.21.15.94:0/2504091094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f7c156790 msgr2=0x7f2f7c1f9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f7bfff640 1 --2- 172.21.15.94:0/2504091094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f7c156790 0x7f2f7c1f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f7bfff640 1 -- 172.21.15.94:0/2504091094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7c15a8c0 con 0x7f2f7c072740 2024-04-25T16:51:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f7bfff640 1 --2- 172.21.15.94:0/2504091094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c072740 0x7f2f7c15a380 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f680ae040 tx=0x7f2f68019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f797fa640 1 -- 172.21.15.94:0/2504091094 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f680b0040 con 0x7f2f7c072740 2024-04-25T16:51:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f797fa640 1 -- 172.21.15.94:0/2504091094 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f680b4620 con 0x7f2f7c072740 2024-04-25T16:51:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f8308b640 1 -- 172.21.15.94:0/2504091094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f7c1658c0 con 0x7f2f7c072740 2024-04-25T16:51:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f797fa640 1 -- 172.21.15.94:0/2504091094 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f680b4a10 con 0x7f2f7c072740 2024-04-25T16:51:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f8308b640 1 -- 172.21.15.94:0/2504091094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f7c165e80 con 0x7f2f7c072740 2024-04-25T16:51:01.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f8308b640 1 -- 172.21.15.94:0/2504091094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f7c165630 con 0x7f2f7c072740 2024-04-25T16:51:01.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f797fa640 1 -- 172.21.15.94:0/2504091094 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2f680ab4c0 con 0x7f2f7c072740 2024-04-25T16:51:01.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f797fa640 1 --2- 172.21.15.94:0/2504091094 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f6006f430 0x7f2f60071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f797fa640 1 -- 172.21.15.94:0/2504091094 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2f68129210 con 0x7f2f7c072740 2024-04-25T16:51:01.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.380+0000 7f2f80e27640 1 --2- 172.21.15.94:0/2504091094 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f6006f430 0x7f2f60071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.384+0000 7f2f80e27640 1 --2- 172.21.15.94:0/2504091094 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f6006f430 0x7f2f60071940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0b06d0 tx=0x7f2f70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.388+0000 7f2f797fa640 1 -- 172.21.15.94:0/2504091094 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2f68016110 con 0x7f2f7c072740 2024-04-25T16:51:01.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f8308b640 1 -- 172.21.15.94:0/2504091094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f7c06e720 con 0x7f2f7c072740 2024-04-25T16:51:01.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f797fa640 1 -- 172.21.15.94:0/2504091094 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f2f680f5e50 con 0x7f2f7c072740 2024-04-25T16:51:01.565 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:01.565 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2352186041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2352186041},{"type":"v1","addr":"172.21.15.138:6833","nonce":2352186041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3210927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3210927873},{"type":"v1","addr":"172.21.15.94:6835","nonce":3210927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2237413225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2237413225},{"type":"v1","addr":"172.21.15.94:6837","nonce":2237413225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:50:54.403299+0000","modified":"2024-04-25T16:50:59.529114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/3652545132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3652545132},{"type":"v1","addr":"172.21.15.138:6835","nonce":3652545132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T16:51:01.565 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 7 2024-04-25T16:51:01.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f8308b640 1 -- 172.21.15.94:0/2504091094 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f6006f430 msgr2=0x7f2f60071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f8308b640 1 --2- 172.21.15.94:0/2504091094 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f6006f430 0x7f2f60071940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0b06d0 tx=0x7f2f70008040 comp rx=0 tx=0).stop 2024-04-25T16:51:01.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f8308b640 1 -- 172.21.15.94:0/2504091094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c072740 msgr2=0x7f2f7c15a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f8308b640 1 --2- 172.21.15.94:0/2504091094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c072740 0x7f2f7c15a380 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f680ae040 tx=0x7f2f68019230 comp rx=0 tx=0).stop 2024-04-25T16:51:01.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f8308b640 1 -- 172.21.15.94:0/2504091094 shutdown_connections 2024-04-25T16:51:01.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f8308b640 1 --2- 172.21.15.94:0/2504091094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f7c156790 0x7f2f7c1f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f8308b640 1 --2- 172.21.15.94:0/2504091094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c072740 0x7f2f7c15a380 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f8308b640 1 --2- 172.21.15.94:0/2504091094 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f6006f430 0x7f2f60071940 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f8308b640 1 --2- 172.21.15.94:0/2504091094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c06fbe0 0x7f2f7c13e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f8308b640 1 -- 172.21.15.94:0/2504091094 >> 172.21.15.94:0/2504091094 conn(0x7f2f7c0b47f0 msgr2=0x7f2f7c0b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:01.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f8308b640 1 -- 172.21.15.94:0/2504091094 shutdown_connections 2024-04-25T16:51:01.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.568+0000 7f2f8308b640 1 -- 172.21.15.94:0/2504091094 wait complete. 2024-04-25T16:51:01.584 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:01.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f091121b640 1 Processor -- start 2024-04-25T16:51:01.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f091121b640 1 -- start start 2024-04-25T16:51:01.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f091121b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f090c0d44a0 0x7f090c167f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f091121b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f090c168490 0x7f090c0d1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f091121b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f090c0d17d0 0x7f090c0d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f091121b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f090c0c8ad0 con 0x7f090c0d44a0 2024-04-25T16:51:01.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f091121b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f090c1688a0 con 0x7f090c0d17d0 2024-04-25T16:51:01.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f091121b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f090c168e00 con 0x7f090c168490 2024-04-25T16:51:01.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f090ad9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f090c0d44a0 0x7f090c167f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f090ad9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f090c0d44a0 0x7f090c167f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48986/0 (socket says 172.21.15.94:48986) 2024-04-25T16:51:01.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f090ad9d640 1 -- 172.21.15.94:0/3230814650 learned_addr learned my addr 172.21.15.94:0/3230814650 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:01.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f090a59c640 1 --2- 172.21.15.94:0/3230814650 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f090c168490 0x7f090c0d1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f090b59e640 1 --2- 172.21.15.94:0/3230814650 >> v2:172.21.15.138:3300/0 conn(0x7f090c0d17d0 0x7f090c0d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f090ad9d640 1 -- 172.21.15.94:0/3230814650 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f090c168490 msgr2=0x7f090c0d1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:01.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f090ad9d640 1 --2- 172.21.15.94:0/3230814650 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f090c168490 0x7f090c0d1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f090ad9d640 1 -- 172.21.15.94:0/3230814650 >> v2:172.21.15.138:3300/0 conn(0x7f090c0d17d0 msgr2=0x7f090c0d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:01.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f090ad9d640 1 --2- 172.21.15.94:0/3230814650 >> v2:172.21.15.138:3300/0 conn(0x7f090c0d17d0 0x7f090c0d4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.736+0000 7f090ad9d640 1 -- 172.21.15.94:0/3230814650 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090c0d3ce0 con 0x7f090c0d44a0 2024-04-25T16:51:01.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.740+0000 7f090ad9d640 1 --2- 172.21.15.94:0/3230814650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f090c0d44a0 0x7f090c167f50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f090c167eb0 tx=0x7f090401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52593d49f829f075 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.740+0000 7f0909d9b640 1 -- 172.21.15.94:0/3230814650 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09040a58a0 con 0x7f090c0d44a0 2024-04-25T16:51:01.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.740+0000 7f0909d9b640 1 -- 172.21.15.94:0/3230814650 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09040a5a40 con 0x7f090c0d44a0 2024-04-25T16:51:01.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.740+0000 7f0909d9b640 1 -- 172.21.15.94:0/3230814650 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09040a5e10 con 0x7f090c0d44a0 2024-04-25T16:51:01.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.740+0000 7f091121b640 1 -- 172.21.15.94:0/3230814650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f090c0d44a0 msgr2=0x7f090c167f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.740+0000 7f091121b640 1 --2- 172.21.15.94:0/3230814650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f090c0d44a0 0x7f090c167f50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f090c167eb0 tx=0x7f090401ca20 comp rx=0 tx=0).stop 2024-04-25T16:51:01.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.740+0000 7f091121b640 1 -- 172.21.15.94:0/3230814650 shutdown_connections 2024-04-25T16:51:01.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.740+0000 7f091121b640 1 --2- 172.21.15.94:0/3230814650 >> v2:172.21.15.138:3300/0 conn(0x7f090c0d17d0 0x7f090c0d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.740+0000 7f091121b640 1 --2- 172.21.15.94:0/3230814650 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f090c168490 0x7f090c0d1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.740+0000 7f091121b640 1 --2- 172.21.15.94:0/3230814650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f090c0d44a0 0x7f090c167f50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.740+0000 7f091121b640 1 -- 172.21.15.94:0/3230814650 >> 172.21.15.94:0/3230814650 conn(0x7f090c09e950 msgr2=0x7f090c0b85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:01.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.740+0000 7f091121b640 1 -- 172.21.15.94:0/3230814650 shutdown_connections 2024-04-25T16:51:01.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.740+0000 7f091121b640 1 -- 172.21.15.94:0/3230814650 wait complete. 2024-04-25T16:51:01.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f091121b640 1 Processor -- start 2024-04-25T16:51:01.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f091121b640 1 -- start start 2024-04-25T16:51:01.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f091121b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f090c0d17d0 0x7f090c15e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f091121b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f090c0d44a0 0x7f090c0b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f091121b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f090c168490 0x7f090c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f091121b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f090c0c8d20 con 0x7f090c0d44a0 2024-04-25T16:51:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f091121b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f090c164790 con 0x7f090c0d17d0 2024-04-25T16:51:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f091121b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f090c0d48b0 con 0x7f090c168490 2024-04-25T16:51:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f090a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f090c0d44a0 0x7f090c0b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f090a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f090c0d44a0 0x7f090c0b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48988/0 (socket says 172.21.15.94:48988) 2024-04-25T16:51:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f090a59c640 1 -- 172.21.15.94:0/593663820 learned_addr learned my addr 172.21.15.94:0/593663820 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f090ad9d640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f090c0d17d0 0x7f090c15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f090a59c640 1 -- 172.21.15.94:0/593663820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f090c168490 msgr2=0x7f090c0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f090b59e640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f090c168490 0x7f090c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f090a59c640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f090c168490 0x7f090c0b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f090a59c640 1 -- 172.21.15.94:0/593663820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f090c0d17d0 msgr2=0x7f090c15e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f090a59c640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f090c0d17d0 0x7f090c15e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f090a59c640 1 -- 172.21.15.94:0/593663820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090c0cd9a0 con 0x7f090c0d44a0 2024-04-25T16:51:01.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f090ad9d640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f090c0d17d0 0x7f090c15e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:01.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f090a59c640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f090c0d44a0 0x7f090c0b2650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f090000d620 tx=0x7f090000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f08f37fe640 1 -- 172.21.15.94:0/593663820 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090000b620 con 0x7f090c0d44a0 2024-04-25T16:51:01.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f08f37fe640 1 -- 172.21.15.94:0/593663820 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0900019070 con 0x7f090c0d44a0 2024-04-25T16:51:01.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f08f37fe640 1 -- 172.21.15.94:0/593663820 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090000b9e0 con 0x7f090c0d44a0 2024-04-25T16:51:01.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f091121b640 1 -- 172.21.15.94:0/593663820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090c0cdc80 con 0x7f090c0d44a0 2024-04-25T16:51:01.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f091121b640 1 -- 172.21.15.94:0/593663820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f090c0ce240 con 0x7f090c0d44a0 2024-04-25T16:51:01.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.744+0000 7f091121b640 1 -- 172.21.15.94:0/593663820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f090c06a730 con 0x7f090c0d44a0 2024-04-25T16:51:01.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.748+0000 7f08f37fe640 1 -- 172.21.15.94:0/593663820 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f090000bb80 con 0x7f090c0d44a0 2024-04-25T16:51:01.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.748+0000 7f08f37fe640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f08f806f430 0x7f08f8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.748+0000 7f08f37fe640 1 -- 172.21.15.94:0/593663820 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f09000903c0 con 0x7f090c0d44a0 2024-04-25T16:51:01.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.748+0000 7f090ad9d640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f08f806f430 0x7f08f8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.748+0000 7f090ad9d640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f08f806f430 0x7f08f8071940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f09040ae000 tx=0x7f09040a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.752+0000 7f08f37fe640 1 -- 172.21.15.94:0/593663820 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f090005d0e0 con 0x7f090c0d44a0 2024-04-25T16:51:01.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.900+0000 7f091121b640 1 -- 172.21.15.94:0/593663820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f090c0d4c40 con 0x7f090c0d44a0 2024-04-25T16:51:01.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.904+0000 7f08f37fe640 1 -- 172.21.15.94:0/593663820 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f090005c9e0 con 0x7f090c0d44a0 2024-04-25T16:51:01.903 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:01.903 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:59.528743+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:01.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.908+0000 7f091121b640 1 -- 172.21.15.94:0/593663820 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f08f806f430 msgr2=0x7f08f8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.908+0000 7f091121b640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f08f806f430 0x7f08f8071940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f09040ae000 tx=0x7f09040a7070 comp rx=0 tx=0).stop 2024-04-25T16:51:01.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.908+0000 7f091121b640 1 -- 172.21.15.94:0/593663820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f090c0d44a0 msgr2=0x7f090c0b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.908+0000 7f091121b640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f090c0d44a0 0x7f090c0b2650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f090000d620 tx=0x7f090000dae0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.908+0000 7f091121b640 1 -- 172.21.15.94:0/593663820 shutdown_connections 2024-04-25T16:51:01.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.908+0000 7f091121b640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f08f806f430 0x7f08f8071940 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.908+0000 7f091121b640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f090c168490 0x7f090c0b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.908+0000 7f091121b640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f090c0d44a0 0x7f090c0b2650 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.908+0000 7f091121b640 1 --2- 172.21.15.94:0/593663820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f090c0d17d0 0x7f090c15e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.908+0000 7f091121b640 1 -- 172.21.15.94:0/593663820 >> 172.21.15.94:0/593663820 conn(0x7f090c09e950 msgr2=0x7f090c163d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:01.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.908+0000 7f091121b640 1 -- 172.21.15.94:0/593663820 shutdown_connections 2024-04-25T16:51:01.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:01.908+0000 7f091121b640 1 -- 172.21.15.94:0/593663820 wait complete. 2024-04-25T16:51:02.326 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T16:50:54.403299+0000', 'modified': '2024-04-25T16:50:59.529114+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4297}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4297': {'gid': 4297, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6835/3652545132', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 3652545132}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 3652545132}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4297, 'qdb_cluster': [4297]} 2024-04-25T16:51:02.326 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T16:51:02.326 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T16:51:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.488+0000 7f1bad169640 1 Processor -- start 2024-04-25T16:51:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.488+0000 7f1bad169640 1 -- start start 2024-04-25T16:51:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1bad169640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ba80d3450 0x7f1ba80d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1bad169640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1ba80d3da0 0x7f1ba80b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1bad169640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba8164510 con 0x7f1ba80b7510 2024-04-25T16:51:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1bad169640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba8164a90 con 0x7f1ba80d3da0 2024-04-25T16:51:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1bad169640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba80d51b0 con 0x7f1ba80d3450 2024-04-25T16:51:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1b9ffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1ba80d3da0 0x7f1ba80b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1b9ffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1ba80d3da0 0x7f1ba80b6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48990/0 (socket says 172.21.15.94:48990) 2024-04-25T16:51:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1b9ffff640 1 -- 172.21.15.94:0/3275952067 learned_addr learned my addr 172.21.15.94:0/3275952067 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1b9ffff640 1 -- 172.21.15.94:0/3275952067 >> v2:172.21.15.138:3300/0 conn(0x7f1ba80d3450 msgr2=0x7f1ba80d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/3275952067 >> v2:172.21.15.138:3300/0 conn(0x7f1ba80d3450 0x7f1ba80d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1b9ffff640 1 --2- 172.21.15.94:0/3275952067 >> v2:172.21.15.138:3300/0 conn(0x7f1ba80d3450 0x7f1ba80d3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1b9ffff640 1 -- 172.21.15.94:0/3275952067 >> v1:172.21.15.138:6789/0 conn(0x7f1ba80b7510 legacy=0x7f1ba80d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1b9ffff640 1 -- 172.21.15.94:0/3275952067 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba81647b0 con 0x7f1ba80d3da0 2024-04-25T16:51:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1b9ffff640 1 --2- 172.21.15.94:0/3275952067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba80d3da0 0x7f1ba80b6fd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8164c10 tx=0x7f1b9001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5527bec1f41bc806 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1ba659c640 1 -- 172.21.15.94:0/3275952067 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900ab620 con 0x7f1ba80d3da0 2024-04-25T16:51:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1ba659c640 1 -- 172.21.15.94:0/3275952067 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b900b0070 con 0x7f1ba80d3da0 2024-04-25T16:51:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.492+0000 7f1ba659c640 1 -- 172.21.15.94:0/3275952067 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900ab9e0 con 0x7f1ba80d3da0 2024-04-25T16:51:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 -- 172.21.15.94:0/3275952067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba80d3da0 msgr2=0x7f1ba80b6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 --2- 172.21.15.94:0/3275952067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba80d3da0 0x7f1ba80b6fd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8164c10 tx=0x7f1b9001c420 comp rx=0 tx=0).stop 2024-04-25T16:51:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 -- 172.21.15.94:0/3275952067 shutdown_connections 2024-04-25T16:51:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 --2- 172.21.15.94:0/3275952067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba80d3da0 0x7f1ba80b6fd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 --2- 172.21.15.94:0/3275952067 >> v2:172.21.15.138:3300/0 conn(0x7f1ba80d3450 0x7f1ba80d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 -- 172.21.15.94:0/3275952067 >> 172.21.15.94:0/3275952067 conn(0x7f1ba809e8d0 msgr2=0x7f1ba80b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 -- 172.21.15.94:0/3275952067 shutdown_connections 2024-04-25T16:51:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 -- 172.21.15.94:0/3275952067 wait complete. 2024-04-25T16:51:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 Processor -- start 2024-04-25T16:51:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 -- start start 2024-04-25T16:51:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba80b7510 0x7f1ba8154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ba80d3450 0x7f1ba8151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba80d3da0 0x7f1ba8151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba80b7d30 con 0x7f1ba80b7510 2024-04-25T16:51:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba80d41b0 con 0x7f1ba80d3da0 2024-04-25T16:51:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1bad169640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba80b8cd0 con 0x7f1ba80d3450 2024-04-25T16:51:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.496+0000 7f1ba6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba80b7510 0x7f1ba8154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1ba6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba80b7510 0x7f1ba8154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48996/0 (socket says 172.21.15.94:48996) 2024-04-25T16:51:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1ba6d9d640 1 -- 172.21.15.94:0/2225174247 learned_addr learned my addr 172.21.15.94:0/2225174247 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1ba759e640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba80d3da0 0x7f1ba8151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1b9ffff640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ba80d3450 0x7f1ba8151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1ba6d9d640 1 -- 172.21.15.94:0/2225174247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ba80d3450 msgr2=0x7f1ba8151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ba80d3450 0x7f1ba8151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1ba6d9d640 1 -- 172.21.15.94:0/2225174247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba80d3da0 msgr2=0x7f1ba8151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba80d3da0 0x7f1ba8151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1ba6d9d640 1 -- 172.21.15.94:0/2225174247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba8152a50 con 0x7f1ba80b7510 2024-04-25T16:51:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba80b7510 0x7f1ba8154f40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800c820 tx=0x7f1b9800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1b9f7fe640 1 -- 172.21.15.94:0/2225174247 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98004030 con 0x7f1ba80b7510 2024-04-25T16:51:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1b9f7fe640 1 -- 172.21.15.94:0/2225174247 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b980041d0 con 0x7f1ba80b7510 2024-04-25T16:51:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1b9f7fe640 1 -- 172.21.15.94:0/2225174247 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98007830 con 0x7f1ba80b7510 2024-04-25T16:51:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1bad169640 1 -- 172.21.15.94:0/2225174247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba8155f90 con 0x7f1ba80b7510 2024-04-25T16:51:02.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1bad169640 1 -- 172.21.15.94:0/2225174247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba814cc40 con 0x7f1ba80b7510 2024-04-25T16:51:02.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1b9f7fe640 1 -- 172.21.15.94:0/2225174247 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1b980079d0 con 0x7f1ba80b7510 2024-04-25T16:51:02.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1b9f7fe640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1b8806f400 0x7f1b88071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1b9ffff640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1b8806f400 0x7f1b88071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1b9f7fe640 1 -- 172.21.15.94:0/2225174247 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1b9808f580 con 0x7f1ba80b7510 2024-04-25T16:51:02.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1b9ffff640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1b8806f400 0x7f1b88071910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b900af000 tx=0x7f1b900a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:02.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1bb44c6000 1 Processor -- start 2024-04-25T16:51:02.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.500+0000 7f1bb44c6000 1 -- start start 2024-04-25T16:51:02.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 0x5644e921fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90adea0 0x5644e9220420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e9220960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5644e908f160 con 0x5644e8ca6280 2024-04-25T16:51:02.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5644e9024e20 con 0x5644e90c1c20 2024-04-25T16:51:02.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5644e8fdda20 con 0x5644e90adea0 2024-04-25T16:51:02.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1ba759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e9220960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1ba759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e9220960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49010/0 (socket says 172.21.15.94:49010) 2024-04-25T16:51:02.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1ba759e640 1 -- 172.21.15.94:0/3969462226 learned_addr learned my addr 172.21.15.94:0/3969462226 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/3969462226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 0x5644e921fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1ba759e640 1 -- 172.21.15.94:0/3969462226 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90adea0 msgr2=0x5644e9220420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1b9ffff640 1 --2- 172.21.15.94:0/3969462226 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90adea0 0x5644e9220420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1ba759e640 1 --2- 172.21.15.94:0/3969462226 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90adea0 0x5644e9220420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1ba759e640 1 -- 172.21.15.94:0/3969462226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 msgr2=0x5644e921fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1ba759e640 1 --2- 172.21.15.94:0/3969462226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 0x5644e921fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1ba759e640 1 -- 172.21.15.94:0/3969462226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e8fe2100 con 0x5644e8ca6280 2024-04-25T16:51:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/3969462226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 0x5644e921fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1b9ffff640 1 --2- 172.21.15.94:0/3969462226 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90adea0 0x5644e9220420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1ba759e640 1 --2- 172.21.15.94:0/3969462226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e9220960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009f60 tx=0x7f1ba0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d05cf4dfcc38807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1b9cff9640 1 -- 172.21.15.94:0/3969462226 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba001a070 con 0x5644e8ca6280 2024-04-25T16:51:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1b9cff9640 1 -- 172.21.15.94:0/3969462226 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba000a5f0 con 0x5644e8ca6280 2024-04-25T16:51:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1b9cff9640 1 -- 172.21.15.94:0/3969462226 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000a9c0 con 0x5644e8ca6280 2024-04-25T16:51:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 -- 172.21.15.94:0/3969462226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 msgr2=0x5644e9220960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/3969462226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e9220960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009f60 tx=0x7f1ba0009f90 comp rx=0 tx=0).stop 2024-04-25T16:51:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 -- 172.21.15.94:0/3969462226 shutdown_connections 2024-04-25T16:51:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/3969462226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e9220960 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/3969462226 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90adea0 0x5644e9220420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/3969462226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 0x5644e921fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 -- 172.21.15.94:0/3969462226 >> 172.21.15.94:0/3969462226 conn(0x5644e8ce5ff0 msgr2=0x5644e91d54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 -- 172.21.15.94:0/3969462226 shutdown_connections 2024-04-25T16:51:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 -- 172.21.15.94:0/3969462226 wait complete. 2024-04-25T16:51:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 -- start start 2024-04-25T16:51:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.504+0000 7f1bb44c6000 1 Processor -- start 2024-04-25T16:51:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e8ca6280 0x5644e9226e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e90adea0 0x5644e92273a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 0x5644e91e20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5644e9032220 con 0x5644e90adea0 2024-04-25T16:51:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5644e8fc8ea0 con 0x5644e90c1c20 2024-04-25T16:51:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5644e901fb00 con 0x5644e8ca6280 2024-04-25T16:51:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 10 client.0 _ll_register_callbacks cb 0x5644e9031dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e90adea0 0x5644e92273a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e90adea0 0x5644e92273a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49012/0 (socket says 172.21.15.94:49012) 2024-04-25T16:51:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b9ffff640 1 -- 172.21.15.94:0/3616602701 learned_addr learned my addr 172.21.15.94:0/3616602701 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e8ca6280 0x5644e9226e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1ba759e640 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 0x5644e91e20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b9ffff640 1 -- 172.21.15.94:0/3616602701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e8ca6280 msgr2=0x5644e9226e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b9ffff640 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e8ca6280 0x5644e9226e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b9ffff640 1 -- 172.21.15.94:0/3616602701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 msgr2=0x5644e91e20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b9ffff640 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 0x5644e91e20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b9ffff640 1 -- 172.21.15.94:0/3616602701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e8fc4500 con 0x5644e90adea0 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1ba759e640 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 0x5644e91e20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b9ffff640 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e90adea0 0x5644e92273a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1b90000c10 tx=0x7f1b9001ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabd323e0b69b0c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b85ffb640 1 -- 172.21.15.94:0/3616602701 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900cb480 con 0x5644e90adea0 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b85ffb640 1 -- 172.21.15.94:0/3616602701 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1b900cb620 con 0x5644e90adea0 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 1 -- 172.21.15.94:0/3616602701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ba000a5f0 con 0x5644e90adea0 2024-04-25T16:51:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b85ffb640 1 -- 172.21.15.94:0/3616602701 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f1b900cb820 con 0x5644e90adea0 2024-04-25T16:51:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 10 client.4389 fetch_fsmap learned FSMap version 7 2024-04-25T16:51:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 1 -- 172.21.15.94:0/3616602701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5644e8faaea0 con 0x5644e90adea0 2024-04-25T16:51:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b85ffb640 1 -- 172.21.15.94:0/3616602701 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba000a5f0 con 0x5644e90adea0 2024-04-25T16:51:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b85ffb640 1 -- 172.21.15.94:0/3616602701 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f1b900cb9c0 con 0x5644e90adea0 2024-04-25T16:51:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 10 client.4389 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T16:51:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 10 client.4389 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T16:51:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 20 client.4389 populate_metadata read hostname 'smithi094' 2024-04-25T16:51:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x5644e8cca7d0 0x5644e91e2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 4 client.4389 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:51:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1bb44c6000 1 -- 172.21.15.94:0/3616602701 --> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644e8fbdbe0 con 0x5644e8cca7d0 2024-04-25T16:51:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x5644e8cca7d0 0x5644e91e2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1b85ffb640 10 client.4389 ms_handle_connect on v2:172.21.15.138:6834/3652545132 2024-04-25T16:51:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.508+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x5644e8cca7d0 0x5644e91e2e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e741539748ffe4 server_cookie=3dfced9d58b50ae9 in_seq=0 out_seq=0 2024-04-25T16:51:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1b85ffb640 1 -- 172.21.15.94:0/3616602701 <== mds.0 v2:172.21.15.138:6834/3652545132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644e8fbdbe0 con 0x5644e8cca7d0 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1b85ffb640 10 client.4389 handle_command_reply: tid=0 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1bb44c6000 1 client.4389 shutdown 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1bb44c6000 1 -- 172.21.15.94:0/3616602701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e90adea0 msgr2=0x5644e92273a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e90adea0 0x5644e92273a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1b90000c10 tx=0x7f1b9001ac60 comp rx=0 tx=0).stop 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1bb44c6000 1 -- 172.21.15.94:0/3616602701 shutdown_connections 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x5644e8cca7d0 0x5644e91e2e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 0x5644e91e20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e90adea0 0x5644e92273a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1b85ffb640 0 client.4389 ms_handle_reset on v2:172.21.15.138:6834/3652545132 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/3616602701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e8ca6280 0x5644e9226e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1bb44c6000 1 -- 172.21.15.94:0/3616602701 >> 172.21.15.94:0/3616602701 conn(0x5644e8ce5ff0 msgr2=0x5644e91d5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1bb44c6000 1 -- 172.21.15.94:0/3616602701 shutdown_connections 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1bb44c6000 1 -- 172.21.15.94:0/3616602701 wait complete. 2024-04-25T16:51:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.512+0000 7f1bb44c6000 20 client.4389 trim_cache size 0 max 16384 2024-04-25T16:51:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 Processor -- start 2024-04-25T16:51:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 -- start start 2024-04-25T16:51:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e922c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90adea0 0x5644e9235ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 0x5644e9236530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8207170 con 0x5644e8ca6280 2024-04-25T16:51:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba82061b0 con 0x5644e90c1c20 2024-04-25T16:51:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba8205bc0 con 0x5644e90adea0 2024-04-25T16:51:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1ba6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e922c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1ba6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e922c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49020/0 (socket says 172.21.15.94:49020) 2024-04-25T16:51:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1ba6d9d640 1 -- 172.21.15.94:0/1828142451 learned_addr learned my addr 172.21.15.94:0/1828142451 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1ba6d9d640 1 -- 172.21.15.94:0/1828142451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90adea0 msgr2=0x5644e9235ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/1828142451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90adea0 0x5644e9235ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1ba6d9d640 1 -- 172.21.15.94:0/1828142451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 msgr2=0x5644e9236530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/1828142451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 0x5644e9236530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1ba6d9d640 1 -- 172.21.15.94:0/1828142451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e91e2430 con 0x5644e8ca6280 2024-04-25T16:51:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/1828142451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e922c4d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b980581f0 tx=0x7f1b98006ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bba3e4371cbd87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1b87fff640 1 -- 172.21.15.94:0/1828142451 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9801e040 con 0x5644e8ca6280 2024-04-25T16:51:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1b87fff640 1 -- 172.21.15.94:0/1828142451 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b9805c130 con 0x5644e8ca6280 2024-04-25T16:51:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1b87fff640 1 -- 172.21.15.94:0/1828142451 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9805ba30 con 0x5644e8ca6280 2024-04-25T16:51:02.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 -- 172.21.15.94:0/1828142451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 msgr2=0x5644e922c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/1828142451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e922c4d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b980581f0 tx=0x7f1b98006ab0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 -- 172.21.15.94:0/1828142451 shutdown_connections 2024-04-25T16:51:02.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/1828142451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90c1c20 0x5644e9236530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/1828142451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90adea0 0x5644e9235ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/1828142451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e922c4d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 -- 172.21.15.94:0/1828142451 >> 172.21.15.94:0/1828142451 conn(0x5644e8ce5ff0 msgr2=0x5644e91dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 -- 172.21.15.94:0/1828142451 shutdown_connections 2024-04-25T16:51:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.532+0000 7f1bb44c6000 1 -- 172.21.15.94:0/1828142451 wait complete. 2024-04-25T16:51:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 1 -- start start 2024-04-25T16:51:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 1 Processor -- start 2024-04-25T16:51:02.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e9292590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90adea0 0x5644e9292ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90c1c20 0x5644e922a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5644e9024130 con 0x5644e8ca6280 2024-04-25T16:51:02.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5644e8fba7d0 con 0x5644e90adea0 2024-04-25T16:51:02.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5644e9029450 con 0x5644e90c1c20 2024-04-25T16:51:02.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 10 client.0 _ll_register_callbacks cb 0x7f1b900b9050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:02.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1ba6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e9292590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1ba6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e9292590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49028/0 (socket says 172.21.15.94:49028) 2024-04-25T16:51:02.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1ba6d9d640 1 -- 172.21.15.94:0/4053395930 learned_addr learned my addr 172.21.15.94:0/4053395930 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:02.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1b9ffff640 1 --2- 172.21.15.94:0/4053395930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90adea0 0x5644e9292ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1ba759e640 1 --2- 172.21.15.94:0/4053395930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90c1c20 0x5644e922a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1ba6d9d640 1 -- 172.21.15.94:0/4053395930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90c1c20 msgr2=0x5644e922a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/4053395930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90c1c20 0x5644e922a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1ba6d9d640 1 -- 172.21.15.94:0/4053395930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90adea0 msgr2=0x5644e9292ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/4053395930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90adea0 0x5644e9292ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1ba6d9d640 1 -- 172.21.15.94:0/4053395930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e8fddc70 con 0x5644e8ca6280 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1ba6d9d640 1 --2- 172.21.15.94:0/4053395930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e9292590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1b98057ab0 tx=0x7f1b90004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00fe14d04e4490b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1b85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1b85ffb640 1 -- 172.21.15.94:0/4053395930 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9801e040 con 0x5644e8ca6280 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1b85ffb640 1 -- 172.21.15.94:0/4053395930 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1b9808db90 con 0x5644e8ca6280 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 1 -- 172.21.15.94:0/4053395930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b9805c130 con 0x5644e8ca6280 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1b85ffb640 1 -- 172.21.15.94:0/4053395930 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f1b98064890 con 0x5644e8ca6280 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 10 client.4395 fetch_fsmap learned FSMap version 7 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 1 -- 172.21.15.94:0/4053395930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5644e90a7180 con 0x5644e8ca6280 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1b85ffb640 1 -- 172.21.15.94:0/4053395930 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b9805c130 con 0x5644e8ca6280 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1b85ffb640 1 -- 172.21.15.94:0/4053395930 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f1b9808f0b0 con 0x5644e8ca6280 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 10 client.4395 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T16:51:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 10 client.4395 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 20 client.4395 populate_metadata read hostname 'smithi094' 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/4053395930 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x5644e8cb8830 0x5644e922b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 4 client.4395 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1bb44c6000 1 -- 172.21.15.94:0/4053395930 --> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] -- command(tid 0: {"prefix": "status"}) -- 0x5644e9032a30 con 0x5644e8cb8830 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.536+0000 7f1b9ffff640 1 --2- 172.21.15.94:0/4053395930 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x5644e8cb8830 0x5644e922b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1b85ffb640 10 client.4395 ms_handle_connect on v2:172.21.15.138:6834/3652545132 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1b9ffff640 1 --2- 172.21.15.94:0/4053395930 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x5644e8cb8830 0x5644e922b250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb516657cce96ac5 server_cookie=7ea9e9e70fd0dd97 in_seq=0 out_seq=0 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1b85ffb640 1 -- 172.21.15.94:0/4053395930 <== mds.0 v2:172.21.15.138:6834/3652545132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5644e9032a30 con 0x5644e8cb8830 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1b85ffb640 10 client.4395 handle_command_reply: tid=0 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bb44c6000 1 client.4395 shutdown 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bb44c6000 1 -- 172.21.15.94:0/4053395930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 msgr2=0x5644e9292590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/4053395930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e9292590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1b98057ab0 tx=0x7f1b90004740 comp rx=0 tx=0).stop 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bb44c6000 1 -- 172.21.15.94:0/4053395930 shutdown_connections 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/4053395930 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x5644e8cb8830 0x5644e922b250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/4053395930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644e90c1c20 0x5644e922a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/4053395930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644e90adea0 0x5644e9292ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1b85ffb640 0 client.4395 ms_handle_reset on v2:172.21.15.138:6834/3652545132 2024-04-25T16:51:02.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bb44c6000 1 --2- 172.21.15.94:0/4053395930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644e8ca6280 0x5644e9292590 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bb44c6000 1 -- 172.21.15.94:0/4053395930 >> 172.21.15.94:0/4053395930 conn(0x5644e8ce5ff0 msgr2=0x5644e9225040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:02.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bb44c6000 1 -- 172.21.15.94:0/4053395930 shutdown_connections 2024-04-25T16:51:02.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bb44c6000 1 -- 172.21.15.94:0/4053395930 wait complete. 2024-04-25T16:51:02.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bb44c6000 20 client.4395 trim_cache size 0 max 16384 2024-04-25T16:51:02.541 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:51:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "cc26689a-ed7a-405e-9abc-b7fa2c1906cf", 2024-04-25T16:51:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-04-25T16:51:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4297, 2024-04-25T16:51:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T16:51:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T16:51:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T16:51:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 7.4608496200000003, 2024-04-25T16:51:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 7, 2024-04-25T16:51:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 24, 2024-04-25T16:51:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T16:51:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 15.090580473999999 2024-04-25T16:51:02.542 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:51:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bad169640 1 -- 172.21.15.94:0/2225174247 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1b8806f400 msgr2=0x7f1b88071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bad169640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1b8806f400 0x7f1b88071910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b900af000 tx=0x7f1b900a7000 comp rx=0 tx=0).stop 2024-04-25T16:51:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bad169640 1 -- 172.21.15.94:0/2225174247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba80b7510 msgr2=0x7f1ba8154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bad169640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba80b7510 0x7f1ba8154f40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800c820 tx=0x7f1b9800cce0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bad169640 1 -- 172.21.15.94:0/2225174247 shutdown_connections 2024-04-25T16:51:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bad169640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba80d3da0 0x7f1ba8151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bad169640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1b8806f400 0x7f1b88071910 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bad169640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ba80d3450 0x7f1ba8151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bad169640 1 --2- 172.21.15.94:0/2225174247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba80b7510 0x7f1ba8154f40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bad169640 1 -- 172.21.15.94:0/2225174247 >> 172.21.15.94:0/2225174247 conn(0x7f1ba809e8d0 msgr2=0x7f1ba8163810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bad169640 1 -- 172.21.15.94:0/2225174247 shutdown_connections 2024-04-25T16:51:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.540+0000 7f1bad169640 1 -- 172.21.15.94:0/2225174247 wait complete. 2024-04-25T16:51:02.555 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T16:51:02.555 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:51:02.555 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T16:51:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0ce06640 1 Processor -- start 2024-04-25T16:51:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0ce06640 1 -- start start 2024-04-25T16:51:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0ce06640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fea0806f6c0 0x7fea080d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0ce06640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea0815ff50 0x7fea080d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0ce06640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea080d1d40 con 0x7fea080b7050 2024-04-25T16:51:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0ce06640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fea080d2fd0 con 0x7fea0806f6c0 2024-04-25T16:51:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0ce06640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fea080d2070 con 0x7fea0815ff50 2024-04-25T16:51:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0659c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fea0806f6c0 0x7fea080d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0659c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fea0806f6c0 0x7fea080d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49034/0 (socket says 172.21.15.94:49034) 2024-04-25T16:51:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0659c640 1 -- 172.21.15.94:0/1875269133 learned_addr learned my addr 172.21.15.94:0/1875269133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fe9fdd9b640 1 --2- 172.21.15.94:0/1875269133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea0815ff50 0x7fea080d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0659c640 1 -- 172.21.15.94:0/1875269133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea0815ff50 msgr2=0x7fea080d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0659c640 1 --2- 172.21.15.94:0/1875269133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea0815ff50 0x7fea080d3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0659c640 1 -- 172.21.15.94:0/1875269133 >> v1:172.21.15.138:6789/0 conn(0x7fea080b7050 legacy=0x7fea080b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fea0659c640 1 -- 172.21.15.94:0/1875269133 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea080b2240 con 0x7fea0806f6c0 2024-04-25T16:51:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.688+0000 7fe9fdd9b640 1 --2- 172.21.15.94:0/1875269133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea0815ff50 0x7fea080d3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:02.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.692+0000 7fea0659c640 1 --2- 172.21.15.94:0/1875269133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea0806f6c0 0x7fea080d34d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009860 tx=0x7fe9f801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d82c3e3320acca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:02.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.692+0000 7fea05d9b640 1 -- 172.21.15.94:0/1875269133 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80a9620 con 0x7fea0806f6c0 2024-04-25T16:51:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.692+0000 7fea05d9b640 1 -- 172.21.15.94:0/1875269133 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f80a97c0 con 0x7fea0806f6c0 2024-04-25T16:51:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.692+0000 7fea05d9b640 1 -- 172.21.15.94:0/1875269133 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80a9b90 con 0x7fea0806f6c0 2024-04-25T16:51:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.692+0000 7fea0ce06640 1 -- 172.21.15.94:0/1875269133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea0806f6c0 msgr2=0x7fea080d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.692+0000 7fea0ce06640 1 --2- 172.21.15.94:0/1875269133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea0806f6c0 0x7fea080d34d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009860 tx=0x7fe9f801c540 comp rx=0 tx=0).stop 2024-04-25T16:51:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.692+0000 7fea0ce06640 1 -- 172.21.15.94:0/1875269133 shutdown_connections 2024-04-25T16:51:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.692+0000 7fea0ce06640 1 --2- 172.21.15.94:0/1875269133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea0815ff50 0x7fea080d3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.692+0000 7fea0ce06640 1 --2- 172.21.15.94:0/1875269133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea0806f6c0 0x7fea080d34d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.692+0000 7fea0ce06640 1 -- 172.21.15.94:0/1875269133 >> 172.21.15.94:0/1875269133 conn(0x7fea0809e950 msgr2=0x7fea0808d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.692+0000 7fea0ce06640 1 -- 172.21.15.94:0/1875269133 shutdown_connections 2024-04-25T16:51:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.692+0000 7fea0ce06640 1 -- 172.21.15.94:0/1875269133 wait complete. 2024-04-25T16:51:02.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fea0ce06640 1 Processor -- start 2024-04-25T16:51:02.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fea0ce06640 1 -- start start 2024-04-25T16:51:02.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fea0ce06640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea0806f6c0 0x7fea0806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fea0ce06640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea080b7050 0x7fea08154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fea0ce06640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea0815ff50 0x7fea080caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fea0ce06640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea080d2aa0 con 0x7fea080b7050 2024-04-25T16:51:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fea0ce06640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fea0806fe20 con 0x7fea0806f6c0 2024-04-25T16:51:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fea0ce06640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fea0816d1e0 con 0x7fea0815ff50 2024-04-25T16:51:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fe9fdd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea080b7050 0x7fea08154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fe9fdd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea080b7050 0x7fea08154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49044/0 (socket says 172.21.15.94:49044) 2024-04-25T16:51:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fe9fdd9b640 1 -- 172.21.15.94:0/4100048129 learned_addr learned my addr 172.21.15.94:0/4100048129 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fe9fdd9b640 1 -- 172.21.15.94:0/4100048129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea0815ff50 msgr2=0x7fea080caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fe9fdd9b640 1 --2- 172.21.15.94:0/4100048129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea0815ff50 0x7fea080caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fe9fdd9b640 1 -- 172.21.15.94:0/4100048129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea0806f6c0 msgr2=0x7fea0806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fea0659c640 1 --2- 172.21.15.94:0/4100048129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea0806f6c0 0x7fea0806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fe9fdd9b640 1 --2- 172.21.15.94:0/4100048129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea0806f6c0 0x7fea0806edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fe9fdd9b640 1 -- 172.21.15.94:0/4100048129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0806df70 con 0x7fea080b7050 2024-04-25T16:51:02.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fe9fdd9b640 1 --2- 172.21.15.94:0/4100048129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea080b7050 0x7fea08154f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000d800 tx=0x7fe9f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:02.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fe9ff7fe640 1 -- 172.21.15.94:0/4100048129 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0018020 con 0x7fea080b7050 2024-04-25T16:51:02.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fe9ff7fe640 1 -- 172.21.15.94:0/4100048129 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f00096c0 con 0x7fea080b7050 2024-04-25T16:51:02.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fea0ce06640 1 -- 172.21.15.94:0/4100048129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0806c730 con 0x7fea080b7050 2024-04-25T16:51:02.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fe9ff7fe640 1 -- 172.21.15.94:0/4100048129 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0009ab0 con 0x7fea080b7050 2024-04-25T16:51:02.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.696+0000 7fea0ce06640 1 -- 172.21.15.94:0/4100048129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea080ceb50 con 0x7fea080b7050 2024-04-25T16:51:02.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.700+0000 7fea0ce06640 1 -- 172.21.15.94:0/4100048129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea08076830 con 0x7fea080b7050 2024-04-25T16:51:02.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.700+0000 7fe9ff7fe640 1 -- 172.21.15.94:0/4100048129 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe9f001b020 con 0x7fea080b7050 2024-04-25T16:51:02.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.700+0000 7fe9ff7fe640 1 --2- 172.21.15.94:0/4100048129 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe9e806f430 0x7fe9e8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:02.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.700+0000 7fe9ff7fe640 1 -- 172.21.15.94:0/4100048129 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe9f008faa0 con 0x7fea080b7050 2024-04-25T16:51:02.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.700+0000 7fea0659c640 1 --2- 172.21.15.94:0/4100048129 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe9e806f430 0x7fe9e8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:02.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.700+0000 7fea0659c640 1 --2- 172.21.15.94:0/4100048129 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe9e806f430 0x7fe9e8071940 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80ae000 tx=0x7fe9f80a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:02.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.704+0000 7fe9ff7fe640 1 -- 172.21.15.94:0/4100048129 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe9f0022020 con 0x7fea080b7050 2024-04-25T16:51:02.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.892+0000 7fea0ce06640 1 -- 172.21.15.94:0/4100048129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fea0815a350 con 0x7fea080b7050 2024-04-25T16:51:02.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.892+0000 7fe9ff7fe640 1 -- 172.21.15.94:0/4100048129 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7fe9f005c6e0 con 0x7fea080b7050 2024-04-25T16:51:02.891 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T16:51:02.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.896+0000 7fea0ce06640 1 -- 172.21.15.94:0/4100048129 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe9e806f430 msgr2=0x7fe9e8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.896+0000 7fea0ce06640 1 --2- 172.21.15.94:0/4100048129 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe9e806f430 0x7fe9e8071940 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80ae000 tx=0x7fe9f80a7000 comp rx=0 tx=0).stop 2024-04-25T16:51:02.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.896+0000 7fea0ce06640 1 -- 172.21.15.94:0/4100048129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea080b7050 msgr2=0x7fea08154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:02.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.896+0000 7fea0ce06640 1 --2- 172.21.15.94:0/4100048129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea080b7050 0x7fea08154f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000d800 tx=0x7fe9f000dcc0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.896+0000 7fea0ce06640 1 -- 172.21.15.94:0/4100048129 shutdown_connections 2024-04-25T16:51:02.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.896+0000 7fea0ce06640 1 --2- 172.21.15.94:0/4100048129 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe9e806f430 0x7fe9e8071940 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.896+0000 7fea0ce06640 1 --2- 172.21.15.94:0/4100048129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea0815ff50 0x7fea080caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.896+0000 7fea0ce06640 1 --2- 172.21.15.94:0/4100048129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea080b7050 0x7fea08154f20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.896+0000 7fea0ce06640 1 --2- 172.21.15.94:0/4100048129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea0806f6c0 0x7fea0806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:02.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.896+0000 7fea0ce06640 1 -- 172.21.15.94:0/4100048129 >> 172.21.15.94:0/4100048129 conn(0x7fea0809e950 msgr2=0x7fea08160ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:02.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.896+0000 7fea0ce06640 1 -- 172.21.15.94:0/4100048129 shutdown_connections 2024-04-25T16:51:02.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:02.896+0000 7fea0ce06640 1 -- 172.21.15.94:0/4100048129 wait complete. 2024-04-25T16:51:02.911 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T16:51:02.911 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T16:51:02.911 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-04-25T16:51:02.911 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T16:51:02.911 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:51:02.911 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T16:51:02.911 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T16:51:02.912 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T16:51:02.912 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T16:51:02.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:02.912 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-04-25T16:51:02.917 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:51:02.917 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:51:02.917 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:51:02.917 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:02.917 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-25T16:51:02.917 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:02.917 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:02.917 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:02.917 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42463sec preferred_lft 42463sec 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:f8:37:e2:6a brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:02.918 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T16:51:02.919 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:02.919 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T16:51:02.919 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:02.919 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:02.919 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-04-25T16:51:02.920 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-04-25T16:51:02.920 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-04-25T16:51:02.920 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T16:51:02.920 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:03.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:03.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:03.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:03.042 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T16:51:03.093 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-04-25T16:51:03.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:03.094 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-04-25T16:51:03.227 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-04-25T16:51:03.228 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T16:51:03.228 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T16:51:03.228 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T16:51:03.228 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T16:51:03.228 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T16:51:03.228 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T16:51:03.228 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:03.228 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:03.228 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:03.228 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T16:51:03.229 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:03.229 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:03.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T16:51:03.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T16:51:03.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T16:51:03.229 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:03.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:03.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:03.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:03.311 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T16:51:03.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:03.357 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-04-25T16:51:03.405 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:03.405 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:03.405 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:51:03.405 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T16:51:03.405 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:03.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:03.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:03.521 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T16:51:03.521 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:03.521 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:03.521 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T16:51:03.522 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T16:51:03.522 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T16:51:03.522 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T16:51:03.522 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T16:51:03.522 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:03.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:03.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:03.622 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:03.622 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:03.622 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-04-25T16:51:03.622 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T16:51:03.622 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:03.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:03.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:03.724 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T16:51:03.724 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:51:03.724 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:51:03.772 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T16:51:03.772 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:51:03.772 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:51:03.820 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:51:03.877 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:51:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:51:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:51:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:51:03.925 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:51:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:51:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:51:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:51:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-04-25T16:51:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:51:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:51:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:51:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:51:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:51:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:51:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:51:03.926 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-04-25T16:51:03.927 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:03.927 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:51:03.927 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:03.927 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-04-25T16:51:03.927 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:51:03.927 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:51:03.927 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:51:03.927 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:51:03.927 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:51:03.927 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:51:03.927 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:51:03.927 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:51:03.928 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:51:03.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:03.929 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:51:03.976 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T16:51:03.977 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T16:51:04.021 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:51:04.035 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:04.081 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:51:04.081 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:51:04.082 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:51:04.082 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:51:04.082 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:51:04.082 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:51:04.082 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:51:04.082 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:51:04.082 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-04-25T16:51:04.082 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:51:04.082 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:51:04.082 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:51:04.083 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:04.083 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:51:04.083 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:51:04.083 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:51:04.083 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:51:04.083 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-04-25T16:51:04.083 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:04.083 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:51:04.083 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:04.083 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-04-25T16:51:04.083 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:51:04.083 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:51:04.084 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:51:04.084 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:51:04.084 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:51:04.084 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:51:04.084 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:51:04.084 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:51:04.084 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:51:04.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:04.084 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:51:04.134 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi094.stderr:2024-04-25T16:51:04.136+0000 7ff10312f580 -1 init, newargv = 0x562baf230260 newargc=15 2024-04-25T16:51:04.134 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi094.stderr:ceph-fuse[33681]: starting ceph client 2024-04-25T16:51:04.143 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi094.stderr:ceph-fuse[33681]: starting fuse 2024-04-25T16:51:05.234 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:51:05.250 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:05.301 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:51:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:51:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:51:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:51:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:51:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:51:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:51:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:51:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-04-25T16:51:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:51:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:51:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:51:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:05.302 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:51:05.303 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:51:05.304 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:51:05.304 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:51:05.304 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-25T16:51:05.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:05.305 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:51:05.352 INFO:teuthology.orchestra.run.smithi094.stdout:45 2024-04-25T16:51:05.352 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-04-25T16:51:05.353 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-04-25T16:51:05.354 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:51:05.354 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-04-25T16:51:05.354 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T16:51:05.354 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T16:51:05.354 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T16:51:05.354 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-04-25T16:51:05.354 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-04-25T16:51:05.354 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-04-25T16:51:05.354 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T16:51:05.354 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:51:05.354 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.admin")) 2024-04-25T16:51:05.354 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-04-25T16:51:05.477 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.admin.33681.asok 2024-04-25T16:51:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:05.489 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.33681.asok 2024-04-25T16:51:05.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:05.489 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.33681.asok status 2024-04-25T16:51:05.671 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:51:05.672 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-04-25T16:51:05.672 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T16:51:05.672 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T16:51:05.672 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "admin", 2024-04-25T16:51:05.672 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-04-25T16:51:05.672 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T16:51:05.672 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "33681", 2024-04-25T16:51:05.672 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-04-25T16:51:05.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:51:05.672 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-04-25T16:51:05.672 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4407, 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 4407 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2499996639 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-04-25T16:51:05.673 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-04-25T16:51:05.674 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-04-25T16:51:05.674 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2499996639 2024-04-25T16:51:05.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:51:05.674 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.4407 192.168.0.1:0/2499996639", 2024-04-25T16:51:05.674 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/2499996639", 2024-04-25T16:51:05.674 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-04-25T16:51:05.674 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 7, 2024-04-25T16:51:05.674 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 24, 2024-04-25T16:51:05.674 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-04-25T16:51:05.674 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-04-25T16:51:05.674 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-04-25T16:51:05.674 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:51:05.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:05.688 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-04-25T16:51:05.688 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:51:05.694 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-04-25T16:51:05.694 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:51:05.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:05.694 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:51:05.748 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T16:51:05.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:51:05.874 INFO:teuthology.orchestra.run.smithi094.stdout:# file: . 2024-04-25T16:51:05.875 INFO:teuthology.orchestra.run.smithi094.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T16:51:05.875 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:05.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:05.881 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:51:05.882 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:51:05.930 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:06.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691e2a7640 1 Processor -- start 2024-04-25T16:51:06.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691e2a7640 1 -- start start 2024-04-25T16:51:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691e2a7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6918070080 0x7f691815e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691e2a7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f691815ebd0 0x7f691806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691e2a7640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f691806fc60 con 0x7f691815e7c0 2024-04-25T16:51:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691e2a7640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f6918158550 con 0x7f691815ebd0 2024-04-25T16:51:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691e2a7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f691806f7d0 con 0x7f6918070080 2024-04-25T16:51:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691c844640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f691815ebd0 0x7f691806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691c844640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f691815ebd0 0x7f691806cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49058/0 (socket says 172.21.15.94:49058) 2024-04-25T16:51:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691c844640 1 -- 172.21.15.94:0/2609670234 learned_addr learned my addr 172.21.15.94:0/2609670234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f6917fff640 1 --2- 172.21.15.94:0/2609670234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6918070080 0x7f691815e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691c844640 1 -- 172.21.15.94:0/2609670234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6918070080 msgr2=0x7f691815e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:06.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691c844640 1 --2- 172.21.15.94:0/2609670234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6918070080 0x7f691815e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691c844640 1 -- 172.21.15.94:0/2609670234 >> v1:172.21.15.94:6789/0 conn(0x7f691815e7c0 legacy=0x7f691806c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.132+0000 7f691c844640 1 -- 172.21.15.94:0/2609670234 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69180aef70 con 0x7f691815ebd0 2024-04-25T16:51:06.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.136+0000 7f691c844640 1 --2- 172.21.15.94:0/2609670234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f691815ebd0 0x7f691806cd10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6910009e50 tx=0x7f691001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c94e87f963a7b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:06.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.136+0000 7f6916ffd640 1 -- 172.21.15.94:0/2609670234 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69100a56e0 con 0x7f691815ebd0 2024-04-25T16:51:06.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.136+0000 7f6916ffd640 1 -- 172.21.15.94:0/2609670234 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69100a5880 con 0x7f691815ebd0 2024-04-25T16:51:06.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.136+0000 7f6916ffd640 1 -- 172.21.15.94:0/2609670234 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69100a5c50 con 0x7f691815ebd0 2024-04-25T16:51:06.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.136+0000 7f691e2a7640 1 -- 172.21.15.94:0/2609670234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f691815ebd0 msgr2=0x7f691806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.136+0000 7f691e2a7640 1 --2- 172.21.15.94:0/2609670234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f691815ebd0 0x7f691806cd10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6910009e50 tx=0x7f691001c8c0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.136+0000 7f691e2a7640 1 -- 172.21.15.94:0/2609670234 shutdown_connections 2024-04-25T16:51:06.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.136+0000 7f691e2a7640 1 --2- 172.21.15.94:0/2609670234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f691815ebd0 0x7f691806cd10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.136+0000 7f691e2a7640 1 --2- 172.21.15.94:0/2609670234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6918070080 0x7f691815e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.136+0000 7f691e2a7640 1 -- 172.21.15.94:0/2609670234 >> 172.21.15.94:0/2609670234 conn(0x7f69180b47f0 msgr2=0x7f69180b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:06.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.136+0000 7f691e2a7640 1 -- 172.21.15.94:0/2609670234 shutdown_connections 2024-04-25T16:51:06.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.136+0000 7f691e2a7640 1 -- 172.21.15.94:0/2609670234 wait complete. 2024-04-25T16:51:06.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f691e2a7640 1 Processor -- start 2024-04-25T16:51:06.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f691e2a7640 1 -- start start 2024-04-25T16:51:06.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f691e2a7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6918070080 0x7f691814d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f691e2a7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f691815e7c0 0x7f69181547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f691e2a7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f691815ebd0 0x7f69181525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f691e2a7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f691806f460 con 0x7f6918070080 2024-04-25T16:51:06.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f691e2a7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69180ae3e0 con 0x7f691815e7c0 2024-04-25T16:51:06.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f691e2a7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f69180ae6c0 con 0x7f691815ebd0 2024-04-25T16:51:06.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f6917fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6918070080 0x7f691814d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f6917fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6918070080 0x7f691814d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49074/0 (socket says 172.21.15.94:49074) 2024-04-25T16:51:06.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f6917fff640 1 -- 172.21.15.94:0/3019111289 learned_addr learned my addr 172.21.15.94:0/3019111289 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f691c844640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f691815ebd0 0x7f69181525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f69177fe640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f691815e7c0 0x7f69181547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f6917fff640 1 -- 172.21.15.94:0/3019111289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f691815ebd0 msgr2=0x7f69181525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f6917fff640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f691815ebd0 0x7f69181525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f6917fff640 1 -- 172.21.15.94:0/3019111289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f691815e7c0 msgr2=0x7f69181547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f6917fff640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f691815e7c0 0x7f69181547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f6917fff640 1 -- 172.21.15.94:0/3019111289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691814dc40 con 0x7f6918070080 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f69177fe640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f691815e7c0 0x7f69181547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f6917fff640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6918070080 0x7f691814d700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f690000d600 tx=0x7f690000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f6914ff9640 1 -- 172.21.15.94:0/3019111289 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900004240 con 0x7f6918070080 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f6914ff9640 1 -- 172.21.15.94:0/3019111289 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69000043e0 con 0x7f6918070080 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f691e2a7640 1 -- 172.21.15.94:0/3019111289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69180cf670 con 0x7f6918070080 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f6914ff9640 1 -- 172.21.15.94:0/3019111289 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900003840 con 0x7f6918070080 2024-04-25T16:51:06.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f691e2a7640 1 -- 172.21.15.94:0/3019111289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69180cfc30 con 0x7f6918070080 2024-04-25T16:51:06.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.140+0000 7f691e2a7640 1 -- 172.21.15.94:0/3019111289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6918076830 con 0x7f6918070080 2024-04-25T16:51:06.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.144+0000 7f6914ff9640 1 -- 172.21.15.94:0/3019111289 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f69000039e0 con 0x7f6918070080 2024-04-25T16:51:06.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.144+0000 7f6914ff9640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f68f806f400 0x7f68f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.144+0000 7f6914ff9640 1 -- 172.21.15.94:0/3019111289 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6900090320 con 0x7f6918070080 2024-04-25T16:51:06.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.144+0000 7f69177fe640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f68f806f400 0x7f68f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.144+0000 7f69177fe640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f68f806f400 0x7f68f8071910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6918151e00 tx=0x7f690800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:06.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.148+0000 7f6914ff9640 1 -- 172.21.15.94:0/3019111289 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f690005cfb0 con 0x7f6918070080 2024-04-25T16:51:06.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.304+0000 7f691e2a7640 1 -- 172.21.15.94:0/3019111289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f691815fd70 con 0x7f6918070080 2024-04-25T16:51:06.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.308+0000 7f6914ff9640 1 -- 172.21.15.94:0/3019111289 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f690005c8b0 con 0x7f6918070080 2024-04-25T16:51:06.305 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:06.305 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:59.528743+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:06.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.308+0000 7f691e2a7640 1 -- 172.21.15.94:0/3019111289 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f68f806f400 msgr2=0x7f68f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.308+0000 7f691e2a7640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f68f806f400 0x7f68f8071910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6918151e00 tx=0x7f690800b040 comp rx=0 tx=0).stop 2024-04-25T16:51:06.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.308+0000 7f691e2a7640 1 -- 172.21.15.94:0/3019111289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6918070080 msgr2=0x7f691814d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.308+0000 7f691e2a7640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6918070080 0x7f691814d700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f690000d600 tx=0x7f690000dac0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.308+0000 7f691e2a7640 1 -- 172.21.15.94:0/3019111289 shutdown_connections 2024-04-25T16:51:06.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.308+0000 7f691e2a7640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f68f806f400 0x7f68f8071910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.308+0000 7f691e2a7640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f691815ebd0 0x7f69181525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.308+0000 7f691e2a7640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f691815e7c0 0x7f69181547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.308+0000 7f691e2a7640 1 --2- 172.21.15.94:0/3019111289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6918070080 0x7f691814d700 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.308+0000 7f691e2a7640 1 -- 172.21.15.94:0/3019111289 >> 172.21.15.94:0/3019111289 conn(0x7f69180b47f0 msgr2=0x7f6918156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:06.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.308+0000 7f691e2a7640 1 -- 172.21.15.94:0/3019111289 shutdown_connections 2024-04-25T16:51:06.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.308+0000 7f691e2a7640 1 -- 172.21.15.94:0/3019111289 wait complete. 2024-04-25T16:51:06.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:51:06.326 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:51:06.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.464+0000 7f704e7ca640 1 Processor -- start 2024-04-25T16:51:06.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.464+0000 7f704e7ca640 1 -- start start 2024-04-25T16:51:06.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.464+0000 7f704e7ca640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f70480d3f70 0x7f70480d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.464+0000 7f704e7ca640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70480d1970 0x7f70480d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.464+0000 7f704e7ca640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7048164510 con 0x7f70480d3450 2024-04-25T16:51:06.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.464+0000 7f704e7ca640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7048164a90 con 0x7f70480d3f70 2024-04-25T16:51:06.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.464+0000 7f704e7ca640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f70480d51b0 con 0x7f70480d1970 2024-04-25T16:51:06.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.464+0000 7f70477fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f70480d3f70 0x7f70480d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.464+0000 7f70477fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f70480d3f70 0x7f70480d1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49090/0 (socket says 172.21.15.94:49090) 2024-04-25T16:51:06.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.464+0000 7f70477fe640 1 -- 172.21.15.94:0/1118515204 learned_addr learned my addr 172.21.15.94:0/1118515204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:06.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.464+0000 7f704cd67640 1 --2- 172.21.15.94:0/1118515204 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70480d1970 0x7f70480d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f70477fe640 1 -- 172.21.15.94:0/1118515204 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70480d1970 msgr2=0x7f70480d1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:06.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f70477fe640 1 --2- 172.21.15.94:0/1118515204 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70480d1970 0x7f70480d1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f70477fe640 1 -- 172.21.15.94:0/1118515204 >> v1:172.21.15.138:6789/0 conn(0x7f70480d3450 legacy=0x7f70480d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f70477fe640 1 -- 172.21.15.94:0/1118515204 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70481647b0 con 0x7f70480d3f70 2024-04-25T16:51:06.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f70477fe640 1 --2- 172.21.15.94:0/1118515204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70480d3f70 0x7f70480d1430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7030009870 tx=0x7f703001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e022ca9417bc02d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:06.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f7046ffd640 1 -- 172.21.15.94:0/1118515204 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70300aa600 con 0x7f70480d3f70 2024-04-25T16:51:06.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f7046ffd640 1 -- 172.21.15.94:0/1118515204 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70300af070 con 0x7f70480d3f70 2024-04-25T16:51:06.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f7046ffd640 1 -- 172.21.15.94:0/1118515204 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70300aa9c0 con 0x7f70480d3f70 2024-04-25T16:51:06.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f704e7ca640 1 -- 172.21.15.94:0/1118515204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70480d3f70 msgr2=0x7f70480d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f704e7ca640 1 --2- 172.21.15.94:0/1118515204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70480d3f70 0x7f70480d1430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7030009870 tx=0x7f703001c4a0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f704e7ca640 1 -- 172.21.15.94:0/1118515204 shutdown_connections 2024-04-25T16:51:06.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f704e7ca640 1 --2- 172.21.15.94:0/1118515204 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70480d1970 0x7f70480d1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f704e7ca640 1 --2- 172.21.15.94:0/1118515204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70480d3f70 0x7f70480d1430 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f704e7ca640 1 -- 172.21.15.94:0/1118515204 >> 172.21.15.94:0/1118515204 conn(0x7f704809e8d0 msgr2=0x7f70480b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:06.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f704e7ca640 1 -- 172.21.15.94:0/1118515204 shutdown_connections 2024-04-25T16:51:06.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.468+0000 7f704e7ca640 1 -- 172.21.15.94:0/1118515204 wait complete. 2024-04-25T16:51:06.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f704e7ca640 1 Processor -- start 2024-04-25T16:51:06.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f704e7ca640 1 -- start start 2024-04-25T16:51:06.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f704e7ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70480d1970 0x7f704814f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f704e7ca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70480d3450 0x7f7048161780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f704e7ca640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70480d3f70 0x7f70480cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f704e7ca640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f70480b8cd0 con 0x7f70480d1970 2024-04-25T16:51:06.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f704e7ca640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70480cfc10 con 0x7f70480d3450 2024-04-25T16:51:06.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f704e7ca640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f70480b8e50 con 0x7f70480d3f70 2024-04-25T16:51:06.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f7047fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70480d1970 0x7f704814f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f7047fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70480d1970 0x7f704814f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49102/0 (socket says 172.21.15.94:49102) 2024-04-25T16:51:06.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f7047fff640 1 -- 172.21.15.94:0/583060405 learned_addr learned my addr 172.21.15.94:0/583060405 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:06.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f704cd67640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70480d3f70 0x7f70480cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f70477fe640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70480d3450 0x7f7048161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f7047fff640 1 -- 172.21.15.94:0/583060405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70480d3f70 msgr2=0x7f70480cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f7047fff640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70480d3f70 0x7f70480cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f7047fff640 1 -- 172.21.15.94:0/583060405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70480d3450 msgr2=0x7f7048161780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f7047fff640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70480d3450 0x7f7048161780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.472+0000 7f7047fff640 1 -- 172.21.15.94:0/583060405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704814ff30 con 0x7f70480d1970 2024-04-25T16:51:06.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.476+0000 7f70477fe640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70480d3450 0x7f7048161780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:06.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.476+0000 7f7047fff640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70480d1970 0x7f704814f9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f704000c730 tx=0x7f704000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:06.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.476+0000 7f7044ff9640 1 -- 172.21.15.94:0/583060405 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040016020 con 0x7f70480d1970 2024-04-25T16:51:06.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.476+0000 7f7044ff9640 1 -- 172.21.15.94:0/583060405 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7040006d80 con 0x7f70480d1970 2024-04-25T16:51:06.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.476+0000 7f704e7ca640 1 -- 172.21.15.94:0/583060405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7048161cc0 con 0x7f70480d1970 2024-04-25T16:51:06.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.476+0000 7f7044ff9640 1 -- 172.21.15.94:0/583060405 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040003bc0 con 0x7f70480d1970 2024-04-25T16:51:06.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.476+0000 7f704e7ca640 1 -- 172.21.15.94:0/583060405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70480cae50 con 0x7f70480d1970 2024-04-25T16:51:06.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.476+0000 7f704e7ca640 1 -- 172.21.15.94:0/583060405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7048076830 con 0x7f70480d1970 2024-04-25T16:51:06.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.476+0000 7f7044ff9640 1 -- 172.21.15.94:0/583060405 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f70400050d0 con 0x7f70480d1970 2024-04-25T16:51:06.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.476+0000 7f7044ff9640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f702806f400 0x7f7028071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.476+0000 7f7044ff9640 1 -- 172.21.15.94:0/583060405 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7040090100 con 0x7f70480d1970 2024-04-25T16:51:06.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.476+0000 7f70477fe640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f702806f400 0x7f7028071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.476+0000 7f70477fe640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f702806f400 0x7f7028071910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70300ae000 tx=0x7f7030008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:06.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.480+0000 7f7044ff9640 1 -- 172.21.15.94:0/583060405 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7040020020 con 0x7f70480d1970 2024-04-25T16:51:06.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.628+0000 7f704e7ca640 1 -- 172.21.15.94:0/583060405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f70480d2c40 con 0x7f70480d1970 2024-04-25T16:51:06.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.628+0000 7f7044ff9640 1 -- 172.21.15.94:0/583060405 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f704005ceb0 con 0x7f70480d1970 2024-04-25T16:51:06.627 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:51:06.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.632+0000 7f704e7ca640 1 -- 172.21.15.94:0/583060405 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f702806f400 msgr2=0x7f7028071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.632+0000 7f704e7ca640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f702806f400 0x7f7028071910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70300ae000 tx=0x7f7030008040 comp rx=0 tx=0).stop 2024-04-25T16:51:06.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.632+0000 7f704e7ca640 1 -- 172.21.15.94:0/583060405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70480d1970 msgr2=0x7f704814f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.632+0000 7f704e7ca640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70480d1970 0x7f704814f9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f704000c730 tx=0x7f704000cbf0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.632+0000 7f704e7ca640 1 -- 172.21.15.94:0/583060405 shutdown_connections 2024-04-25T16:51:06.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.632+0000 7f704e7ca640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f702806f400 0x7f7028071910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.632+0000 7f704e7ca640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70480d3f70 0x7f70480cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.632+0000 7f704e7ca640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70480d3450 0x7f7048161780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.632+0000 7f704e7ca640 1 --2- 172.21.15.94:0/583060405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70480d1970 0x7f704814f9f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.632+0000 7f704e7ca640 1 -- 172.21.15.94:0/583060405 >> 172.21.15.94:0/583060405 conn(0x7f704809e8d0 msgr2=0x7f70480b7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:06.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.632+0000 7f704e7ca640 1 -- 172.21.15.94:0/583060405 shutdown_connections 2024-04-25T16:51:06.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.632+0000 7f704e7ca640 1 -- 172.21.15.94:0/583060405 wait complete. 2024-04-25T16:51:06.647 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:51:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d6874c640 1 Processor -- start 2024-04-25T16:51:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d6874c640 1 -- start start 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d6874c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0d600b0420 0x7f0d6006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d6874c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0d6015ec30 0x7f0d6006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d6874c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d60070630 con 0x7f0d6006f2a0 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d6874c640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f0d600aeaa0 con 0x7f0d6015ec30 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d6874c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0d600703e0 con 0x7f0d600b0420 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d66ce9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0d6015ec30 0x7f0d6006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d66ce9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0d6015ec30 0x7f0d6006cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49116/0 (socket says 172.21.15.94:49116) 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d66ce9640 1 -- 172.21.15.94:0/1105543734 learned_addr learned my addr 172.21.15.94:0/1105543734 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d664e8640 1 --2- 172.21.15.94:0/1105543734 >> v2:172.21.15.138:3300/0 conn(0x7f0d600b0420 0x7f0d6006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d66ce9640 1 -- 172.21.15.94:0/1105543734 >> v2:172.21.15.138:3300/0 conn(0x7f0d600b0420 msgr2=0x7f0d6006ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d66ce9640 1 --2- 172.21.15.94:0/1105543734 >> v2:172.21.15.138:3300/0 conn(0x7f0d600b0420 0x7f0d6006ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d66ce9640 1 -- 172.21.15.94:0/1105543734 >> v1:172.21.15.138:6789/0 conn(0x7f0d6006f2a0 legacy=0x7f0d6006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d66ce9640 1 -- 172.21.15.94:0/1105543734 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6006f6b0 con 0x7f0d6015ec30 2024-04-25T16:51:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.788+0000 7f0d664e8640 1 --2- 172.21.15.94:0/1105543734 >> v2:172.21.15.138:3300/0 conn(0x7f0d600b0420 0x7f0d6006ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:06.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d66ce9640 1 --2- 172.21.15.94:0/1105543734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6015ec30 0x7f0d6006cd10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d60071670 tx=0x7f0d5801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d252dc27b29f4225 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:06.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d654e6640 1 -- 172.21.15.94:0/1105543734 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d580a5840 con 0x7f0d6015ec30 2024-04-25T16:51:06.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d654e6640 1 -- 172.21.15.94:0/1105543734 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d580a59e0 con 0x7f0d6015ec30 2024-04-25T16:51:06.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d654e6640 1 -- 172.21.15.94:0/1105543734 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d580a5db0 con 0x7f0d6015ec30 2024-04-25T16:51:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d6874c640 1 -- 172.21.15.94:0/1105543734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6015ec30 msgr2=0x7f0d6006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d6874c640 1 --2- 172.21.15.94:0/1105543734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6015ec30 0x7f0d6006cd10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d60071670 tx=0x7f0d5801c980 comp rx=0 tx=0).stop 2024-04-25T16:51:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d6874c640 1 -- 172.21.15.94:0/1105543734 shutdown_connections 2024-04-25T16:51:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d6874c640 1 --2- 172.21.15.94:0/1105543734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6015ec30 0x7f0d6006cd10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d6874c640 1 --2- 172.21.15.94:0/1105543734 >> v2:172.21.15.138:3300/0 conn(0x7f0d600b0420 0x7f0d6006ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d6874c640 1 -- 172.21.15.94:0/1105543734 >> 172.21.15.94:0/1105543734 conn(0x7f0d600b47f0 msgr2=0x7f0d600b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d6874c640 1 -- 172.21.15.94:0/1105543734 shutdown_connections 2024-04-25T16:51:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d6874c640 1 -- 172.21.15.94:0/1105543734 wait complete. 2024-04-25T16:51:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d6874c640 1 Processor -- start 2024-04-25T16:51:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.792+0000 7f0d6874c640 1 -- start start 2024-04-25T16:51:06.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d6874c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6006f2a0 0x7f0d601588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d6874c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d600b0420 0x7f0d60149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d6874c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d6015ec30 0x7f0d601561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d6874c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d600b0830 con 0x7f0d6006f2a0 2024-04-25T16:51:06.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d6874c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d600aef30 con 0x7f0d600b0420 2024-04-25T16:51:06.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d6874c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0d600b09b0 con 0x7f0d6015ec30 2024-04-25T16:51:06.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d664e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6006f2a0 0x7f0d601588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d664e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6006f2a0 0x7f0d601588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49124/0 (socket says 172.21.15.94:49124) 2024-04-25T16:51:06.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d664e8640 1 -- 172.21.15.94:0/2234527296 learned_addr learned my addr 172.21.15.94:0/2234527296 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d664e8640 1 -- 172.21.15.94:0/2234527296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d6015ec30 msgr2=0x7f0d601561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d664e8640 1 --2- 172.21.15.94:0/2234527296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d6015ec30 0x7f0d601561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d65ce7640 1 --2- 172.21.15.94:0/2234527296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d600b0420 0x7f0d60149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d664e8640 1 -- 172.21.15.94:0/2234527296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d600b0420 msgr2=0x7f0d60149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d664e8640 1 --2- 172.21.15.94:0/2234527296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d600b0420 0x7f0d60149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d664e8640 1 -- 172.21.15.94:0/2234527296 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d60149b50 con 0x7f0d6006f2a0 2024-04-25T16:51:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d664e8640 1 --2- 172.21.15.94:0/2234527296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6006f2a0 0x7f0d601588d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00c7f0 tx=0x7f0d5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d56ffd640 1 -- 172.21.15.94:0/2234527296 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c015070 con 0x7f0d6006f2a0 2024-04-25T16:51:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d56ffd640 1 -- 172.21.15.94:0/2234527296 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5c007820 con 0x7f0d6006f2a0 2024-04-25T16:51:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d6874c640 1 -- 172.21.15.94:0/2234527296 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d600cfef0 con 0x7f0d6006f2a0 2024-04-25T16:51:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d56ffd640 1 -- 172.21.15.94:0/2234527296 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c007bd0 con 0x7f0d6006f2a0 2024-04-25T16:51:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d6874c640 1 -- 172.21.15.94:0/2234527296 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d60158f40 con 0x7f0d6006f2a0 2024-04-25T16:51:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d6874c640 1 -- 172.21.15.94:0/2234527296 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d601591a0 con 0x7f0d6006f2a0 2024-04-25T16:51:06.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d56ffd640 1 -- 172.21.15.94:0/2234527296 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0d5c018070 con 0x7f0d6006f2a0 2024-04-25T16:51:06.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d56ffd640 1 --2- 172.21.15.94:0/2234527296 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0d4406f400 0x7f0d44071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:06.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d56ffd640 1 -- 172.21.15.94:0/2234527296 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0d5c08f680 con 0x7f0d6006f2a0 2024-04-25T16:51:06.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.796+0000 7f0d65ce7640 1 --2- 172.21.15.94:0/2234527296 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0d4406f400 0x7f0d44071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:06.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.800+0000 7f0d65ce7640 1 --2- 172.21.15.94:0/2234527296 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0d4406f400 0x7f0d44071910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0d6015c010 tx=0x7f0d50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.804+0000 7f0d56ffd640 1 -- 172.21.15.94:0/2234527296 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0d5c05c350 con 0x7f0d6006f2a0 2024-04-25T16:51:06.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.948+0000 7f0d6874c640 1 -- 172.21.15.94:0/2234527296 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0d60076830 con 0x7f0d6006f2a0 2024-04-25T16:51:06.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.952+0000 7f0d56ffd640 1 -- 172.21.15.94:0/2234527296 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f0d5c05bc50 con 0x7f0d6006f2a0 2024-04-25T16:51:06.948 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.952+0000 7f0d6874c640 1 -- 172.21.15.94:0/2234527296 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0d4406f400 msgr2=0x7f0d44071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.952+0000 7f0d6874c640 1 --2- 172.21.15.94:0/2234527296 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0d4406f400 0x7f0d44071910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0d6015c010 tx=0x7f0d50008040 comp rx=0 tx=0).stop 2024-04-25T16:51:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.952+0000 7f0d6874c640 1 -- 172.21.15.94:0/2234527296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6006f2a0 msgr2=0x7f0d601588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.952+0000 7f0d6874c640 1 --2- 172.21.15.94:0/2234527296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6006f2a0 0x7f0d601588d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00c7f0 tx=0x7f0d5c00ccb0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.952+0000 7f0d6874c640 1 -- 172.21.15.94:0/2234527296 shutdown_connections 2024-04-25T16:51:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.952+0000 7f0d6874c640 1 --2- 172.21.15.94:0/2234527296 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0d4406f400 0x7f0d44071910 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.952+0000 7f0d6874c640 1 --2- 172.21.15.94:0/2234527296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d6015ec30 0x7f0d601561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.952+0000 7f0d6874c640 1 --2- 172.21.15.94:0/2234527296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d600b0420 0x7f0d60149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.952+0000 7f0d6874c640 1 --2- 172.21.15.94:0/2234527296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6006f2a0 0x7f0d601588d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:06.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.952+0000 7f0d6874c640 1 -- 172.21.15.94:0/2234527296 >> 172.21.15.94:0/2234527296 conn(0x7f0d600b47f0 msgr2=0x7f0d600b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:06.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.952+0000 7f0d6874c640 1 -- 172.21.15.94:0/2234527296 shutdown_connections 2024-04-25T16:51:06.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:06.952+0000 7f0d6874c640 1 -- 172.21.15.94:0/2234527296 wait complete. 2024-04-25T16:51:06.968 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:51:07.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.116+0000 7fce3383a640 1 Processor -- start 2024-04-25T16:51:07.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.116+0000 7fce3383a640 1 -- start start 2024-04-25T16:51:07.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.116+0000 7fce3383a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c160d00 0x7fce2c0b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.116+0000 7fce3383a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fce2c06dea0 0x7fce2c06e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.116+0000 7fce3383a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce2c06e7f0 con 0x7fce2c1636f0 2024-04-25T16:51:07.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.116+0000 7fce3383a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fce2c1604d0 con 0x7fce2c06dea0 2024-04-25T16:51:07.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.116+0000 7fce3383a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fce2c161110 con 0x7fce2c160d00 2024-04-25T16:51:07.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.116+0000 7fce30dd5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fce2c06dea0 0x7fce2c06e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.120+0000 7fce315d6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c160d00 0x7fce2c0b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.120+0000 7fce31dd7640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fce2c1636f0 0x7fce2c163b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:36260/0 (socket says 172.21.15.94:36260) 2024-04-25T16:51:07.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.120+0000 7fce31dd7640 1 -- 172.21.15.94:0/2329317248 learned_addr learned my addr 172.21.15.94:0/2329317248 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:07.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.120+0000 7fce2bfff640 1 -- 172.21.15.94:0/2329317248 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1995551181 0 0) 0x7fce1c001120 con 0x7fce2c1636f0 2024-04-25T16:51:07.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.120+0000 7fce2bfff640 1 -- 172.21.15.94:0/2329317248 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce14003530 con 0x7fce2c1636f0 2024-04-25T16:51:07.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.120+0000 7fce315d6640 1 -- 172.21.15.94:0/2329317248 >> v2:172.21.15.138:3300/0 conn(0x7fce2c06dea0 msgr2=0x7fce2c06e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:07.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.120+0000 7fce315d6640 1 --2- 172.21.15.94:0/2329317248 >> v2:172.21.15.138:3300/0 conn(0x7fce2c06dea0 0x7fce2c06e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.120+0000 7fce315d6640 1 -- 172.21.15.94:0/2329317248 >> v1:172.21.15.138:6789/0 conn(0x7fce2c1636f0 legacy=0x7fce2c163b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.120+0000 7fce315d6640 1 -- 172.21.15.94:0/2329317248 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c070040 con 0x7fce2c160d00 2024-04-25T16:51:07.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.120+0000 7fce315d6640 1 --2- 172.21.15.94:0/2329317248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c160d00 0x7fce2c0b4a50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fce24009d90 tx=0x7fce2401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb5414082c91f7e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:07.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.120+0000 7fce2bfff640 1 -- 172.21.15.94:0/2329317248 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce240b2070 con 0x7fce2c160d00 2024-04-25T16:51:07.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.120+0000 7fce2bfff640 1 -- 172.21.15.94:0/2329317248 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce240a58b0 con 0x7fce2c160d00 2024-04-25T16:51:07.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.120+0000 7fce2bfff640 1 -- 172.21.15.94:0/2329317248 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce240a5c60 con 0x7fce2c160d00 2024-04-25T16:51:07.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 -- 172.21.15.94:0/2329317248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c160d00 msgr2=0x7fce2c0b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 --2- 172.21.15.94:0/2329317248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c160d00 0x7fce2c0b4a50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fce24009d90 tx=0x7fce2401c8e0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 -- 172.21.15.94:0/2329317248 shutdown_connections 2024-04-25T16:51:07.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 --2- 172.21.15.94:0/2329317248 >> v2:172.21.15.138:3300/0 conn(0x7fce2c06dea0 0x7fce2c06e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 --2- 172.21.15.94:0/2329317248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c160d00 0x7fce2c0b4a50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 -- 172.21.15.94:0/2329317248 >> 172.21.15.94:0/2329317248 conn(0x7fce2c072fd0 msgr2=0x7fce2c0733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:07.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 -- 172.21.15.94:0/2329317248 shutdown_connections 2024-04-25T16:51:07.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 -- 172.21.15.94:0/2329317248 wait complete. 2024-04-25T16:51:07.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 Processor -- start 2024-04-25T16:51:07.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 -- start start 2024-04-25T16:51:07.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce2c06dea0 0x7fce2c0cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce2c160d00 0x7fce2c0d0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c1636f0 0x7fce2c0d0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c0b6c80 con 0x7fce2c160d00 2024-04-25T16:51:07.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c0b59f0 con 0x7fce2c06dea0 2024-04-25T16:51:07.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce3383a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fce2c0b61f0 con 0x7fce2c1636f0 2024-04-25T16:51:07.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.124+0000 7fce30dd5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce2c160d00 0x7fce2c0d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce30dd5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce2c160d00 0x7fce2c0d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49128/0 (socket says 172.21.15.94:49128) 2024-04-25T16:51:07.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce30dd5640 1 -- 172.21.15.94:0/243599590 learned_addr learned my addr 172.21.15.94:0/243599590 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:07.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce31dd7640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c1636f0 0x7fce2c0d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce315d6640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce2c06dea0 0x7fce2c0cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce30dd5640 1 -- 172.21.15.94:0/243599590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c1636f0 msgr2=0x7fce2c0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce30dd5640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c1636f0 0x7fce2c0d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce30dd5640 1 -- 172.21.15.94:0/243599590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce2c06dea0 msgr2=0x7fce2c0cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce30dd5640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce2c06dea0 0x7fce2c0cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce30dd5640 1 -- 172.21.15.94:0/243599590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c0aecd0 con 0x7fce2c160d00 2024-04-25T16:51:07.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce30dd5640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce2c160d00 0x7fce2c0d0400 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fce2000e870 tx=0x7fce2000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:07.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce29ffb640 1 -- 172.21.15.94:0/243599590 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20017070 con 0x7fce2c160d00 2024-04-25T16:51:07.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce29ffb640 1 -- 172.21.15.94:0/243599590 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2000c940 con 0x7fce2c160d00 2024-04-25T16:51:07.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce3383a640 1 -- 172.21.15.94:0/243599590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2c0cf760 con 0x7fce2c160d00 2024-04-25T16:51:07.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce29ffb640 1 -- 172.21.15.94:0/243599590 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2000ccf0 con 0x7fce2c160d00 2024-04-25T16:51:07.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce3383a640 1 -- 172.21.15.94:0/243599590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2c0cfa40 con 0x7fce2c160d00 2024-04-25T16:51:07.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce3383a640 1 -- 172.21.15.94:0/243599590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2c0cf4d0 con 0x7fce2c160d00 2024-04-25T16:51:07.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce29ffb640 1 -- 172.21.15.94:0/243599590 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fce2001a070 con 0x7fce2c160d00 2024-04-25T16:51:07.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce29ffb640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fce1806f430 0x7fce18071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce315d6640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fce1806f430 0x7fce18071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce29ffb640 1 -- 172.21.15.94:0/243599590 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fce2008f470 con 0x7fce2c160d00 2024-04-25T16:51:07.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.128+0000 7fce315d6640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fce1806f430 0x7fce18071940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce2c06cc10 tx=0x7fce240a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:07.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.132+0000 7fce29ffb640 1 -- 172.21.15.94:0/243599590 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fce2005c0b0 con 0x7fce2c160d00 2024-04-25T16:51:07.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce3383a640 1 -- 172.21.15.94:0/243599590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce2c06a730 con 0x7fce2c160d00 2024-04-25T16:51:07.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce29ffb640 1 -- 172.21.15.94:0/243599590 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fce2005b9b0 con 0x7fce2c160d00 2024-04-25T16:51:07.297 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:51:07.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce3383a640 1 -- 172.21.15.94:0/243599590 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fce1806f430 msgr2=0x7fce18071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce3383a640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fce1806f430 0x7fce18071940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce2c06cc10 tx=0x7fce240a7070 comp rx=0 tx=0).stop 2024-04-25T16:51:07.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce3383a640 1 -- 172.21.15.94:0/243599590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce2c160d00 msgr2=0x7fce2c0d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce3383a640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce2c160d00 0x7fce2c0d0400 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fce2000e870 tx=0x7fce2000ec70 comp rx=0 tx=0).stop 2024-04-25T16:51:07.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce3383a640 1 -- 172.21.15.94:0/243599590 shutdown_connections 2024-04-25T16:51:07.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce3383a640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fce1806f430 0x7fce18071940 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce3383a640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c1636f0 0x7fce2c0d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce3383a640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce2c160d00 0x7fce2c0d0400 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce3383a640 1 --2- 172.21.15.94:0/243599590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce2c06dea0 0x7fce2c0cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce3383a640 1 -- 172.21.15.94:0/243599590 >> 172.21.15.94:0/243599590 conn(0x7fce2c072fd0 msgr2=0x7fce2c0b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:07.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce3383a640 1 -- 172.21.15.94:0/243599590 shutdown_connections 2024-04-25T16:51:07.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.300+0000 7fce3383a640 1 -- 172.21.15.94:0/243599590 wait complete. 2024-04-25T16:51:07.317 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:51:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.472+0000 7f337dcd4640 1 Processor -- start 2024-04-25T16:51:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.472+0000 7f337dcd4640 1 -- start start 2024-04-25T16:51:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.472+0000 7f337dcd4640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f33780d05a0 0x7f3378073a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.472+0000 7f337dcd4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3378073f50 0x7f33780b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.472+0000 7f337dcd4640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3378074360 con 0x7f33780d0190 2024-04-25T16:51:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.472+0000 7f337dcd4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f337815b920 con 0x7f3378073f50 2024-04-25T16:51:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.472+0000 7f337dcd4640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f337815e420 con 0x7f33780d05a0 2024-04-25T16:51:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.472+0000 7f3377fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3378073f50 0x7f33780b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.472+0000 7f3377fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3378073f50 0x7f33780b56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49144/0 (socket says 172.21.15.94:49144) 2024-04-25T16:51:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.472+0000 7f3377fff640 1 -- 172.21.15.94:0/1448995884 learned_addr learned my addr 172.21.15.94:0/1448995884 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.472+0000 7f3376ffd640 1 --2- 172.21.15.94:0/1448995884 >> v2:172.21.15.138:3300/0 conn(0x7f33780d05a0 0x7f3378073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f3377fff640 1 -- 172.21.15.94:0/1448995884 >> v2:172.21.15.138:3300/0 conn(0x7f33780d05a0 msgr2=0x7f3378073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f3377fff640 1 --2- 172.21.15.94:0/1448995884 >> v2:172.21.15.138:3300/0 conn(0x7f33780d05a0 0x7f3378073a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f3377fff640 1 -- 172.21.15.94:0/1448995884 >> v1:172.21.15.138:6789/0 conn(0x7f33780d0190 legacy=0x7f33780b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f3377fff640 1 -- 172.21.15.94:0/1448995884 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337815e700 con 0x7f3378073f50 2024-04-25T16:51:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f3377fff640 1 --2- 172.21.15.94:0/1448995884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3378073f50 0x7f33780b56b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f33780d09b0 tx=0x7f337001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4374adc635d673 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f33767fc640 1 -- 172.21.15.94:0/1448995884 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33700a6500 con 0x7f3378073f50 2024-04-25T16:51:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f33767fc640 1 -- 172.21.15.94:0/1448995884 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33700a66a0 con 0x7f3378073f50 2024-04-25T16:51:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f33767fc640 1 -- 172.21.15.94:0/1448995884 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33700a6a70 con 0x7f3378073f50 2024-04-25T16:51:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f337dcd4640 1 -- 172.21.15.94:0/1448995884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3378073f50 msgr2=0x7f33780b56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f337dcd4640 1 --2- 172.21.15.94:0/1448995884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3378073f50 0x7f33780b56b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f33780d09b0 tx=0x7f337001c740 comp rx=0 tx=0).stop 2024-04-25T16:51:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f337dcd4640 1 -- 172.21.15.94:0/1448995884 shutdown_connections 2024-04-25T16:51:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f337dcd4640 1 --2- 172.21.15.94:0/1448995884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3378073f50 0x7f33780b56b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f337dcd4640 1 --2- 172.21.15.94:0/1448995884 >> v2:172.21.15.138:3300/0 conn(0x7f33780d05a0 0x7f3378073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f337dcd4640 1 -- 172.21.15.94:0/1448995884 >> 172.21.15.94:0/1448995884 conn(0x7f33780d2690 msgr2=0x7f33780d2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f337dcd4640 1 -- 172.21.15.94:0/1448995884 shutdown_connections 2024-04-25T16:51:07.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.476+0000 7f337dcd4640 1 -- 172.21.15.94:0/1448995884 wait complete. 2024-04-25T16:51:07.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f337dcd4640 1 Processor -- start 2024-04-25T16:51:07.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f337dcd4640 1 -- start start 2024-04-25T16:51:07.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f337dcd4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3378073f50 0x7f3378154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f337dcd4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33780d0190 0x7f3378154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f337dcd4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33780d05a0 0x7f337806e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f337dcd4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33780703a0 con 0x7f33780d05a0 2024-04-25T16:51:07.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f337dcd4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33780b6000 con 0x7f33780d0190 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f337dcd4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3378070520 con 0x7f3378073f50 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f3377fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33780d05a0 0x7f337806e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f3377fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33780d05a0 0x7f337806e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49156/0 (socket says 172.21.15.94:49156) 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f3377fff640 1 -- 172.21.15.94:0/596756400 learned_addr learned my addr 172.21.15.94:0/596756400 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f3376ffd640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33780d0190 0x7f3378154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f3377fff640 1 -- 172.21.15.94:0/596756400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3378073f50 msgr2=0x7f3378154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f33777fe640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3378073f50 0x7f3378154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f3377fff640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3378073f50 0x7f3378154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f3377fff640 1 -- 172.21.15.94:0/596756400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33780d0190 msgr2=0x7f3378154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f3377fff640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33780d0190 0x7f3378154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f3377fff640 1 -- 172.21.15.94:0/596756400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33781515d0 con 0x7f33780d05a0 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f3376ffd640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33780d0190 0x7f3378154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f3377fff640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33780d05a0 0x7f337806e9b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f337001a370 tx=0x7f33700194e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f335ffff640 1 -- 172.21.15.94:0/596756400 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33700a6c60 con 0x7f33780d05a0 2024-04-25T16:51:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f335ffff640 1 -- 172.21.15.94:0/596756400 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33700a6e20 con 0x7f33780d05a0 2024-04-25T16:51:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f337dcd4640 1 -- 172.21.15.94:0/596756400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33780cbc60 con 0x7f33780d05a0 2024-04-25T16:51:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f335ffff640 1 -- 172.21.15.94:0/596756400 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33700af6c0 con 0x7f33780d05a0 2024-04-25T16:51:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.480+0000 7f337dcd4640 1 -- 172.21.15.94:0/596756400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3378155b10 con 0x7f33780d05a0 2024-04-25T16:51:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.484+0000 7f337dcd4640 1 -- 172.21.15.94:0/596756400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f337806a730 con 0x7f33780d05a0 2024-04-25T16:51:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.484+0000 7f335ffff640 1 -- 172.21.15.94:0/596756400 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f33700af860 con 0x7f33780d05a0 2024-04-25T16:51:07.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.484+0000 7f335ffff640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f336406f400 0x7f3364071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.484+0000 7f335ffff640 1 -- 172.21.15.94:0/596756400 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3370129b80 con 0x7f33780d05a0 2024-04-25T16:51:07.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.484+0000 7f33777fe640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f336406f400 0x7f3364071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.484+0000 7f33777fe640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f336406f400 0x7f3364071910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f336c00a750 tx=0x7f336c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:07.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.488+0000 7f335ffff640 1 -- 172.21.15.94:0/596756400 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3378074360 con 0x7f33780d05a0 2024-04-25T16:51:07.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.648+0000 7f337dcd4640 1 -- 172.21.15.94:0/596756400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f33780b1cb0 con 0x7f33780d05a0 2024-04-25T16:51:07.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.648+0000 7f335ffff640 1 -- 172.21.15.94:0/596756400 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f33700f6850 con 0x7f33780d05a0 2024-04-25T16:51:07.647 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:07.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.652+0000 7f337dcd4640 1 -- 172.21.15.94:0/596756400 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f336406f400 msgr2=0x7f3364071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.652+0000 7f337dcd4640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f336406f400 0x7f3364071910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f336c00a750 tx=0x7f336c008040 comp rx=0 tx=0).stop 2024-04-25T16:51:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.652+0000 7f337dcd4640 1 -- 172.21.15.94:0/596756400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33780d05a0 msgr2=0x7f337806e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.652+0000 7f337dcd4640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33780d05a0 0x7f337806e9b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f337001a370 tx=0x7f33700194e0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.652+0000 7f337dcd4640 1 -- 172.21.15.94:0/596756400 shutdown_connections 2024-04-25T16:51:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.652+0000 7f337dcd4640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33780d05a0 0x7f337806e9b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.652+0000 7f337dcd4640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33780d0190 0x7f3378154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.652+0000 7f337dcd4640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f336406f400 0x7f3364071910 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.652+0000 7f337dcd4640 1 --2- 172.21.15.94:0/596756400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3378073f50 0x7f3378154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.652+0000 7f337dcd4640 1 -- 172.21.15.94:0/596756400 >> 172.21.15.94:0/596756400 conn(0x7f33780d2690 msgr2=0x7f3378158760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.652+0000 7f337dcd4640 1 -- 172.21.15.94:0/596756400 shutdown_connections 2024-04-25T16:51:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.652+0000 7f337dcd4640 1 -- 172.21.15.94:0/596756400 wait complete. 2024-04-25T16:51:07.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:51:07.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.812+0000 7fe57e233640 1 Processor -- start 2024-04-25T16:51:07.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.812+0000 7fe57e233640 1 -- start start 2024-04-25T16:51:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe57e233640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe57806d710 0x7fe57806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe57e233640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe578074380 con 0x7fe57815c7e0 2024-04-25T16:51:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe57e233640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe57806e060 con 0x7fe57806e7a0 2024-04-25T16:51:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe57e233640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe5780727e0 con 0x7fe57806d710 2024-04-25T16:51:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5777fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fe57815c7e0 0x7fe5780afdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50274/0 (socket says 172.21.15.94:50274) 2024-04-25T16:51:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5777fe640 1 -- 172.21.15.94:0/371247517 learned_addr learned my addr 172.21.15.94:0/371247517 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:07.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe576ffd640 1 --2- 172.21.15.94:0/371247517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe57806d710 0x7fe57806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2285166038 0 0) 0x7fe568001120 con 0x7fe57815c7e0 2024-04-25T16:51:07.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe560005740 con 0x7fe57815c7e0 2024-04-25T16:51:07.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1097921800 0 0) 0x7fe560005740 con 0x7fe57815c7e0 2024-04-25T16:51:07.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe57806d710 msgr2=0x7fe57806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:07.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 --2- 172.21.15.94:0/371247517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe57806d710 0x7fe57806db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 >> v1:172.21.15.138:6789/0 conn(0x7fe57806e7a0 legacy=0x7fe57806ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe578072ac0 con 0x7fe57815c7e0 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fe568001850 con 0x7fe57815c7e0 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 >> v1:172.21.15.94:6789/0 conn(0x7fe57815c7e0 legacy=0x7fe5780afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 --2- 172.21.15.94:0/371247517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe560005c00 0x7fe560006070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 --2- 172.21.15.94:0/371247517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe57806d710 0x7fe560006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 --2- 172.21.15.94:0/371247517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe560006f60 0x7fe560007370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5780727e0 con 0x7fe560005c00 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe56000dda0 con 0x7fe560006f60 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe56000dfa0 con 0x7fe57806d710 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5777fe640 1 --2- 172.21.15.94:0/371247517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe560005c00 0x7fe560006070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe576ffd640 1 --2- 172.21.15.94:0/371247517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe57806d710 0x7fe560006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe577fff640 1 --2- 172.21.15.94:0/371247517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe560006f60 0x7fe560007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5777fe640 1 -- 172.21.15.94:0/371247517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe57806d710 msgr2=0x7fe560006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:07.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5777fe640 1 --2- 172.21.15.94:0/371247517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe57806d710 0x7fe560006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5777fe640 1 -- 172.21.15.94:0/371247517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe560006f60 msgr2=0x7fe560007370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:07.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5777fe640 1 --2- 172.21.15.94:0/371247517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe560006f60 0x7fe560007370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.816+0000 7fe5777fe640 1 -- 172.21.15.94:0/371247517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe560005740 con 0x7fe560005c00 2024-04-25T16:51:07.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.824+0000 7fe5777fe640 1 --2- 172.21.15.94:0/371247517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe560005c00 0x7fe560006070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe56800c4c0 tx=0x7fe56801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02127dbe0fddb73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:07.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.824+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56800c0e0 con 0x7fe560005c00 2024-04-25T16:51:07.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.824+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5680a9070 con 0x7fe560005c00 2024-04-25T16:51:07.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.824+0000 7fe5767fc640 1 -- 172.21.15.94:0/371247517 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568005d40 con 0x7fe560005c00 2024-04-25T16:51:07.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.824+0000 7fe57e233640 1 -- 172.21.15.94:0/371247517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe560005c00 msgr2=0x7fe560006070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.824+0000 7fe57e233640 1 --2- 172.21.15.94:0/371247517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe560005c00 0x7fe560006070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe56800c4c0 tx=0x7fe56801cc50 comp rx=0 tx=0).stop 2024-04-25T16:51:07.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.824+0000 7fe577fff640 1 -- 172.21.15.94:0/371247517 reap_dead start 2024-04-25T16:51:07.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.824+0000 7fe57e233640 1 -- 172.21.15.94:0/371247517 shutdown_connections 2024-04-25T16:51:07.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.824+0000 7fe57e233640 1 -- 172.21.15.94:0/371247517 >> 172.21.15.94:0/371247517 conn(0x7fe5780b47f0 msgr2=0x7fe5780b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:07.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.824+0000 7fe57e233640 1 -- 172.21.15.94:0/371247517 shutdown_connections 2024-04-25T16:51:07.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.824+0000 7fe57e233640 1 -- 172.21.15.94:0/371247517 wait complete. 2024-04-25T16:51:07.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.824+0000 7fe57e233640 1 Processor -- start 2024-04-25T16:51:07.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.824+0000 7fe57e233640 1 -- start start 2024-04-25T16:51:07.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe57e233640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5780b3470 0x7fe578145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe57e233640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe57815e5f0 0x7fe57815ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe57e233640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe578146ab0 0x7fe578146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe57e233640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe57806f2c0 con 0x7fe5780b3470 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe57e233640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe578073ca0 con 0x7fe578146ab0 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe57e233640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe57815cbf0 con 0x7fe57815e5f0 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe5777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5780b3470 0x7fe578145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe5777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5780b3470 0x7fe578145470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49182/0 (socket says 172.21.15.94:49182) 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe5777fe640 1 -- 172.21.15.94:0/2021395769 learned_addr learned my addr 172.21.15.94:0/2021395769 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe5777fe640 1 -- 172.21.15.94:0/2021395769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe57815e5f0 msgr2=0x7fe57815ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe5777fe640 1 --2- 172.21.15.94:0/2021395769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe57815e5f0 0x7fe57815ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe5777fe640 1 -- 172.21.15.94:0/2021395769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe578146ab0 msgr2=0x7fe578146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe5777fe640 1 --2- 172.21.15.94:0/2021395769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe578146ab0 0x7fe578146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe5777fe640 1 -- 172.21.15.94:0/2021395769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57815d160 con 0x7fe5780b3470 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe5777fe640 1 --2- 172.21.15.94:0/2021395769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5780b3470 0x7fe578145470 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe568001b50 tx=0x7fe560007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe55ffff640 1 -- 172.21.15.94:0/2021395769 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568001120 con 0x7fe5780b3470 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe55ffff640 1 -- 172.21.15.94:0/2021395769 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5680048b0 con 0x7fe5780b3470 2024-04-25T16:51:07.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe55ffff640 1 -- 172.21.15.94:0/2021395769 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568004c60 con 0x7fe5780b3470 2024-04-25T16:51:07.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe57e233640 1 -- 172.21.15.94:0/2021395769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5780b30b0 con 0x7fe5780b3470 2024-04-25T16:51:07.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe57e233640 1 -- 172.21.15.94:0/2021395769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5780b2dd0 con 0x7fe5780b3470 2024-04-25T16:51:07.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe57e233640 1 -- 172.21.15.94:0/2021395769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe578076830 con 0x7fe5780b3470 2024-04-25T16:51:07.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe55ffff640 1 -- 172.21.15.94:0/2021395769 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe568007180 con 0x7fe5780b3470 2024-04-25T16:51:07.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe55ffff640 1 --2- 172.21.15.94:0/2021395769 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe56406f400 0x7fe564071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:07.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe55ffff640 1 -- 172.21.15.94:0/2021395769 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe568128b00 con 0x7fe5780b3470 2024-04-25T16:51:07.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe576ffd640 1 --2- 172.21.15.94:0/2021395769 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe56406f400 0x7fe564071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:07.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.828+0000 7fe576ffd640 1 --2- 172.21.15.94:0/2021395769 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe56406f400 0x7fe564071910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009160 tx=0x7fe56c006e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:07.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.832+0000 7fe55ffff640 1 -- 172.21.15.94:0/2021395769 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe578074380 con 0x7fe5780b3470 2024-04-25T16:51:07.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.992+0000 7fe57e233640 1 -- 172.21.15.94:0/2021395769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe5780afdb0 con 0x7fe5780b3470 2024-04-25T16:51:07.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.992+0000 7fe55ffff640 1 -- 172.21.15.94:0/2021395769 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fe5680f57d0 con 0x7fe5780b3470 2024-04-25T16:51:07.992 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:51:07.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.996+0000 7fe57e233640 1 -- 172.21.15.94:0/2021395769 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe56406f400 msgr2=0x7fe564071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.996+0000 7fe57e233640 1 --2- 172.21.15.94:0/2021395769 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe56406f400 0x7fe564071910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009160 tx=0x7fe56c006e00 comp rx=0 tx=0).stop 2024-04-25T16:51:07.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.996+0000 7fe57e233640 1 -- 172.21.15.94:0/2021395769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5780b3470 msgr2=0x7fe578145470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:07.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.996+0000 7fe57e233640 1 --2- 172.21.15.94:0/2021395769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5780b3470 0x7fe578145470 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe568001b50 tx=0x7fe560007a50 comp rx=0 tx=0).stop 2024-04-25T16:51:07.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.996+0000 7fe57e233640 1 -- 172.21.15.94:0/2021395769 shutdown_connections 2024-04-25T16:51:07.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.996+0000 7fe57e233640 1 --2- 172.21.15.94:0/2021395769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe578146ab0 0x7fe578146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.996+0000 7fe57e233640 1 --2- 172.21.15.94:0/2021395769 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe56406f400 0x7fe564071910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.996+0000 7fe57e233640 1 --2- 172.21.15.94:0/2021395769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe57815e5f0 0x7fe57815ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.996+0000 7fe57e233640 1 --2- 172.21.15.94:0/2021395769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5780b3470 0x7fe578145470 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:07.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.996+0000 7fe57e233640 1 -- 172.21.15.94:0/2021395769 >> 172.21.15.94:0/2021395769 conn(0x7fe5780b47f0 msgr2=0x7fe578149ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:07.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.996+0000 7fe57e233640 1 -- 172.21.15.94:0/2021395769 shutdown_connections 2024-04-25T16:51:07.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:07.996+0000 7fe57e233640 1 -- 172.21.15.94:0/2021395769 wait complete. 2024-04-25T16:51:08.011 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:51:08.011 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:51:08.017 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.208+0000 7f7d3bcc8640 1 Processor -- start 2024-04-25T16:51:08.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.208+0000 7f7d3bcc8640 1 -- start start 2024-04-25T16:51:08.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d3bcc8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7d34070080 0x7f7d3415e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d3bcc8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7d3415e7c0 0x7f7d3406c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d3bcc8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d3415ebd0 0x7f7d3406cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d3bcc8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7d34158550 con 0x7f7d3415e7c0 2024-04-25T16:51:08.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d3bcc8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7d3406f7d0 con 0x7f7d34070080 2024-04-25T16:51:08.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d3bcc8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3406fc60 con 0x7f7d3415ebd0 2024-04-25T16:51:08.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d39263640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7d3415e7c0 0x7f7d3406c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d39263640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7d3415e7c0 0x7f7d3406c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35942/0 (socket says 172.21.15.94:35942) 2024-04-25T16:51:08.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d39263640 1 -- 172.21.15.94:0/1231574656 learned_addr learned my addr 172.21.15.94:0/1231574656 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:08.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d39a64640 1 --2- 172.21.15.94:0/1231574656 >> v2:172.21.15.138:3300/0 conn(0x7f7d34070080 0x7f7d3415e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d3a265640 1 --2- 172.21.15.94:0/1231574656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d3415ebd0 0x7f7d3406cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d39263640 1 -- 172.21.15.94:0/1231574656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d3415ebd0 msgr2=0x7f7d3406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:08.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d39263640 1 --2- 172.21.15.94:0/1231574656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d3415ebd0 0x7f7d3406cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d39263640 1 -- 172.21.15.94:0/1231574656 >> v2:172.21.15.138:3300/0 conn(0x7f7d34070080 msgr2=0x7f7d3415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:08.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d39263640 1 --2- 172.21.15.94:0/1231574656 >> v2:172.21.15.138:3300/0 conn(0x7f7d34070080 0x7f7d3415e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.212+0000 7f7d39263640 1 -- 172.21.15.94:0/1231574656 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d340aef70 con 0x7f7d3415e7c0 2024-04-25T16:51:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d39263640 1 --2- 172.21.15.94:0/1231574656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3415e7c0 0x7f7d3406c600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7d3406f950 tx=0x7f7d2401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1fe1ac71b3e675 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d38a62640 1 -- 172.21.15.94:0/1231574656 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d240aa610 con 0x7f7d3415e7c0 2024-04-25T16:51:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d38a62640 1 -- 172.21.15.94:0/1231574656 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d240af070 con 0x7f7d3415e7c0 2024-04-25T16:51:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d38a62640 1 -- 172.21.15.94:0/1231574656 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d240aa9d0 con 0x7f7d3415e7c0 2024-04-25T16:51:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/1231574656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3415e7c0 msgr2=0x7f7d3406c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d3bcc8640 1 --2- 172.21.15.94:0/1231574656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3415e7c0 0x7f7d3406c600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7d3406f950 tx=0x7f7d2401c4f0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/1231574656 shutdown_connections 2024-04-25T16:51:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d3bcc8640 1 --2- 172.21.15.94:0/1231574656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d3415ebd0 0x7f7d3406cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d3bcc8640 1 --2- 172.21.15.94:0/1231574656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3415e7c0 0x7f7d3406c600 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d3bcc8640 1 --2- 172.21.15.94:0/1231574656 >> v2:172.21.15.138:3300/0 conn(0x7f7d34070080 0x7f7d3415e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/1231574656 >> 172.21.15.94:0/1231574656 conn(0x7f7d340b47f0 msgr2=0x7f7d340b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/1231574656 shutdown_connections 2024-04-25T16:51:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/1231574656 wait complete. 2024-04-25T16:51:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d3bcc8640 1 Processor -- start 2024-04-25T16:51:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.216+0000 7f7d3bcc8640 1 -- start start 2024-04-25T16:51:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d3bcc8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d34070080 0x7f7d340d1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d3bcc8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3415e7c0 0x7f7d340ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d3bcc8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3415ebd0 0x7f7d341f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d3bcc8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d34070490 con 0x7f7d3415e7c0 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d3bcc8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d340ae6c0 con 0x7f7d3415ebd0 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d3bcc8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3406d490 con 0x7f7d34070080 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d39263640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3415e7c0 0x7f7d340ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d39263640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3415e7c0 0x7f7d340ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35944/0 (socket says 172.21.15.94:35944) 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d39263640 1 -- 172.21.15.94:0/4181922078 learned_addr learned my addr 172.21.15.94:0/4181922078 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d39a64640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d34070080 0x7f7d340d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d3a265640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3415ebd0 0x7f7d341f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d39263640 1 -- 172.21.15.94:0/4181922078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d34070080 msgr2=0x7f7d340d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d39263640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d34070080 0x7f7d340d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d39263640 1 -- 172.21.15.94:0/4181922078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3415ebd0 msgr2=0x7f7d341f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d39263640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3415ebd0 0x7f7d341f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d39263640 1 -- 172.21.15.94:0/4181922078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d340cd8c0 con 0x7f7d3415e7c0 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d3a265640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3415ebd0 0x7f7d341f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d39263640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3415e7c0 0x7f7d340ce680 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d2401aba0 tx=0x7f7d2401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:08.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d2a7fc640 1 -- 172.21.15.94:0/4181922078 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d240bd5f0 con 0x7f7d3415e7c0 2024-04-25T16:51:08.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d2a7fc640 1 -- 172.21.15.94:0/4181922078 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d240bd790 con 0x7f7d3415e7c0 2024-04-25T16:51:08.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/4181922078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d340cdba0 con 0x7f7d3415e7c0 2024-04-25T16:51:08.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d2a7fc640 1 -- 172.21.15.94:0/4181922078 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d240bdb80 con 0x7f7d3415e7c0 2024-04-25T16:51:08.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/4181922078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d341f7f30 con 0x7f7d3415e7c0 2024-04-25T16:51:08.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/4181922078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d34076830 con 0x7f7d3415e7c0 2024-04-25T16:51:08.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d2a7fc640 1 -- 172.21.15.94:0/4181922078 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7d240b33f0 con 0x7f7d3415e7c0 2024-04-25T16:51:08.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d2a7fc640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7d1806f400 0x7f7d18071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.220+0000 7f7d39a64640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7d1806f400 0x7f7d18071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.224+0000 7f7d2a7fc640 1 -- 172.21.15.94:0/4181922078 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7d24129f90 con 0x7f7d3415e7c0 2024-04-25T16:51:08.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.224+0000 7f7d39a64640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7d1806f400 0x7f7d18071910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d340b50c0 tx=0x7f7d30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.228+0000 7f7d2a7fc640 1 -- 172.21.15.94:0/4181922078 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7d240c3060 con 0x7f7d3415e7c0 2024-04-25T16:51:08.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.376+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/4181922078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d3409e8b0 con 0x7f7d3415e7c0 2024-04-25T16:51:08.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.380+0000 7f7d2a7fc640 1 -- 172.21.15.94:0/4181922078 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f7d240a7070 con 0x7f7d3415e7c0 2024-04-25T16:51:08.376 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:08.376 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:59.528743+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:08.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.380+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/4181922078 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7d1806f400 msgr2=0x7f7d18071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.380+0000 7f7d3bcc8640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7d1806f400 0x7f7d18071910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d340b50c0 tx=0x7f7d30008040 comp rx=0 tx=0).stop 2024-04-25T16:51:08.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.380+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/4181922078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3415e7c0 msgr2=0x7f7d340ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.380+0000 7f7d3bcc8640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3415e7c0 0x7f7d340ce680 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d2401aba0 tx=0x7f7d2401ad30 comp rx=0 tx=0).stop 2024-04-25T16:51:08.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.380+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/4181922078 shutdown_connections 2024-04-25T16:51:08.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.380+0000 7f7d3bcc8640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3415ebd0 0x7f7d341f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.380+0000 7f7d3bcc8640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3415e7c0 0x7f7d340ce680 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.380+0000 7f7d3bcc8640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7d1806f400 0x7f7d18071910 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.380+0000 7f7d3bcc8640 1 --2- 172.21.15.94:0/4181922078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d34070080 0x7f7d340d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.380+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/4181922078 >> 172.21.15.94:0/4181922078 conn(0x7f7d340b47f0 msgr2=0x7f7d340b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:08.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.380+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/4181922078 shutdown_connections 2024-04-25T16:51:08.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.380+0000 7f7d3bcc8640 1 -- 172.21.15.94:0/4181922078 wait complete. 2024-04-25T16:51:08.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:51:08.398 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:51:08.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcbc947640 1 Processor -- start 2024-04-25T16:51:08.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcbc947640 1 -- start start 2024-04-25T16:51:08.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcbc947640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdcb815c7e0 0x7fdcb80afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcbc947640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdcb80b0330 0x7fdcb806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcbc947640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcb8074380 con 0x7fdcb806dc50 2024-04-25T16:51:08.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcbc947640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fdcb806e060 con 0x7fdcb80b0330 2024-04-25T16:51:08.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcbc947640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdcb80727e0 con 0x7fdcb815c7e0 2024-04-25T16:51:08.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcb5d9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdcb80b0330 0x7fdcb806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcb6d9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fdcb806dc50 0x7fdcb806e7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39096/0 (socket says 172.21.15.94:39096) 2024-04-25T16:51:08.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcb5d9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdcb80b0330 0x7fdcb806d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35958/0 (socket says 172.21.15.94:35958) 2024-04-25T16:51:08.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcb6d9d640 1 -- 172.21.15.94:0/788953703 learned_addr learned my addr 172.21.15.94:0/788953703 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:08.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcb659c640 1 --2- 172.21.15.94:0/788953703 >> v2:172.21.15.138:3300/0 conn(0x7fdcb815c7e0 0x7fdcb80afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcb5d9b640 1 -- 172.21.15.94:0/788953703 >> v2:172.21.15.138:3300/0 conn(0x7fdcb815c7e0 msgr2=0x7fdcb80afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:08.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcb5d9b640 1 --2- 172.21.15.94:0/788953703 >> v2:172.21.15.138:3300/0 conn(0x7fdcb815c7e0 0x7fdcb80afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcb5d9b640 1 -- 172.21.15.94:0/788953703 >> v1:172.21.15.94:6789/0 conn(0x7fdcb806dc50 legacy=0x7fdcb806e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.548+0000 7fdcb5d9b640 1 -- 172.21.15.94:0/788953703 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb8072ac0 con 0x7fdcb80b0330 2024-04-25T16:51:08.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.552+0000 7fdcb5d9b640 1 --2- 172.21.15.94:0/788953703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb80b0330 0x7fdcb806d710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8070940 tx=0x7fdca801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7fea99c608060c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:08.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.552+0000 7fdcb559a640 1 -- 172.21.15.94:0/788953703 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca80b1070 con 0x7fdcb80b0330 2024-04-25T16:51:08.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.552+0000 7fdcb559a640 1 -- 172.21.15.94:0/788953703 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdca80ac620 con 0x7fdcb80b0330 2024-04-25T16:51:08.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.552+0000 7fdcb559a640 1 -- 172.21.15.94:0/788953703 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca80ac9f0 con 0x7fdcb80b0330 2024-04-25T16:51:08.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.552+0000 7fdcbc947640 1 -- 172.21.15.94:0/788953703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb80b0330 msgr2=0x7fdcb806d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.552+0000 7fdcbc947640 1 --2- 172.21.15.94:0/788953703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb80b0330 0x7fdcb806d710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8070940 tx=0x7fdca801c3c0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.552+0000 7fdcbc947640 1 -- 172.21.15.94:0/788953703 shutdown_connections 2024-04-25T16:51:08.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.552+0000 7fdcbc947640 1 --2- 172.21.15.94:0/788953703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb80b0330 0x7fdcb806d710 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.552+0000 7fdcbc947640 1 --2- 172.21.15.94:0/788953703 >> v2:172.21.15.138:3300/0 conn(0x7fdcb815c7e0 0x7fdcb80afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.552+0000 7fdcbc947640 1 -- 172.21.15.94:0/788953703 >> 172.21.15.94:0/788953703 conn(0x7fdcb80b47f0 msgr2=0x7fdcb80b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:08.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.552+0000 7fdcbc947640 1 -- 172.21.15.94:0/788953703 shutdown_connections 2024-04-25T16:51:08.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.552+0000 7fdcbc947640 1 -- 172.21.15.94:0/788953703 wait complete. 2024-04-25T16:51:08.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcbc947640 1 Processor -- start 2024-04-25T16:51:08.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcbc947640 1 -- start start 2024-04-25T16:51:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcbc947640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcb806dc50 0x7fdcb81f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcbc947640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb815c7e0 0x7fdcb815ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcbc947640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcb815efd0 0x7fdcb80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcbc947640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb806f2c0 con 0x7fdcb815c7e0 2024-04-25T16:51:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcbc947640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8073ca0 con 0x7fdcb806dc50 2024-04-25T16:51:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcbc947640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb815cbf0 con 0x7fdcb815efd0 2024-04-25T16:51:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcb5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb815c7e0 0x7fdcb815ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcb5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb815c7e0 0x7fdcb815ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35970/0 (socket says 172.21.15.94:35970) 2024-04-25T16:51:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcb5d9b640 1 -- 172.21.15.94:0/2521689005 learned_addr learned my addr 172.21.15.94:0/2521689005 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcb5d9b640 1 -- 172.21.15.94:0/2521689005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcb815efd0 msgr2=0x7fdcb80b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcb6d9d640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcb815efd0 0x7fdcb80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcb659c640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcb806dc50 0x7fdcb81f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcb5d9b640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcb815efd0 0x7fdcb80b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcb5d9b640 1 -- 172.21.15.94:0/2521689005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcb806dc50 msgr2=0x7fdcb81f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcb5d9b640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcb806dc50 0x7fdcb81f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcb5d9b640 1 -- 172.21.15.94:0/2521689005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb814ab40 con 0x7fdcb815c7e0 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcb659c640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcb806dc50 0x7fdcb81f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcb5d9b640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb815c7e0 0x7fdcb815ea90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdca8019ff0 tx=0x7fdca8019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcaeffd640 1 -- 172.21.15.94:0/2521689005 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca80b1040 con 0x7fdcb815c7e0 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcaeffd640 1 -- 172.21.15.94:0/2521689005 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdca80bb070 con 0x7fdcb815c7e0 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcbc947640 1 -- 172.21.15.94:0/2521689005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb814bbe0 con 0x7fdcb815c7e0 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcaeffd640 1 -- 172.21.15.94:0/2521689005 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca80b5650 con 0x7fdcb815c7e0 2024-04-25T16:51:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcbc947640 1 -- 172.21.15.94:0/2521689005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb814b100 con 0x7fdcb815c7e0 2024-04-25T16:51:08.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.556+0000 7fdcbc947640 1 -- 172.21.15.94:0/2521689005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb8076830 con 0x7fdcb815c7e0 2024-04-25T16:51:08.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.560+0000 7fdcaeffd640 1 -- 172.21.15.94:0/2521689005 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdca80b57f0 con 0x7fdcb815c7e0 2024-04-25T16:51:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.560+0000 7fdcaeffd640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdc9806f400 0x7fdc98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.560+0000 7fdcaeffd640 1 -- 172.21.15.94:0/2521689005 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdca81290d0 con 0x7fdcb815c7e0 2024-04-25T16:51:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.560+0000 7fdcb659c640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdc9806f400 0x7fdc98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.560+0000 7fdcb659c640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdc9806f400 0x7fdc98071910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdca00097a0 tx=0x7fdca0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:08.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.564+0000 7fdcaeffd640 1 -- 172.21.15.94:0/2521689005 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdca80c6060 con 0x7fdcb815c7e0 2024-04-25T16:51:08.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.720+0000 7fdcbc947640 1 -- 172.21.15.94:0/2521689005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdcb81601d0 con 0x7fdcb815c7e0 2024-04-25T16:51:08.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.720+0000 7fdcaeffd640 1 -- 172.21.15.94:0/2521689005 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fdca8016070 con 0x7fdcb815c7e0 2024-04-25T16:51:08.718 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:51:08.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.724+0000 7fdcbc947640 1 -- 172.21.15.94:0/2521689005 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdc9806f400 msgr2=0x7fdc98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.724+0000 7fdcbc947640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdc9806f400 0x7fdc98071910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdca00097a0 tx=0x7fdca0008040 comp rx=0 tx=0).stop 2024-04-25T16:51:08.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.724+0000 7fdcbc947640 1 -- 172.21.15.94:0/2521689005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb815c7e0 msgr2=0x7fdcb815ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.724+0000 7fdcbc947640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb815c7e0 0x7fdcb815ea90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdca8019ff0 tx=0x7fdca8019160 comp rx=0 tx=0).stop 2024-04-25T16:51:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.724+0000 7fdcbc947640 1 -- 172.21.15.94:0/2521689005 shutdown_connections 2024-04-25T16:51:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.724+0000 7fdcbc947640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdc9806f400 0x7fdc98071910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.724+0000 7fdcbc947640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcb815efd0 0x7fdcb80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.724+0000 7fdcbc947640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb815c7e0 0x7fdcb815ea90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.724+0000 7fdcbc947640 1 --2- 172.21.15.94:0/2521689005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcb806dc50 0x7fdcb81f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.724+0000 7fdcbc947640 1 -- 172.21.15.94:0/2521689005 >> 172.21.15.94:0/2521689005 conn(0x7fdcb80b47f0 msgr2=0x7fdcb806c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.724+0000 7fdcbc947640 1 -- 172.21.15.94:0/2521689005 shutdown_connections 2024-04-25T16:51:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.724+0000 7fdcbc947640 1 -- 172.21.15.94:0/2521689005 wait complete. 2024-04-25T16:51:08.738 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:51:08.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.892+0000 7f7443c49640 1 Processor -- start 2024-04-25T16:51:08.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.892+0000 7f7443c49640 1 -- start start 2024-04-25T16:51:08.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.892+0000 7f7443c49640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f743c0ccfa0 0x7f743c0d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.892+0000 7f7443c49640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f743c0d1570 con 0x7f743c160a90 2024-04-25T16:51:08.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.892+0000 7f7443c49640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f743c0d1850 con 0x7f743c0b7ae0 2024-04-25T16:51:08.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.892+0000 7f7443c49640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f743c0d1b30 con 0x7f743c0ccfa0 2024-04-25T16:51:08.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74411e4640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f743c160a90 0x7f743c0b73d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39102/0 (socket says 172.21.15.94:39102) 2024-04-25T16:51:08.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74411e4640 1 -- 172.21.15.94:0/527938881 learned_addr learned my addr 172.21.15.94:0/527938881 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:08.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74419e5640 1 --2- 172.21.15.94:0/527938881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f743c0ccfa0 0x7f743c0d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2599818070 0 0) 0x7f743c0d1570 con 0x7f743c160a90 2024-04-25T16:51:08.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7428003530 con 0x7f743c160a90 2024-04-25T16:51:08.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1632042544 0 0) 0x7f7428003530 con 0x7f743c160a90 2024-04-25T16:51:08.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f743c0ccfa0 msgr2=0x7f743c0d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:08.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 --2- 172.21.15.94:0/527938881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f743c0ccfa0 0x7f743c0d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 >> v1:172.21.15.138:6789/0 conn(0x7f743c0b7ae0 legacy=0x7f743c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c0d1e10 con 0x7f743c160a90 2024-04-25T16:51:08.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f742c001680 con 0x7f743c160a90 2024-04-25T16:51:08.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 >> v1:172.21.15.94:6789/0 conn(0x7f743c160a90 legacy=0x7f743c0b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 --2- 172.21.15.94:0/527938881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74280039f0 0x7f7428003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 --2- 172.21.15.94:0/527938881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7428004810 0x7f7428004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 --2- 172.21.15.94:0/527938881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f743c0ccfa0 0x7f74280052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f743c0d1b30 con 0x7f7428004810 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f742800bb60 con 0x7f74280039f0 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f742800bd60 con 0x7f743c0ccfa0 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74411e4640 1 --2- 172.21.15.94:0/527938881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7428004810 0x7f7428004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74419e5640 1 --2- 172.21.15.94:0/527938881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74280039f0 0x7f7428003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74421e6640 1 --2- 172.21.15.94:0/527938881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f743c0ccfa0 0x7f74280052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74411e4640 1 -- 172.21.15.94:0/527938881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f743c0ccfa0 msgr2=0x7f74280052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74411e4640 1 --2- 172.21.15.94:0/527938881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f743c0ccfa0 0x7f74280052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74411e4640 1 -- 172.21.15.94:0/527938881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74280039f0 msgr2=0x7f7428003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74411e4640 1 --2- 172.21.15.94:0/527938881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74280039f0 0x7f7428003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74411e4640 1 -- 172.21.15.94:0/527938881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7428003530 con 0x7f7428004810 2024-04-25T16:51:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74411e4640 1 --2- 172.21.15.94:0/527938881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7428004810 0x7f7428004c20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f743c0738b0 tx=0x7f742c01f290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9864225d4b5e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c007c50 con 0x7f7428004810 2024-04-25T16:51:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742c0ac020 con 0x7f7428004810 2024-04-25T16:51:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.896+0000 7f74409e3640 1 -- 172.21.15.94:0/527938881 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c0058e0 con 0x7f7428004810 2024-04-25T16:51:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 -- 172.21.15.94:0/527938881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7428004810 msgr2=0x7f7428004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 --2- 172.21.15.94:0/527938881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7428004810 0x7f7428004c20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f743c0738b0 tx=0x7f742c01f290 comp rx=0 tx=0).stop 2024-04-25T16:51:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f74421e6640 1 -- 172.21.15.94:0/527938881 reap_dead start 2024-04-25T16:51:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 -- 172.21.15.94:0/527938881 shutdown_connections 2024-04-25T16:51:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 -- 172.21.15.94:0/527938881 >> 172.21.15.94:0/527938881 conn(0x7f743c0d33d0 msgr2=0x7f743c0d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 -- 172.21.15.94:0/527938881 shutdown_connections 2024-04-25T16:51:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 -- 172.21.15.94:0/527938881 wait complete. 2024-04-25T16:51:08.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 Processor -- start 2024-04-25T16:51:08.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 -- start start 2024-04-25T16:51:08.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7428004810 0x7f743c1f9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f743c156030 0x7f743c148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f743c0b4210 0x7f743c0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f743c0cd670 con 0x7f7428004810 2024-04-25T16:51:08.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f743c0cd4f0 con 0x7f743c0b4210 2024-04-25T16:51:08.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f7443c49640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f743c0d0830 con 0x7f743c156030 2024-04-25T16:51:08.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f74419e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7428004810 0x7f743c1f9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f74419e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7428004810 0x7f743c1f9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35994/0 (socket says 172.21.15.94:35994) 2024-04-25T16:51:08.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.900+0000 7f74419e5640 1 -- 172.21.15.94:0/1120309976 learned_addr learned my addr 172.21.15.94:0/1120309976 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:08.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74411e4640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f743c156030 0x7f743c148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74421e6640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f743c0b4210 0x7f743c0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74419e5640 1 -- 172.21.15.94:0/1120309976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f743c156030 msgr2=0x7f743c148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74419e5640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f743c156030 0x7f743c148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74419e5640 1 -- 172.21.15.94:0/1120309976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f743c0b4210 msgr2=0x7f743c0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:08.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74419e5640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f743c0b4210 0x7f743c0b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:08.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74419e5640 1 -- 172.21.15.94:0/1120309976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c0b4b60 con 0x7f7428004810 2024-04-25T16:51:08.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74419e5640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7428004810 0x7f743c1f9d20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7438007f00 tx=0x7f74380115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:08.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74327fc640 1 -- 172.21.15.94:0/1120309976 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743801a020 con 0x7f7428004810 2024-04-25T16:51:08.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74327fc640 1 -- 172.21.15.94:0/1120309976 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7438016470 con 0x7f7428004810 2024-04-25T16:51:08.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f7443c49640 1 -- 172.21.15.94:0/1120309976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c159220 con 0x7f7428004810 2024-04-25T16:51:08.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74327fc640 1 -- 172.21.15.94:0/1120309976 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438016860 con 0x7f7428004810 2024-04-25T16:51:08.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f7443c49640 1 -- 172.21.15.94:0/1120309976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c06e110 con 0x7f7428004810 2024-04-25T16:51:08.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f7443c49640 1 -- 172.21.15.94:0/1120309976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743c076830 con 0x7f7428004810 2024-04-25T16:51:08.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74327fc640 1 -- 172.21.15.94:0/1120309976 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f743801d020 con 0x7f7428004810 2024-04-25T16:51:08.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74327fc640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f742006f430 0x7f7420071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:08.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74327fc640 1 -- 172.21.15.94:0/1120309976 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7438090430 con 0x7f7428004810 2024-04-25T16:51:08.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74411e4640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f742006f430 0x7f7420071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:08.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.904+0000 7f74411e4640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f742006f430 0x7f7420071940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f742c0ad000 tx=0x7f743c0d1a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:08.908+0000 7f74327fc640 1 -- 172.21.15.94:0/1120309976 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7438024030 con 0x7f7428004810 2024-04-25T16:51:09.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.060+0000 7f7443c49640 1 -- 172.21.15.94:0/1120309976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f743c0d12b0 con 0x7f7428004810 2024-04-25T16:51:09.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.060+0000 7f74327fc640 1 -- 172.21.15.94:0/1120309976 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f743c0b4b60 con 0x7f7428004810 2024-04-25T16:51:09.058 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:09.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.064+0000 7f7443c49640 1 -- 172.21.15.94:0/1120309976 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f742006f430 msgr2=0x7f7420071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.064+0000 7f7443c49640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f742006f430 0x7f7420071940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f742c0ad000 tx=0x7f743c0d1a20 comp rx=0 tx=0).stop 2024-04-25T16:51:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.064+0000 7f7443c49640 1 -- 172.21.15.94:0/1120309976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7428004810 msgr2=0x7f743c1f9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.064+0000 7f7443c49640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7428004810 0x7f743c1f9d20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7438007f00 tx=0x7f74380115a0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.064+0000 7f7443c49640 1 -- 172.21.15.94:0/1120309976 shutdown_connections 2024-04-25T16:51:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.064+0000 7f7443c49640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f743c0b4210 0x7f743c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.064+0000 7f7443c49640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f742006f430 0x7f7420071940 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.064+0000 7f7443c49640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f743c156030 0x7f743c148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.064+0000 7f7443c49640 1 --2- 172.21.15.94:0/1120309976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7428004810 0x7f743c1f9d20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.064+0000 7f7443c49640 1 -- 172.21.15.94:0/1120309976 >> 172.21.15.94:0/1120309976 conn(0x7f743c0d33d0 msgr2=0x7f743c163ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.064+0000 7f7443c49640 1 -- 172.21.15.94:0/1120309976 shutdown_connections 2024-04-25T16:51:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.064+0000 7f7443c49640 1 -- 172.21.15.94:0/1120309976 wait complete. 2024-04-25T16:51:09.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:51:09.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c67d1b640 1 Processor -- start 2024-04-25T16:51:09.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c67d1b640 1 -- start start 2024-04-25T16:51:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c67d1b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6c600b0420 0x7f6c6006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c67d1b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c60070630 con 0x7f6c6006f2a0 2024-04-25T16:51:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c67d1b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c6006f6b0 con 0x7f6c6015ec30 2024-04-25T16:51:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c67d1b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6c600aeaa0 con 0x7f6c600b0420 2024-04-25T16:51:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c652b6640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f6c6006f2a0 0x7f6c6006c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39110/0 (socket says 172.21.15.94:39110) 2024-04-25T16:51:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c652b6640 1 -- 172.21.15.94:0/2384735798 learned_addr learned my addr 172.21.15.94:0/2384735798 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c65ab7640 1 --2- 172.21.15.94:0/2384735798 >> v2:172.21.15.138:3300/0 conn(0x7f6c600b0420 0x7f6c6006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1204406411 0 0) 0x7f6c60070630 con 0x7f6c6006f2a0 2024-04-25T16:51:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c4c003530 con 0x7f6c6006f2a0 2024-04-25T16:51:09.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1644268004 0 0) 0x7f6c4c003530 con 0x7f6c6006f2a0 2024-04-25T16:51:09.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 >> v2:172.21.15.138:3300/0 conn(0x7f6c600b0420 msgr2=0x7f6c6006ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:09.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 --2- 172.21.15.94:0/2384735798 >> v2:172.21.15.138:3300/0 conn(0x7f6c600b0420 0x7f6c6006ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 >> v1:172.21.15.138:6789/0 conn(0x7f6c6015ec30 legacy=0x7f6c6006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6006d420 con 0x7f6c6006f2a0 2024-04-25T16:51:09.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f6c50001680 con 0x7f6c6006f2a0 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 >> v1:172.21.15.94:6789/0 conn(0x7f6c6006f2a0 legacy=0x7f6c6006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 --2- 172.21.15.94:0/2384735798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c4c0039c0 0x7f6c4c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 --2- 172.21.15.94:0/2384735798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4c004810 0x7f6c4c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 --2- 172.21.15.94:0/2384735798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c4c0051d0 0x7f6c4c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c600aeaa0 con 0x7f6c4c0051d0 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c00c170 con 0x7f6c4c0039c0 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.232+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6c4c00c370 con 0x7f6c4c004810 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c662b8640 1 --2- 172.21.15.94:0/2384735798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c4c0051d0 0x7f6c4c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c65ab7640 1 --2- 172.21.15.94:0/2384735798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c4c0039c0 0x7f6c4c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c652b6640 1 --2- 172.21.15.94:0/2384735798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4c004810 0x7f6c4c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c662b8640 1 -- 172.21.15.94:0/2384735798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4c004810 msgr2=0x7f6c4c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c662b8640 1 --2- 172.21.15.94:0/2384735798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4c004810 0x7f6c4c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c662b8640 1 -- 172.21.15.94:0/2384735798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c4c0039c0 msgr2=0x7f6c4c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c662b8640 1 --2- 172.21.15.94:0/2384735798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c4c0039c0 0x7f6c4c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c662b8640 1 -- 172.21.15.94:0/2384735798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4c003530 con 0x7f6c4c0051d0 2024-04-25T16:51:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c662b8640 1 -- 172.21.15.94:0/2384735798 reap_dead start 2024-04-25T16:51:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c662b8640 1 --2- 172.21.15.94:0/2384735798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c4c0051d0 0x7f6c4c00b9d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6c58001e70 tx=0x7f6c5801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5acea8e8039d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58001a90 con 0x7f6c4c0051d0 2024-04-25T16:51:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c58003ba0 con 0x7f6c4c0051d0 2024-04-25T16:51:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c64ab5640 1 -- 172.21.15.94:0/2384735798 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c580050d0 con 0x7f6c4c0051d0 2024-04-25T16:51:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2384735798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c4c0051d0 msgr2=0x7f6c4c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c67d1b640 1 --2- 172.21.15.94:0/2384735798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c4c0051d0 0x7f6c4c00b9d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6c58001e70 tx=0x7f6c5801c500 comp rx=0 tx=0).stop 2024-04-25T16:51:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2384735798 shutdown_connections 2024-04-25T16:51:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c67d1b640 1 --2- 172.21.15.94:0/2384735798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c4c0051d0 0x7f6c4c00b9d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2384735798 >> 172.21.15.94:0/2384735798 conn(0x7f6c600b47f0 msgr2=0x7f6c600b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2384735798 shutdown_connections 2024-04-25T16:51:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.236+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2384735798 wait complete. 2024-04-25T16:51:09.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c67d1b640 1 Processor -- start 2024-04-25T16:51:09.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c67d1b640 1 -- start start 2024-04-25T16:51:09.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c67d1b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4c0051d0 0x7f6c6009b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c67d1b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c6009b870 0x7f6c601e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c67d1b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c601e8040 0x7f6c601e8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c67d1b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c600ae730 con 0x7f6c6009b870 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c67d1b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c600ae450 con 0x7f6c601e8040 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c67d1b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6c600703e0 con 0x7f6c4c0051d0 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c652b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c6009b870 0x7f6c601e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c652b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c6009b870 0x7f6c601e7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36014/0 (socket says 172.21.15.94:36014) 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c652b6640 1 -- 172.21.15.94:0/2132086936 learned_addr learned my addr 172.21.15.94:0/2132086936 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c65ab7640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4c0051d0 0x7f6c6009b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c662b8640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c601e8040 0x7f6c601e8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c652b6640 1 -- 172.21.15.94:0/2132086936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4c0051d0 msgr2=0x7f6c6009b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c652b6640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4c0051d0 0x7f6c6009b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c652b6640 1 -- 172.21.15.94:0/2132086936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c601e8040 msgr2=0x7f6c601e8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c652b6640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c601e8040 0x7f6c601e8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c652b6640 1 -- 172.21.15.94:0/2132086936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c601e8990 con 0x7f6c6009b870 2024-04-25T16:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c65ab7640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4c0051d0 0x7f6c6009b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c652b6640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c6009b870 0x7f6c601e7b00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000e710 tx=0x7f6c5000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c567fc640 1 -- 172.21.15.94:0/2132086936 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000ee20 con 0x7f6c6009b870 2024-04-25T16:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c567fc640 1 -- 172.21.15.94:0/2132086936 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c50018070 con 0x7f6c6009b870 2024-04-25T16:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2132086936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c6009c060 con 0x7f6c6009b870 2024-04-25T16:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c567fc640 1 -- 172.21.15.94:0/2132086936 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000caf0 con 0x7f6c6009b870 2024-04-25T16:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.240+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2132086936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c6009c620 con 0x7f6c6009b870 2024-04-25T16:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.244+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2132086936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c60076830 con 0x7f6c6009b870 2024-04-25T16:51:09.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.244+0000 7f6c567fc640 1 -- 172.21.15.94:0/2132086936 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6c5000cc90 con 0x7f6c6009b870 2024-04-25T16:51:09.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.244+0000 7f6c567fc640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6c4406f400 0x7f6c44071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.244+0000 7f6c65ab7640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6c4406f400 0x7f6c44071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.244+0000 7f6c567fc640 1 -- 172.21.15.94:0/2132086936 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6c5008f350 con 0x7f6c6009b870 2024-04-25T16:51:09.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.244+0000 7f6c65ab7640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6c4406f400 0x7f6c44071910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c600cd2f0 tx=0x7f6c5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:09.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.248+0000 7f6c567fc640 1 -- 172.21.15.94:0/2132086936 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6c5005c020 con 0x7f6c6009b870 2024-04-25T16:51:09.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.408+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2132086936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6c6006fb20 con 0x7f6c6009b870 2024-04-25T16:51:09.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.408+0000 7f6c567fc640 1 -- 172.21.15.94:0/2132086936 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f6c5005b920 con 0x7f6c6009b870 2024-04-25T16:51:09.408 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:51:09.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.412+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2132086936 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6c4406f400 msgr2=0x7f6c44071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.412+0000 7f6c67d1b640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6c4406f400 0x7f6c44071910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c600cd2f0 tx=0x7f6c5c008040 comp rx=0 tx=0).stop 2024-04-25T16:51:09.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.412+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2132086936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c6009b870 msgr2=0x7f6c601e7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.412+0000 7f6c67d1b640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c6009b870 0x7f6c601e7b00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000e710 tx=0x7f6c5000ebd0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.412+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2132086936 shutdown_connections 2024-04-25T16:51:09.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.412+0000 7f6c67d1b640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c601e8040 0x7f6c601e8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.412+0000 7f6c67d1b640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c6009b870 0x7f6c601e7b00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.412+0000 7f6c67d1b640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6c4406f400 0x7f6c44071910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.412+0000 7f6c67d1b640 1 --2- 172.21.15.94:0/2132086936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4c0051d0 0x7f6c6009b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.412+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2132086936 >> 172.21.15.94:0/2132086936 conn(0x7f6c600b47f0 msgr2=0x7f6c600b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:09.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.412+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2132086936 shutdown_connections 2024-04-25T16:51:09.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.412+0000 7f6c67d1b640 1 -- 172.21.15.94:0/2132086936 wait complete. 2024-04-25T16:51:09.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:51:09.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6eeb273640 1 Processor -- start 2024-04-25T16:51:09.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6eeb273640 1 -- start start 2024-04-25T16:51:09.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6eeb273640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6ee415c7e0 0x7f6ee40afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6eeb273640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ee40b0330 0x7f6ee406d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6eeb273640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee4074380 con 0x7f6ee406dc50 2024-04-25T16:51:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6eeb273640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee406e060 con 0x7f6ee415c7e0 2024-04-25T16:51:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6eeb273640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee40727e0 con 0x7f6ee40b0330 2024-04-25T16:51:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6ee9810640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f6ee406dc50 0x7f6ee406e7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39122/0 (socket says 172.21.15.94:39122) 2024-04-25T16:51:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6ee9810640 1 -- 172.21.15.94:0/69215711 learned_addr learned my addr 172.21.15.94:0/69215711 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6ee900f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6ee415c7e0 0x7f6ee40afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6ee880e640 1 --2- 172.21.15.94:0/69215711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ee40b0330 0x7f6ee406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6ee900f640 1 -- 172.21.15.94:0/69215711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ee40b0330 msgr2=0x7f6ee406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6ee900f640 1 --2- 172.21.15.94:0/69215711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ee40b0330 0x7f6ee406d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6ee900f640 1 -- 172.21.15.94:0/69215711 >> v1:172.21.15.94:6789/0 conn(0x7f6ee406dc50 legacy=0x7f6ee406e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6ee900f640 1 -- 172.21.15.94:0/69215711 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee4072ac0 con 0x7f6ee415c7e0 2024-04-25T16:51:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6edbfff640 1 -- 172.21.15.94:0/69215711 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4240395563 0 0) 0x7f6edc0014a0 con 0x7f6ee406dc50 2024-04-25T16:51:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.584+0000 7f6ee880e640 1 --2- 172.21.15.94:0/69215711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ee40b0330 0x7f6ee406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6ee900f640 1 --2- 172.21.15.94:0/69215711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ee415c7e0 0x7f6ee40afdf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00097c0 tx=0x7f6ee001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87b388c61910889 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6edbfff640 1 -- 172.21.15.94:0/69215711 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00b1070 con 0x7f6ee415c7e0 2024-04-25T16:51:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6edbfff640 1 -- 172.21.15.94:0/69215711 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee00a8050 con 0x7f6ee415c7e0 2024-04-25T16:51:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6edbfff640 1 -- 172.21.15.94:0/69215711 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00ac860 con 0x7f6ee415c7e0 2024-04-25T16:51:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6eeb273640 1 -- 172.21.15.94:0/69215711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ee415c7e0 msgr2=0x7f6ee40afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6eeb273640 1 --2- 172.21.15.94:0/69215711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ee415c7e0 0x7f6ee40afdf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00097c0 tx=0x7f6ee001c3e0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6eeb273640 1 -- 172.21.15.94:0/69215711 shutdown_connections 2024-04-25T16:51:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6eeb273640 1 --2- 172.21.15.94:0/69215711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ee40b0330 0x7f6ee406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6eeb273640 1 --2- 172.21.15.94:0/69215711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ee415c7e0 0x7f6ee40afdf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6eeb273640 1 -- 172.21.15.94:0/69215711 >> 172.21.15.94:0/69215711 conn(0x7f6ee40b47f0 msgr2=0x7f6ee40b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:09.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6eeb273640 1 -- 172.21.15.94:0/69215711 shutdown_connections 2024-04-25T16:51:09.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6eeb273640 1 -- 172.21.15.94:0/69215711 wait complete. 2024-04-25T16:51:09.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6eeb273640 1 Processor -- start 2024-04-25T16:51:09.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.588+0000 7f6eeb273640 1 -- start start 2024-04-25T16:51:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6eeb273640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ee406dc50 0x7f6ee4158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6eeb273640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ee40b0330 0x7f6ee41fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6eeb273640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ee415c7e0 0x7f6ee4166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6eeb273640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee406f2c0 con 0x7f6ee415c7e0 2024-04-25T16:51:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6eeb273640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee4073ca0 con 0x7f6ee406dc50 2024-04-25T16:51:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6eeb273640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee415cbf0 con 0x7f6ee40b0330 2024-04-25T16:51:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ee9810640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ee415c7e0 0x7f6ee4166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ee9810640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ee415c7e0 0x7f6ee4166aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36036/0 (socket says 172.21.15.94:36036) 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ee9810640 1 -- 172.21.15.94:0/214539336 learned_addr learned my addr 172.21.15.94:0/214539336 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ee900f640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ee406dc50 0x7f6ee4158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ee880e640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ee40b0330 0x7f6ee41fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ee9810640 1 -- 172.21.15.94:0/214539336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ee40b0330 msgr2=0x7f6ee41fb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ee9810640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ee40b0330 0x7f6ee41fb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ee9810640 1 -- 172.21.15.94:0/214539336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ee406dc50 msgr2=0x7f6ee4158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ee9810640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ee406dc50 0x7f6ee4158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ee9810640 1 -- 172.21.15.94:0/214539336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee4158400 con 0x7f6ee415c7e0 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ee900f640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ee406dc50 0x7f6ee4158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ee9810640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ee415c7e0 0x7f6ee4166aa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00c6d0 tx=0x7f6edc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ed9ffb640 1 -- 172.21.15.94:0/214539336 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc018070 con 0x7f6ee415c7e0 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ed9ffb640 1 -- 172.21.15.94:0/214539336 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6edc004d70 con 0x7f6ee415c7e0 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6eeb273640 1 -- 172.21.15.94:0/214539336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee41586e0 con 0x7f6ee415c7e0 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ed9ffb640 1 -- 172.21.15.94:0/214539336 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc00a880 con 0x7f6ee415c7e0 2024-04-25T16:51:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6eeb273640 1 -- 172.21.15.94:0/214539336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee41fb8f0 con 0x7f6ee415c7e0 2024-04-25T16:51:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6eeb273640 1 -- 172.21.15.94:0/214539336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee41fa9c0 con 0x7f6ee415c7e0 2024-04-25T16:51:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ed9ffb640 1 -- 172.21.15.94:0/214539336 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6edc01b070 con 0x7f6ee415c7e0 2024-04-25T16:51:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.592+0000 7f6ed9ffb640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6ec806f400 0x7f6ec8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.596+0000 7f6ee900f640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6ec806f400 0x7f6ec8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.596+0000 7f6ed9ffb640 1 -- 172.21.15.94:0/214539336 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6edc08edb0 con 0x7f6ee415c7e0 2024-04-25T16:51:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.596+0000 7f6ee900f640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6ec806f400 0x7f6ec8071910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00ae000 tx=0x7f6ee0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:09.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.600+0000 7f6ed9ffb640 1 -- 172.21.15.94:0/214539336 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6ee4074380 con 0x7f6ee415c7e0 2024-04-25T16:51:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.748+0000 7f6eeb273640 1 -- 172.21.15.94:0/214539336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6ee4076830 con 0x7f6ee415c7e0 2024-04-25T16:51:09.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.752+0000 7f6ed9ffb640 1 -- 172.21.15.94:0/214539336 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f6edc05b960 con 0x7f6ee415c7e0 2024-04-25T16:51:09.748 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.752+0000 7f6eeb273640 1 -- 172.21.15.94:0/214539336 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6ec806f400 msgr2=0x7f6ec8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.752+0000 7f6eeb273640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6ec806f400 0x7f6ec8071910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00ae000 tx=0x7f6ee0008040 comp rx=0 tx=0).stop 2024-04-25T16:51:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.752+0000 7f6eeb273640 1 -- 172.21.15.94:0/214539336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ee415c7e0 msgr2=0x7f6ee4166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.752+0000 7f6eeb273640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ee415c7e0 0x7f6ee4166aa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00c6d0 tx=0x7f6edc00cb90 comp rx=0 tx=0).stop 2024-04-25T16:51:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.752+0000 7f6eeb273640 1 -- 172.21.15.94:0/214539336 shutdown_connections 2024-04-25T16:51:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.752+0000 7f6eeb273640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ee415c7e0 0x7f6ee4166aa0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.752+0000 7f6eeb273640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6ec806f400 0x7f6ec8071910 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.752+0000 7f6eeb273640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ee40b0330 0x7f6ee41fb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.752+0000 7f6eeb273640 1 --2- 172.21.15.94:0/214539336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ee406dc50 0x7f6ee4158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.752+0000 7f6eeb273640 1 -- 172.21.15.94:0/214539336 >> 172.21.15.94:0/214539336 conn(0x7f6ee40b47f0 msgr2=0x7f6ee4160480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.752+0000 7f6eeb273640 1 -- 172.21.15.94:0/214539336 shutdown_connections 2024-04-25T16:51:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.752+0000 7f6eeb273640 1 -- 172.21.15.94:0/214539336 wait complete. 2024-04-25T16:51:09.768 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:51:09.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.916+0000 7fbd1c1b6640 1 Processor -- start 2024-04-25T16:51:09.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.916+0000 7fbd1c1b6640 1 -- start start 2024-04-25T16:51:09.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.916+0000 7fbd1c1b6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd1406f470 0x7fbd1415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.916+0000 7fbd1c1b6640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbd1415e160 0x7fbd1415e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.916+0000 7fbd1c1b6640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd14070630 con 0x7fbd140b0420 2024-04-25T16:51:09.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.916+0000 7fbd1c1b6640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fbd140aeaa0 con 0x7fbd1415e160 2024-04-25T16:51:09.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.916+0000 7fbd1c1b6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd140703e0 con 0x7fbd1406f470 2024-04-25T16:51:09.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.916+0000 7fbd19f52640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fbd140b0420 0x7fbd1406ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39136/0 (socket says 172.21.15.94:39136) 2024-04-25T16:51:09.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.916+0000 7fbd19f52640 1 -- 172.21.15.94:0/1928176952 learned_addr learned my addr 172.21.15.94:0/1928176952 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:09.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.916+0000 7fbd1a753640 1 --2- 172.21.15.94:0/1928176952 >> v2:172.21.15.94:3300/0 conn(0x7fbd1415e160 0x7fbd1415e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.916+0000 7fbd18f50640 1 -- 172.21.15.94:0/1928176952 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 189210483 0 0) 0x7fbd14070630 con 0x7fbd140b0420 2024-04-25T16:51:09.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd18f50640 1 -- 172.21.15.94:0/1928176952 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcfc003530 con 0x7fbd140b0420 2024-04-25T16:51:09.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd19751640 1 --2- 172.21.15.94:0/1928176952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd1406f470 0x7fbd1415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd1a753640 1 -- 172.21.15.94:0/1928176952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd1406f470 msgr2=0x7fbd1415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:09.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd1a753640 1 --2- 172.21.15.94:0/1928176952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd1406f470 0x7fbd1415dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd1a753640 1 -- 172.21.15.94:0/1928176952 >> v1:172.21.15.94:6789/0 conn(0x7fbd140b0420 legacy=0x7fbd1406ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd1a753640 1 -- 172.21.15.94:0/1928176952 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd140ae170 con 0x7fbd1415e160 2024-04-25T16:51:09.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd1a753640 1 --2- 172.21.15.94:0/1928176952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd1415e160 0x7fbd1415e570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009d90 tx=0x7fbd0401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f015b59e8383a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:09.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd18f50640 1 -- 172.21.15.94:0/1928176952 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd040b2070 con 0x7fbd1415e160 2024-04-25T16:51:09.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd18f50640 1 -- 172.21.15.94:0/1928176952 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd040a5a00 con 0x7fbd1415e160 2024-04-25T16:51:09.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd18f50640 1 -- 172.21.15.94:0/1928176952 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd040a5db0 con 0x7fbd1415e160 2024-04-25T16:51:09.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1928176952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd1415e160 msgr2=0x7fbd1415e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd1c1b6640 1 --2- 172.21.15.94:0/1928176952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd1415e160 0x7fbd1415e570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009d90 tx=0x7fbd0401c940 comp rx=0 tx=0).stop 2024-04-25T16:51:09.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1928176952 shutdown_connections 2024-04-25T16:51:09.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd1c1b6640 1 --2- 172.21.15.94:0/1928176952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd1415e160 0x7fbd1415e570 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd1c1b6640 1 --2- 172.21.15.94:0/1928176952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd1406f470 0x7fbd1415dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1928176952 >> 172.21.15.94:0/1928176952 conn(0x7fbd140b47f0 msgr2=0x7fbd140b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:09.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1928176952 shutdown_connections 2024-04-25T16:51:09.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.920+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1928176952 wait complete. 2024-04-25T16:51:09.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1c1b6640 1 Processor -- start 2024-04-25T16:51:09.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1c1b6640 1 -- start start 2024-04-25T16:51:09.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1c1b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd140b0420 0x7fbd140997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1c1b6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd1415e160 0x7fbd14146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1c1b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd1409a480 0x7fbd140ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1c1b6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd140b09b0 con 0x7fbd1409a480 2024-04-25T16:51:09.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1c1b6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd140b0830 con 0x7fbd140b0420 2024-04-25T16:51:09.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1c1b6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd1416a6f0 con 0x7fbd1415e160 2024-04-25T16:51:09.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1a753640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd1409a480 0x7fbd140ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1a753640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd1409a480 0x7fbd140ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36052/0 (socket says 172.21.15.94:36052) 2024-04-25T16:51:09.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1a753640 1 -- 172.21.15.94:0/1189247405 learned_addr learned my addr 172.21.15.94:0/1189247405 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:09.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd19751640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd1415e160 0x7fbd14146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd19f52640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd140b0420 0x7fbd140997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1a753640 1 -- 172.21.15.94:0/1189247405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd1415e160 msgr2=0x7fbd14146590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1a753640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd1415e160 0x7fbd14146590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1a753640 1 -- 172.21.15.94:0/1189247405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd140b0420 msgr2=0x7fbd140997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:09.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1a753640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd140b0420 0x7fbd140997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:09.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1a753640 1 -- 172.21.15.94:0/1189247405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd141536f0 con 0x7fbd1409a480 2024-04-25T16:51:09.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd19f52640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd140b0420 0x7fbd140997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:09.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1a753640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd1409a480 0x7fbd140ce250 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbd0401aff0 tx=0x7fbd040196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:09.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd127fc640 1 -- 172.21.15.94:0/1189247405 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd040b2050 con 0x7fbd1409a480 2024-04-25T16:51:09.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd127fc640 1 -- 172.21.15.94:0/1189247405 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd040be070 con 0x7fbd1409a480 2024-04-25T16:51:09.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1189247405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd14153bf0 con 0x7fbd1409a480 2024-04-25T16:51:09.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd127fc640 1 -- 172.21.15.94:0/1189247405 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd040ada90 con 0x7fbd1409a480 2024-04-25T16:51:09.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.924+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1189247405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd140cd770 con 0x7fbd1409a480 2024-04-25T16:51:09.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.928+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1189247405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd14076830 con 0x7fbd1409a480 2024-04-25T16:51:09.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.928+0000 7fbd127fc640 1 -- 172.21.15.94:0/1189247405 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbd040adc30 con 0x7fbd1409a480 2024-04-25T16:51:09.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.928+0000 7fbd127fc640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbd0006f400 0x7fbd00071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:09.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.928+0000 7fbd19f52640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbd0006f400 0x7fbd00071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:09.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.928+0000 7fbd127fc640 1 -- 172.21.15.94:0/1189247405 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbd0412a8b0 con 0x7fbd1409a480 2024-04-25T16:51:09.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.928+0000 7fbd19f52640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbd0006f400 0x7fbd00071910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c005890 tx=0x7fbd0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:09.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:09.932+0000 7fbd127fc640 1 -- 172.21.15.94:0/1189247405 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbd04016560 con 0x7fbd1409a480 2024-04-25T16:51:10.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.108+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1189247405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd141541c0 con 0x7fbd1409a480 2024-04-25T16:51:10.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.108+0000 7fbd127fc640 1 -- 172.21.15.94:0/1189247405 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fbd04128e20 con 0x7fbd1409a480 2024-04-25T16:51:10.107 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:51:10.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.112+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1189247405 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbd0006f400 msgr2=0x7fbd00071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:10.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.112+0000 7fbd1c1b6640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbd0006f400 0x7fbd00071910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c005890 tx=0x7fbd0c009040 comp rx=0 tx=0).stop 2024-04-25T16:51:10.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.112+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1189247405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd1409a480 msgr2=0x7fbd140ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:10.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.112+0000 7fbd1c1b6640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd1409a480 0x7fbd140ce250 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbd0401aff0 tx=0x7fbd040196e0 comp rx=0 tx=0).stop 2024-04-25T16:51:10.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.112+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1189247405 shutdown_connections 2024-04-25T16:51:10.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.112+0000 7fbd1c1b6640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd1409a480 0x7fbd140ce250 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:10.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.112+0000 7fbd1c1b6640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbd0006f400 0x7fbd00071910 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:10.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.112+0000 7fbd1c1b6640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd1415e160 0x7fbd14146590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:10.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.112+0000 7fbd1c1b6640 1 --2- 172.21.15.94:0/1189247405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd140b0420 0x7fbd140997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:10.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.112+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1189247405 >> 172.21.15.94:0/1189247405 conn(0x7fbd140b47f0 msgr2=0x7fbd14157270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:10.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.112+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1189247405 shutdown_connections 2024-04-25T16:51:10.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:10.112+0000 7fbd1c1b6640 1 -- 172.21.15.94:0/1189247405 wait complete. 2024-04-25T16:51:10.128 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-04-25T16:51:10.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:10.129 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T16:51:10.272 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi094.stderr:ceph-fuse[33681]: fuse finished with error 0 and tester_r 0 2024-04-25T16:51:10.274 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:51:14.194 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:51:14.197 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:51:16.375 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:51:16.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:16.375 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T16:51:16.389 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.592+0000 7fd2137b9640 1 Processor -- start 2024-04-25T16:51:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.592+0000 7fd2137b9640 1 -- start start 2024-04-25T16:51:16.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.592+0000 7fd2137b9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd20c166d70 0x7fd20c169190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:16.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.592+0000 7fd2137b9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd20c06d610 con 0x7fd20c070080 2024-04-25T16:51:16.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.592+0000 7fd2137b9640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd20c06fc60 con 0x7fd20c15e990 2024-04-25T16:51:16.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.592+0000 7fd2137b9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd20c158550 con 0x7fd20c166d70 2024-04-25T16:51:16.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.592+0000 7fd211d56640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd20c166d70 0x7fd20c169190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:16.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.592+0000 7fd211d56640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd20c166d70 0x7fd20c169190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36066/0 (socket says 172.21.15.94:36066) 2024-04-25T16:51:16.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.592+0000 7fd211d56640 1 -- 172.21.15.94:0/1414739807 learned_addr learned my addr 172.21.15.94:0/1414739807 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:16.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.592+0000 7fd211d56640 1 -- 172.21.15.94:0/1414739807 >> v1:172.21.15.138:6789/0 conn(0x7fd20c15e990 legacy=0x7fd20c165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:16.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.592+0000 7fd211d56640 1 -- 172.21.15.94:0/1414739807 >> v1:172.21.15.94:6789/0 conn(0x7fd20c070080 legacy=0x7fd20c15e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:16.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.592+0000 7fd211d56640 1 -- 172.21.15.94:0/1414739807 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20c0aef70 con 0x7fd20c166d70 2024-04-25T16:51:16.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.596+0000 7fd211d56640 1 --2- 172.21.15.94:0/1414739807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd20c166d70 0x7fd20c169190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd20c1586d0 tx=0x7fd20401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd319fdbb559e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:16.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.596+0000 7fd203fff640 1 -- 172.21.15.94:0/1414739807 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2040a5780 con 0x7fd20c166d70 2024-04-25T16:51:16.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.596+0000 7fd203fff640 1 -- 172.21.15.94:0/1414739807 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2040a5920 con 0x7fd20c166d70 2024-04-25T16:51:16.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.596+0000 7fd203fff640 1 -- 172.21.15.94:0/1414739807 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2040a5cf0 con 0x7fd20c166d70 2024-04-25T16:51:16.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.596+0000 7fd2137b9640 1 -- 172.21.15.94:0/1414739807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd20c166d70 msgr2=0x7fd20c169190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:16.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.596+0000 7fd2137b9640 1 --2- 172.21.15.94:0/1414739807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd20c166d70 0x7fd20c169190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd20c1586d0 tx=0x7fd20401c9d0 comp rx=0 tx=0).stop 2024-04-25T16:51:16.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.596+0000 7fd2137b9640 1 -- 172.21.15.94:0/1414739807 shutdown_connections 2024-04-25T16:51:16.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.596+0000 7fd2137b9640 1 --2- 172.21.15.94:0/1414739807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd20c166d70 0x7fd20c169190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:16.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.596+0000 7fd2137b9640 1 -- 172.21.15.94:0/1414739807 >> 172.21.15.94:0/1414739807 conn(0x7fd20c0b47f0 msgr2=0x7fd20c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:16.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.596+0000 7fd2137b9640 1 -- 172.21.15.94:0/1414739807 shutdown_connections 2024-04-25T16:51:16.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.596+0000 7fd2137b9640 1 -- 172.21.15.94:0/1414739807 wait complete. 2024-04-25T16:51:16.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd2137b9640 1 Processor -- start 2024-04-25T16:51:16.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd2137b9640 1 -- start start 2024-04-25T16:51:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd2137b9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd20c070080 0x7fd20c13e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd2137b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd20c15e990 0x7fd20c156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd2137b9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd20c166d70 0x7fd20c150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd2137b9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c070490 con 0x7fd20c15e990 2024-04-25T16:51:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd2137b9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c0ae6c0 con 0x7fd20c070080 2024-04-25T16:51:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd2137b9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd20c15eda0 con 0x7fd20c166d70 2024-04-25T16:51:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd210d54640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd20c15e990 0x7fd20c156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd210d54640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd20c15e990 0x7fd20c156fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36076/0 (socket says 172.21.15.94:36076) 2024-04-25T16:51:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd210d54640 1 -- 172.21.15.94:0/384750656 learned_addr learned my addr 172.21.15.94:0/384750656 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd211d56640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd20c166d70 0x7fd20c150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd211555640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd20c070080 0x7fd20c13e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd210d54640 1 -- 172.21.15.94:0/384750656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd20c166d70 msgr2=0x7fd20c150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd210d54640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd20c166d70 0x7fd20c150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd210d54640 1 -- 172.21.15.94:0/384750656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd20c070080 msgr2=0x7fd20c13e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd210d54640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd20c070080 0x7fd20c13e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd210d54640 1 -- 172.21.15.94:0/384750656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20c167d30 con 0x7fd20c15e990 2024-04-25T16:51:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd210d54640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd20c15e990 0x7fd20c156fb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00b3c0 tx=0x7fd1fc00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd201ffb640 1 -- 172.21.15.94:0/384750656 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc011830 con 0x7fd20c15e990 2024-04-25T16:51:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd201ffb640 1 -- 172.21.15.94:0/384750656 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1fc016070 con 0x7fd20c15e990 2024-04-25T16:51:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd2137b9640 1 -- 172.21.15.94:0/384750656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20c168010 con 0x7fd20c15e990 2024-04-25T16:51:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd201ffb640 1 -- 172.21.15.94:0/384750656 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc011bf0 con 0x7fd20c15e990 2024-04-25T16:51:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.600+0000 7fd2137b9640 1 -- 172.21.15.94:0/384750656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20c1685d0 con 0x7fd20c15e990 2024-04-25T16:51:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.604+0000 7fd2137b9640 1 -- 172.21.15.94:0/384750656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd20c076830 con 0x7fd20c15e990 2024-04-25T16:51:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.604+0000 7fd201ffb640 1 -- 172.21.15.94:0/384750656 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd1fc01b460 con 0x7fd20c15e990 2024-04-25T16:51:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.604+0000 7fd201ffb640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd1f006f400 0x7fd1f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.604+0000 7fd201ffb640 1 -- 172.21.15.94:0/384750656 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd1fc08f2f0 con 0x7fd20c15e990 2024-04-25T16:51:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.604+0000 7fd211555640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd1f006f400 0x7fd1f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.604+0000 7fd211555640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd1f006f400 0x7fd1f0071910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0b50c0 tx=0x7fd208009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:16.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.608+0000 7fd201ffb640 1 -- 172.21.15.94:0/384750656 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd1fc05bea0 con 0x7fd20c15e990 2024-04-25T16:51:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.768+0000 7fd2137b9640 1 -- 172.21.15.94:0/384750656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd20c068bf0 con 0x7fd20c15e990 2024-04-25T16:51:16.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.772+0000 7fd201ffb640 1 -- 172.21.15.94:0/384750656 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7fd1fc05b7a0 con 0x7fd20c15e990 2024-04-25T16:51:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:16.771 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:59.528743+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:16.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.776+0000 7fd2137b9640 1 -- 172.21.15.94:0/384750656 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd1f006f400 msgr2=0x7fd1f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:16.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.776+0000 7fd2137b9640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd1f006f400 0x7fd1f0071910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0b50c0 tx=0x7fd208009040 comp rx=0 tx=0).stop 2024-04-25T16:51:16.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.776+0000 7fd2137b9640 1 -- 172.21.15.94:0/384750656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd20c15e990 msgr2=0x7fd20c156fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:16.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.776+0000 7fd2137b9640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd20c15e990 0x7fd20c156fb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00b3c0 tx=0x7fd1fc00b880 comp rx=0 tx=0).stop 2024-04-25T16:51:16.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.776+0000 7fd2137b9640 1 -- 172.21.15.94:0/384750656 shutdown_connections 2024-04-25T16:51:16.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.776+0000 7fd2137b9640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd1f006f400 0x7fd1f0071910 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:16.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.776+0000 7fd2137b9640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd20c166d70 0x7fd20c150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:16.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.776+0000 7fd2137b9640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd20c15e990 0x7fd20c156fb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:16.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.776+0000 7fd2137b9640 1 --2- 172.21.15.94:0/384750656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd20c070080 0x7fd20c13e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:16.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.776+0000 7fd2137b9640 1 -- 172.21.15.94:0/384750656 >> 172.21.15.94:0/384750656 conn(0x7fd20c0b47f0 msgr2=0x7fd20c0b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:16.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.776+0000 7fd2137b9640 1 -- 172.21.15.94:0/384750656 shutdown_connections 2024-04-25T16:51:16.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.776+0000 7fd2137b9640 1 -- 172.21.15.94:0/384750656 wait complete. 2024-04-25T16:51:16.791 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:16.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.940+0000 7f91da492640 1 Processor -- start 2024-04-25T16:51:16.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.940+0000 7f91da492640 1 -- start start 2024-04-25T16:51:16.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.944+0000 7f91da492640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f91d415e990 0x7f91d4165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:16.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.944+0000 7f91da492640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f91d4165b90 0x7f91d4167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:16.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.944+0000 7f91da492640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91d406d610 con 0x7f91d4070080 2024-04-25T16:51:16.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.944+0000 7f91da492640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f91d4158550 con 0x7f91d415e990 2024-04-25T16:51:16.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.944+0000 7f91da492640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f91d406f7d0 con 0x7f91d4165b90 2024-04-25T16:51:16.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.944+0000 7f91d37fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f91d415e990 0x7f91d4165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:16.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.944+0000 7f91d37fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f91d415e990 0x7f91d4165650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36086/0 (socket says 172.21.15.94:36086) 2024-04-25T16:51:16.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.944+0000 7f91d37fe640 1 -- 172.21.15.94:0/2911434885 learned_addr learned my addr 172.21.15.94:0/2911434885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:16.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.944+0000 7f91d8a2f640 1 --2- 172.21.15.94:0/2911434885 >> v2:172.21.15.138:3300/0 conn(0x7f91d4165b90 0x7f91d4167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:16.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.944+0000 7f91d37fe640 1 -- 172.21.15.94:0/2911434885 >> v2:172.21.15.138:3300/0 conn(0x7f91d4165b90 msgr2=0x7f91d4167fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:16.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.944+0000 7f91d37fe640 1 --2- 172.21.15.94:0/2911434885 >> v2:172.21.15.138:3300/0 conn(0x7f91d4165b90 0x7f91d4167fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:16.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.944+0000 7f91d37fe640 1 -- 172.21.15.94:0/2911434885 >> v1:172.21.15.94:6789/0 conn(0x7f91d4070080 legacy=0x7f91d415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:16.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.944+0000 7f91d37fe640 1 -- 172.21.15.94:0/2911434885 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d406fc60 con 0x7f91d415e990 2024-04-25T16:51:16.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91d37fe640 1 --2- 172.21.15.94:0/2911434885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d415e990 0x7f91d4165650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f91d4071c10 tx=0x7f91c801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a376c13f055e383 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:16.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91d2ffd640 1 -- 172.21.15.94:0/2911434885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c80aa660 con 0x7f91d415e990 2024-04-25T16:51:16.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91d2ffd640 1 -- 172.21.15.94:0/2911434885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c80af070 con 0x7f91d415e990 2024-04-25T16:51:16.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91d2ffd640 1 -- 172.21.15.94:0/2911434885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c80aaa20 con 0x7f91d415e990 2024-04-25T16:51:16.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91da492640 1 -- 172.21.15.94:0/2911434885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d415e990 msgr2=0x7f91d4165650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:16.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91da492640 1 --2- 172.21.15.94:0/2911434885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d415e990 0x7f91d4165650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f91d4071c10 tx=0x7f91c801c500 comp rx=0 tx=0).stop 2024-04-25T16:51:16.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91da492640 1 -- 172.21.15.94:0/2911434885 shutdown_connections 2024-04-25T16:51:16.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91da492640 1 --2- 172.21.15.94:0/2911434885 >> v2:172.21.15.138:3300/0 conn(0x7f91d4165b90 0x7f91d4167fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:16.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91da492640 1 --2- 172.21.15.94:0/2911434885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d415e990 0x7f91d4165650 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:16.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91da492640 1 -- 172.21.15.94:0/2911434885 >> 172.21.15.94:0/2911434885 conn(0x7f91d40b47f0 msgr2=0x7f91d40b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:16.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91da492640 1 -- 172.21.15.94:0/2911434885 shutdown_connections 2024-04-25T16:51:16.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91da492640 1 -- 172.21.15.94:0/2911434885 wait complete. 2024-04-25T16:51:16.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91da492640 1 Processor -- start 2024-04-25T16:51:16.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.948+0000 7f91da492640 1 -- start start 2024-04-25T16:51:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91da492640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d4070080 0x7f91d4167730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91da492640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d415e990 0x7f91d414ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91da492640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d4165b90 0x7f91d41497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91da492640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91d40ae6c0 con 0x7f91d4070080 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91da492640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91d406f460 con 0x7f91d4165b90 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91da492640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f91d4070490 con 0x7f91d415e990 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d4070080 0x7f91d4167730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d4070080 0x7f91d4167730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36088/0 (socket says 172.21.15.94:36088) 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d3fff640 1 -- 172.21.15.94:0/88148320 learned_addr learned my addr 172.21.15.94:0/88148320 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d8a2f640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d4165b90 0x7f91d41497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d3fff640 1 -- 172.21.15.94:0/88148320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d415e990 msgr2=0x7f91d414ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d37fe640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d415e990 0x7f91d414ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d3fff640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d415e990 0x7f91d414ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d3fff640 1 -- 172.21.15.94:0/88148320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d4165b90 msgr2=0x7f91d41497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d3fff640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d4165b90 0x7f91d41497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d3fff640 1 -- 172.21.15.94:0/88148320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d4167c70 con 0x7f91d4070080 2024-04-25T16:51:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d3fff640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d4070080 0x7f91d4167730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f91c400c370 tx=0x7f91c400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d0ff9640 1 -- 172.21.15.94:0/88148320 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c40052d0 con 0x7f91d4070080 2024-04-25T16:51:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d0ff9640 1 -- 172.21.15.94:0/88148320 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c4005470 con 0x7f91d4070080 2024-04-25T16:51:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d0ff9640 1 -- 172.21.15.94:0/88148320 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4007260 con 0x7f91d4070080 2024-04-25T16:51:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91da492640 1 -- 172.21.15.94:0/88148320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d4168180 con 0x7f91d4070080 2024-04-25T16:51:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91da492640 1 -- 172.21.15.94:0/88148320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d414a540 con 0x7f91d4070080 2024-04-25T16:51:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91da492640 1 -- 172.21.15.94:0/88148320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d4076830 con 0x7f91d4070080 2024-04-25T16:51:16.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.952+0000 7f91d0ff9640 1 -- 172.21.15.94:0/88148320 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f91c4007400 con 0x7f91d4070080 2024-04-25T16:51:16.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.956+0000 7f91d0ff9640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f91c006f430 0x7f91c0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:16.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.956+0000 7f91d0ff9640 1 -- 172.21.15.94:0/88148320 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f91c408ea50 con 0x7f91d4070080 2024-04-25T16:51:16.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.956+0000 7f91d37fe640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f91c006f430 0x7f91c0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:16.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.956+0000 7f91d37fe640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f91c006f430 0x7f91c0071940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91c80ae000 tx=0x7f91c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:16.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:16.960+0000 7f91d0ff9640 1 -- 172.21.15.94:0/88148320 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f91d406d610 con 0x7f91d4070080 2024-04-25T16:51:17.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.120+0000 7f91da492640 1 -- 172.21.15.94:0/88148320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91d40cd420 con 0x7f91d4070080 2024-04-25T16:51:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.124+0000 7f91d0ff9640 1 -- 172.21.15.94:0/88148320 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f91c405b690 con 0x7f91d4070080 2024-04-25T16:51:17.121 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:17.121 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:59.528743+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.124+0000 7f91da492640 1 -- 172.21.15.94:0/88148320 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f91c006f430 msgr2=0x7f91c0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.124+0000 7f91da492640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f91c006f430 0x7f91c0071940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91c80ae000 tx=0x7f91c8008040 comp rx=0 tx=0).stop 2024-04-25T16:51:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.124+0000 7f91da492640 1 -- 172.21.15.94:0/88148320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d4070080 msgr2=0x7f91d4167730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.124+0000 7f91da492640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d4070080 0x7f91d4167730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f91c400c370 tx=0x7f91c400c830 comp rx=0 tx=0).stop 2024-04-25T16:51:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.124+0000 7f91da492640 1 -- 172.21.15.94:0/88148320 shutdown_connections 2024-04-25T16:51:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.124+0000 7f91da492640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d4165b90 0x7f91d41497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.124+0000 7f91da492640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f91c006f430 0x7f91c0071940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.124+0000 7f91da492640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d415e990 0x7f91d414ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.124+0000 7f91da492640 1 --2- 172.21.15.94:0/88148320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d4070080 0x7f91d4167730 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.124+0000 7f91da492640 1 -- 172.21.15.94:0/88148320 >> 172.21.15.94:0/88148320 conn(0x7f91d40b47f0 msgr2=0x7f91d415e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.124+0000 7f91da492640 1 -- 172.21.15.94:0/88148320 shutdown_connections 2024-04-25T16:51:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.124+0000 7f91da492640 1 -- 172.21.15.94:0/88148320 wait complete. 2024-04-25T16:51:17.142 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T16:51:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f96860a5640 1 Processor -- start 2024-04-25T16:51:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f96860a5640 1 -- start start 2024-04-25T16:51:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f96860a5640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f968015fb00 0x7f968015ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f96860a5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96800aedd0 0x7f96800af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f96860a5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f968006f3d0 0x7f968006f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f96860a5640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f968006c6a0 con 0x7f968015fb00 2024-04-25T16:51:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f96860a5640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f968015c410 con 0x7f968006f3d0 2024-04-25T16:51:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f96860a5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9680160450 con 0x7f96800aedd0 2024-04-25T16:51:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f967f7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f968015fb00 0x7f968015ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f967f7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f968015fb00 0x7f968015ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36092/0 (socket says 172.21.15.94:36092) 2024-04-25T16:51:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f967f7fe640 1 -- 172.21.15.94:0/4097702562 learned_addr learned my addr 172.21.15.94:0/4097702562 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f967ffff640 1 --2- 172.21.15.94:0/4097702562 >> v2:172.21.15.138:3300/0 conn(0x7f968006f3d0 0x7f968006f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f9677fff640 1 --2- 172.21.15.94:0/4097702562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96800aedd0 0x7f96800af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f967f7fe640 1 -- 172.21.15.94:0/4097702562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96800aedd0 msgr2=0x7f96800af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f967f7fe640 1 --2- 172.21.15.94:0/4097702562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96800aedd0 0x7f96800af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f967f7fe640 1 -- 172.21.15.94:0/4097702562 >> v2:172.21.15.138:3300/0 conn(0x7f968006f3d0 msgr2=0x7f968006f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f967f7fe640 1 --2- 172.21.15.94:0/4097702562 >> v2:172.21.15.138:3300/0 conn(0x7f968006f3d0 0x7f968006f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.284+0000 7f967f7fe640 1 -- 172.21.15.94:0/4097702562 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96800afb00 con 0x7f968015fb00 2024-04-25T16:51:17.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f967f7fe640 1 --2- 172.21.15.94:0/4097702562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f968015fb00 0x7f968015ff10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9680072510 tx=0x7f967001d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257a646976c3acbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f967effd640 1 -- 172.21.15.94:0/4097702562 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96700b3070 con 0x7f968015fb00 2024-04-25T16:51:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f967effd640 1 -- 172.21.15.94:0/4097702562 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96700a68a0 con 0x7f968015fb00 2024-04-25T16:51:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f967effd640 1 -- 172.21.15.94:0/4097702562 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967000b040 con 0x7f968015fb00 2024-04-25T16:51:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f96860a5640 1 -- 172.21.15.94:0/4097702562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f968015fb00 msgr2=0x7f968015ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f96860a5640 1 --2- 172.21.15.94:0/4097702562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f968015fb00 0x7f968015ff10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9680072510 tx=0x7f967001d830 comp rx=0 tx=0).stop 2024-04-25T16:51:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f96860a5640 1 -- 172.21.15.94:0/4097702562 shutdown_connections 2024-04-25T16:51:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f96860a5640 1 --2- 172.21.15.94:0/4097702562 >> v2:172.21.15.138:3300/0 conn(0x7f968006f3d0 0x7f968006f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f96860a5640 1 --2- 172.21.15.94:0/4097702562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96800aedd0 0x7f96800af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f96860a5640 1 --2- 172.21.15.94:0/4097702562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f968015fb00 0x7f968015ff10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f96860a5640 1 -- 172.21.15.94:0/4097702562 >> 172.21.15.94:0/4097702562 conn(0x7f96800b47f0 msgr2=0x7f96800b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f96860a5640 1 -- 172.21.15.94:0/4097702562 shutdown_connections 2024-04-25T16:51:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f96860a5640 1 -- 172.21.15.94:0/4097702562 wait complete. 2024-04-25T16:51:17.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f96860a5640 1 Processor -- start 2024-04-25T16:51:17.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.288+0000 7f96860a5640 1 -- start start 2024-04-25T16:51:17.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f96860a5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f968006f3d0 0x7f968015eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f96860a5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96800aedd0 0x7f9680154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f96860a5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f968015fb00 0x7f9680154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f96860a5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f96800af720 con 0x7f968015fb00 2024-04-25T16:51:17.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f96860a5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9680070130 con 0x7f96800aedd0 2024-04-25T16:51:17.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f96860a5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f96800af8a0 con 0x7f968006f3d0 2024-04-25T16:51:17.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f968015fb00 0x7f9680154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f968015fb00 0x7f9680154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36096/0 (socket says 172.21.15.94:36096) 2024-04-25T16:51:17.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967ffff640 1 -- 172.21.15.94:0/3381385974 learned_addr learned my addr 172.21.15.94:0/3381385974 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:17.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f9677fff640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96800aedd0 0x7f9680154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967f7fe640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f968006f3d0 0x7f968015eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967ffff640 1 -- 172.21.15.94:0/3381385974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f968006f3d0 msgr2=0x7f968015eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:17.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967ffff640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f968006f3d0 0x7f968015eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967ffff640 1 -- 172.21.15.94:0/3381385974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96800aedd0 msgr2=0x7f9680154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967ffff640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96800aedd0 0x7f9680154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967ffff640 1 -- 172.21.15.94:0/3381385974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968015f340 con 0x7f968015fb00 2024-04-25T16:51:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f9677fff640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96800aedd0 0x7f9680154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967ffff640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f968015fb00 0x7f9680154ca0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9678007d40 tx=0x7f967800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967cff9640 1 -- 172.21.15.94:0/3381385974 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678014420 con 0x7f968015fb00 2024-04-25T16:51:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967cff9640 1 -- 172.21.15.94:0/3381385974 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9678019070 con 0x7f968015fb00 2024-04-25T16:51:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f96860a5640 1 -- 172.21.15.94:0/3381385974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9680145a10 con 0x7f968015fb00 2024-04-25T16:51:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967cff9640 1 -- 172.21.15.94:0/3381385974 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678014820 con 0x7f968015fb00 2024-04-25T16:51:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f96860a5640 1 -- 172.21.15.94:0/3381385974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9680145fd0 con 0x7f968015fb00 2024-04-25T16:51:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f96860a5640 1 -- 172.21.15.94:0/3381385974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9680076830 con 0x7f968015fb00 2024-04-25T16:51:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967cff9640 1 -- 172.21.15.94:0/3381385974 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f96780149c0 con 0x7f968015fb00 2024-04-25T16:51:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.292+0000 7f967cff9640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f966006f430 0x7f9660071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.296+0000 7f967f7fe640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f966006f430 0x7f9660071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.296+0000 7f967cff9640 1 -- 172.21.15.94:0/3381385974 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f967808fc10 con 0x7f968015fb00 2024-04-25T16:51:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.296+0000 7f967f7fe640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f966006f430 0x7f9660071940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f96700b0000 tx=0x7f96700a8070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.300+0000 7f967cff9640 1 -- 172.21.15.94:0/3381385974 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f967805c8e0 con 0x7f968015fb00 2024-04-25T16:51:17.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.496+0000 7f96860a5640 1 -- 172.21.15.94:0/3381385974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f968006f1b0 con 0x7f968015fb00 2024-04-25T16:51:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.756+0000 7f967cff9640 1 -- 172.21.15.94:0/3381385974 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f967805c1e0 con 0x7f968015fb00 2024-04-25T16:51:17.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.760+0000 7f96860a5640 1 -- 172.21.15.94:0/3381385974 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f966006f430 msgr2=0x7f9660071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:17.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.760+0000 7f96860a5640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f966006f430 0x7f9660071940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f96700b0000 tx=0x7f96700a8070 comp rx=0 tx=0).stop 2024-04-25T16:51:17.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.760+0000 7f96860a5640 1 -- 172.21.15.94:0/3381385974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f968015fb00 msgr2=0x7f9680154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:17.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.760+0000 7f96860a5640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f968015fb00 0x7f9680154ca0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9678007d40 tx=0x7f967800e4b0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.760+0000 7f96860a5640 1 -- 172.21.15.94:0/3381385974 shutdown_connections 2024-04-25T16:51:17.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.760+0000 7f96860a5640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f968015fb00 0x7f9680154ca0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.760+0000 7f96860a5640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96800aedd0 0x7f9680154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.760+0000 7f96860a5640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f966006f430 0x7f9660071940 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.760+0000 7f96860a5640 1 --2- 172.21.15.94:0/3381385974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f968006f3d0 0x7f968015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.760+0000 7f96860a5640 1 -- 172.21.15.94:0/3381385974 >> 172.21.15.94:0/3381385974 conn(0x7f96800b47f0 msgr2=0x7f9680151180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:17.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.760+0000 7f96860a5640 1 -- 172.21.15.94:0/3381385974 shutdown_connections 2024-04-25T16:51:17.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.760+0000 7f96860a5640 1 -- 172.21.15.94:0/3381385974 wait complete. 2024-04-25T16:51:17.781 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T16:51:17.781 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T16:51:17.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.932+0000 7f10c45b9640 1 Processor -- start 2024-04-25T16:51:17.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.932+0000 7f10c45b9640 1 -- start start 2024-04-25T16:51:17.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.932+0000 7f10c45b9640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10bc0b0420 0x7f10bc06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c45b9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10bc070630 con 0x7f10bc15ec30 2024-04-25T16:51:17.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c45b9640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10bc06f6b0 con 0x7f10bc06f2a0 2024-04-25T16:51:17.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c45b9640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f10bc0aeaa0 con 0x7f10bc0b0420 2024-04-25T16:51:17.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c2b56640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f10bc15ec30 0x7f10bc06cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38974/0 (socket says 172.21.15.94:38974) 2024-04-25T16:51:17.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c2b56640 1 -- 172.21.15.94:0/3535101157 learned_addr learned my addr 172.21.15.94:0/3535101157 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:17.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c2355640 1 --2- 172.21.15.94:0/3535101157 >> v2:172.21.15.138:3300/0 conn(0x7f10bc0b0420 0x7f10bc06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899090147 0 0) 0x7f10bc070630 con 0x7f10bc15ec30 2024-04-25T16:51:17.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10a8003530 con 0x7f10bc15ec30 2024-04-25T16:51:17.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3686003984 0 0) 0x7f10a8003530 con 0x7f10bc15ec30 2024-04-25T16:51:17.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 >> v2:172.21.15.138:3300/0 conn(0x7f10bc0b0420 msgr2=0x7f10bc06ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:17.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 --2- 172.21.15.94:0/3535101157 >> v2:172.21.15.138:3300/0 conn(0x7f10bc0b0420 0x7f10bc06ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 >> v1:172.21.15.138:6789/0 conn(0x7f10bc06f2a0 legacy=0x7f10bc06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:17.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc06d420 con 0x7f10bc15ec30 2024-04-25T16:51:17.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f10b4001a00 con 0x7f10bc15ec30 2024-04-25T16:51:17.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 >> v1:172.21.15.94:6789/0 conn(0x7f10bc15ec30 legacy=0x7f10bc06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:17.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 --2- 172.21.15.94:0/3535101157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10a80039c0 0x7f10a8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 --2- 172.21.15.94:0/3535101157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10a8004810 0x7f10a8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 --2- 172.21.15.94:0/3535101157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a80051d0 0x7f10a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc0aeaa0 con 0x7f10a8004810 2024-04-25T16:51:17.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10a800c170 con 0x7f10a80051d0 2024-04-25T16:51:17.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f10a800c370 con 0x7f10a80039c0 2024-04-25T16:51:17.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1b54640 1 --2- 172.21.15.94:0/3535101157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10a8004810 0x7f10a8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c2b56640 1 --2- 172.21.15.94:0/3535101157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a80051d0 0x7f10a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c2355640 1 --2- 172.21.15.94:0/3535101157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10a80039c0 0x7f10a8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1b54640 1 -- 172.21.15.94:0/3535101157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10a80039c0 msgr2=0x7f10a8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:17.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1b54640 1 --2- 172.21.15.94:0/3535101157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10a80039c0 0x7f10a8003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1b54640 1 -- 172.21.15.94:0/3535101157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a80051d0 msgr2=0x7f10a800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:17.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1b54640 1 --2- 172.21.15.94:0/3535101157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a80051d0 0x7f10a800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c2b56640 1 -- 172.21.15.94:0/3535101157 reap_dead start 2024-04-25T16:51:17.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1b54640 1 -- 172.21.15.94:0/3535101157 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a8003530 con 0x7f10a8004810 2024-04-25T16:51:17.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1b54640 1 --2- 172.21.15.94:0/3535101157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10a8004810 0x7f10a8004c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f10ac001780 tx=0x7f10ac01c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55d7642008b606d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:17.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0013a0 con 0x7f10a8004810 2024-04-25T16:51:17.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10ac0052f0 con 0x7f10a8004810 2024-04-25T16:51:17.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.936+0000 7f10c1353640 1 -- 172.21.15.94:0/3535101157 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac003840 con 0x7f10a8004810 2024-04-25T16:51:17.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 -- 172.21.15.94:0/3535101157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10a8004810 msgr2=0x7f10a8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:17.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 --2- 172.21.15.94:0/3535101157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10a8004810 0x7f10a8004c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f10ac001780 tx=0x7f10ac01c220 comp rx=0 tx=0).stop 2024-04-25T16:51:17.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 -- 172.21.15.94:0/3535101157 shutdown_connections 2024-04-25T16:51:17.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 --2- 172.21.15.94:0/3535101157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10a8004810 0x7f10a8004c20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 -- 172.21.15.94:0/3535101157 >> 172.21.15.94:0/3535101157 conn(0x7f10bc0b47f0 msgr2=0x7f10bc0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:17.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 -- 172.21.15.94:0/3535101157 shutdown_connections 2024-04-25T16:51:17.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 -- 172.21.15.94:0/3535101157 wait complete. 2024-04-25T16:51:17.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 Processor -- start 2024-04-25T16:51:17.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 -- start start 2024-04-25T16:51:17.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a8004810 0x7f10bc154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc1420d0 0x7f10bc144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10bc144670 0x7f10bc155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc0ae450 con 0x7f10bc1420d0 2024-04-25T16:51:17.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc0703e0 con 0x7f10a8004810 2024-04-25T16:51:17.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c45b9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f10bc0ae730 con 0x7f10bc144670 2024-04-25T16:51:17.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.940+0000 7f10c1b54640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc1420d0 0x7f10bc144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c1b54640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc1420d0 0x7f10bc144130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41884/0 (socket says 172.21.15.94:41884) 2024-04-25T16:51:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c1b54640 1 -- 172.21.15.94:0/940130738 learned_addr learned my addr 172.21.15.94:0/940130738 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c2355640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a8004810 0x7f10bc154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c2b56640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10bc144670 0x7f10bc155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c1b54640 1 -- 172.21.15.94:0/940130738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10bc144670 msgr2=0x7f10bc155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c1b54640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10bc144670 0x7f10bc155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c1b54640 1 -- 172.21.15.94:0/940130738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a8004810 msgr2=0x7f10bc154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c1b54640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a8004810 0x7f10bc154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c1b54640 1 -- 172.21.15.94:0/940130738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc144a80 con 0x7f10bc1420d0 2024-04-25T16:51:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c1b54640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc1420d0 0x7f10bc144130 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f10ac005040 tx=0x7f10a8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10b2ffd640 1 -- 172.21.15.94:0/940130738 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0013a0 con 0x7f10bc1420d0 2024-04-25T16:51:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10b2ffd640 1 -- 172.21.15.94:0/940130738 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10ac0ba070 con 0x7f10bc1420d0 2024-04-25T16:51:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c45b9640 1 -- 172.21.15.94:0/940130738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10bc154e40 con 0x7f10bc1420d0 2024-04-25T16:51:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10b2ffd640 1 -- 172.21.15.94:0/940130738 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac007a40 con 0x7f10bc1420d0 2024-04-25T16:51:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c45b9640 1 -- 172.21.15.94:0/940130738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10bc155400 con 0x7f10bc1420d0 2024-04-25T16:51:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c45b9640 1 -- 172.21.15.94:0/940130738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10bc076830 con 0x7f10bc1420d0 2024-04-25T16:51:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10b2ffd640 1 -- 172.21.15.94:0/940130738 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f10ac0052f0 con 0x7f10bc1420d0 2024-04-25T16:51:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10b2ffd640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f10a006f430 0x7f10a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10b2ffd640 1 -- 172.21.15.94:0/940130738 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f10ac129a30 con 0x7f10bc1420d0 2024-04-25T16:51:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c2355640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f10a006f430 0x7f10a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.944+0000 7f10c2355640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f10a006f430 0x7f10a0071940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0cd2f0 tx=0x7f10b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:17.948+0000 7f10b2ffd640 1 -- 172.21.15.94:0/940130738 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f10bc06f6b0 con 0x7f10bc1420d0 2024-04-25T16:51:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.136+0000 7f10c45b9640 1 -- 172.21.15.94:0/940130738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f10bc0cd0d0 con 0x7f10bc1420d0 2024-04-25T16:51:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.772+0000 7f10b2ffd640 1 -- 172.21.15.94:0/940130738 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f10ac0b3e70 con 0x7f10bc1420d0 2024-04-25T16:51:18.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.776+0000 7f10c45b9640 1 -- 172.21.15.94:0/940130738 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f10a006f430 msgr2=0x7f10a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:18.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.776+0000 7f10c45b9640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f10a006f430 0x7f10a0071940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0cd2f0 tx=0x7f10b8008040 comp rx=0 tx=0).stop 2024-04-25T16:51:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.776+0000 7f10c45b9640 1 -- 172.21.15.94:0/940130738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc1420d0 msgr2=0x7f10bc144130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.776+0000 7f10c45b9640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc1420d0 0x7f10bc144130 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f10ac005040 tx=0x7f10a8005840 comp rx=0 tx=0).stop 2024-04-25T16:51:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.776+0000 7f10c45b9640 1 -- 172.21.15.94:0/940130738 shutdown_connections 2024-04-25T16:51:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.776+0000 7f10c45b9640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f10a006f430 0x7f10a0071940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.776+0000 7f10c45b9640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10bc144670 0x7f10bc155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.776+0000 7f10c45b9640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10bc1420d0 0x7f10bc144130 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.776+0000 7f10c45b9640 1 --2- 172.21.15.94:0/940130738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a8004810 0x7f10bc154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.776+0000 7f10c45b9640 1 -- 172.21.15.94:0/940130738 >> 172.21.15.94:0/940130738 conn(0x7f10bc0b47f0 msgr2=0x7f10bc160880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.776+0000 7f10c45b9640 1 -- 172.21.15.94:0/940130738 shutdown_connections 2024-04-25T16:51:18.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.776+0000 7f10c45b9640 1 -- 172.21.15.94:0/940130738 wait complete. 2024-04-25T16:51:18.797 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T16:51:18.797 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:51:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.956+0000 7f9d43130640 1 Processor -- start 2024-04-25T16:51:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.956+0000 7f9d43130640 1 -- start start 2024-04-25T16:51:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.956+0000 7f9d43130640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9d3c070080 0x7f9d3c15e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.956+0000 7f9d43130640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d3c06fc60 con 0x7f9d3c15ebd0 2024-04-25T16:51:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.956+0000 7f9d43130640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d3c0aef70 con 0x7f9d3c15e7c0 2024-04-25T16:51:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.956+0000 7f9d43130640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f9d3c158550 con 0x7f9d3c070080 2024-04-25T16:51:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.956+0000 7f9d40ecc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9d3c070080 0x7f9d3c15e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.956+0000 7f9d40ecc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9d3c070080 0x7f9d3c15e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41894/0 (socket says 172.21.15.94:41894) 2024-04-25T16:51:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.956+0000 7f9d40ecc640 1 -- 172.21.15.94:0/3200470068 learned_addr learned my addr 172.21.15.94:0/3200470068 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.956+0000 7f9d40ecc640 1 -- 172.21.15.94:0/3200470068 >> v1:172.21.15.138:6789/0 conn(0x7f9d3c15e7c0 legacy=0x7f9d3c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.956+0000 7f9d40ecc640 1 -- 172.21.15.94:0/3200470068 >> v1:172.21.15.94:6789/0 conn(0x7f9d3c15ebd0 legacy=0x7f9d3c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.956+0000 7f9d40ecc640 1 -- 172.21.15.94:0/3200470068 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3c0ae100 con 0x7f9d3c070080 2024-04-25T16:51:18.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.960+0000 7f9d40ecc640 1 --2- 172.21.15.94:0/3200470068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d3c070080 0x7f9d3c15e280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009e50 tx=0x7f9d2c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a26baffe10e7f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:18.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.960+0000 7f9d3b7fe640 1 -- 172.21.15.94:0/3200470068 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0a57a0 con 0x7f9d3c070080 2024-04-25T16:51:18.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.960+0000 7f9d3b7fe640 1 -- 172.21.15.94:0/3200470068 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d2c0a5940 con 0x7f9d3c070080 2024-04-25T16:51:18.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.960+0000 7f9d3b7fe640 1 -- 172.21.15.94:0/3200470068 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0a5d10 con 0x7f9d3c070080 2024-04-25T16:51:18.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.960+0000 7f9d43130640 1 -- 172.21.15.94:0/3200470068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d3c070080 msgr2=0x7f9d3c15e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:18.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.960+0000 7f9d43130640 1 --2- 172.21.15.94:0/3200470068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d3c070080 0x7f9d3c15e280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009e50 tx=0x7f9d2c01c9f0 comp rx=0 tx=0).stop 2024-04-25T16:51:18.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.960+0000 7f9d43130640 1 -- 172.21.15.94:0/3200470068 shutdown_connections 2024-04-25T16:51:18.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.960+0000 7f9d43130640 1 --2- 172.21.15.94:0/3200470068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d3c070080 0x7f9d3c15e280 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:18.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.960+0000 7f9d43130640 1 -- 172.21.15.94:0/3200470068 >> 172.21.15.94:0/3200470068 conn(0x7f9d3c0b47f0 msgr2=0x7f9d3c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:18.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.960+0000 7f9d43130640 1 -- 172.21.15.94:0/3200470068 shutdown_connections 2024-04-25T16:51:18.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.960+0000 7f9d43130640 1 -- 172.21.15.94:0/3200470068 wait complete. 2024-04-25T16:51:18.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.960+0000 7f9d43130640 1 Processor -- start 2024-04-25T16:51:18.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d43130640 1 -- start start 2024-04-25T16:51:18.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d43130640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d3c070080 0x7f9d3c09a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:18.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d43130640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d3c15e7c0 0x7f9d3c15cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:18.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d43130640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d3c15ebd0 0x7f9d3c15d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:18.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d43130640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c06d420 con 0x7f9d3c070080 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d43130640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c070490 con 0x7f9d3c15e7c0 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d43130640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3c06d5a0 con 0x7f9d3c15ebd0 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d40ecc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d3c070080 0x7f9d3c09a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d40ecc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d3c070080 0x7f9d3c09a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41906/0 (socket says 172.21.15.94:41906) 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d40ecc640 1 -- 172.21.15.94:0/1850824524 learned_addr learned my addr 172.21.15.94:0/1850824524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d3bfff640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d3c15e7c0 0x7f9d3c15cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d416cd640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d3c15ebd0 0x7f9d3c15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d40ecc640 1 -- 172.21.15.94:0/1850824524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d3c15ebd0 msgr2=0x7f9d3c15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d40ecc640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d3c15ebd0 0x7f9d3c15d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d40ecc640 1 -- 172.21.15.94:0/1850824524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d3c15e7c0 msgr2=0x7f9d3c15cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d40ecc640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d3c15e7c0 0x7f9d3c15cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d40ecc640 1 -- 172.21.15.94:0/1850824524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3c0cdff0 con 0x7f9d3c070080 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d416cd640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d3c15ebd0 0x7f9d3c15d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d40ecc640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d3c070080 0x7f9d3c09a7f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c019600 tx=0x7f9d2c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:18.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d397fa640 1 -- 172.21.15.94:0/1850824524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0bc020 con 0x7f9d3c070080 2024-04-25T16:51:18.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d397fa640 1 -- 172.21.15.94:0/1850824524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d2c0b8670 con 0x7f9d3c070080 2024-04-25T16:51:18.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d397fa640 1 -- 172.21.15.94:0/1850824524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0b8a60 con 0x7f9d3c070080 2024-04-25T16:51:18.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d43130640 1 -- 172.21.15.94:0/1850824524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3c0d0600 con 0x7f9d3c070080 2024-04-25T16:51:18.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d43130640 1 -- 172.21.15.94:0/1850824524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d3c0d0bc0 con 0x7f9d3c070080 2024-04-25T16:51:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.964+0000 7f9d43130640 1 -- 172.21.15.94:0/1850824524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d3c076830 con 0x7f9d3c070080 2024-04-25T16:51:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.968+0000 7f9d397fa640 1 -- 172.21.15.94:0/1850824524 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9d2c0bf020 con 0x7f9d3c070080 2024-04-25T16:51:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.968+0000 7f9d397fa640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9d1c06f400 0x7f9d1c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.968+0000 7f9d397fa640 1 -- 172.21.15.94:0/1850824524 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9d2c12a760 con 0x7f9d3c070080 2024-04-25T16:51:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.968+0000 7f9d3bfff640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9d1c06f400 0x7f9d1c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.968+0000 7f9d3bfff640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9d1c06f400 0x7f9d1c071910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c1fc6d0 tx=0x7f9d24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:18.972+0000 7f9d397fa640 1 -- 172.21.15.94:0/1850824524 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9d2c0c6030 con 0x7f9d3c070080 2024-04-25T16:51:19.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d43130640 1 -- 172.21.15.94:0/1850824524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d3c0686a0 con 0x7f9d3c070080 2024-04-25T16:51:19.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d397fa640 1 -- 172.21.15.94:0/1850824524 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f9d2c0165e0 con 0x7f9d3c070080 2024-04-25T16:51:19.169 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:19.170 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2352186041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2352186041},{"type":"v1","addr":"172.21.15.138:6833","nonce":2352186041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3210927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3210927873},{"type":"v1","addr":"172.21.15.94:6835","nonce":3210927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2237413225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2237413225},{"type":"v1","addr":"172.21.15.94:6837","nonce":2237413225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:50:54.403299+0000","modified":"2024-04-25T16:51:18.139471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/3652545132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3652545132},{"type":"v1","addr":"172.21.15.138:6835","nonce":3652545132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T16:51:19.170 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:51:19.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d43130640 1 -- 172.21.15.94:0/1850824524 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9d1c06f400 msgr2=0x7f9d1c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d43130640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9d1c06f400 0x7f9d1c071910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c1fc6d0 tx=0x7f9d24008040 comp rx=0 tx=0).stop 2024-04-25T16:51:19.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d43130640 1 -- 172.21.15.94:0/1850824524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d3c070080 msgr2=0x7f9d3c09a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d43130640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d3c070080 0x7f9d3c09a7f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c019600 tx=0x7f9d2c019790 comp rx=0 tx=0).stop 2024-04-25T16:51:19.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d43130640 1 -- 172.21.15.94:0/1850824524 shutdown_connections 2024-04-25T16:51:19.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d43130640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9d1c06f400 0x7f9d1c071910 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d43130640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d3c15ebd0 0x7f9d3c15d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d43130640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d3c15e7c0 0x7f9d3c15cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d43130640 1 --2- 172.21.15.94:0/1850824524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d3c070080 0x7f9d3c09a7f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d43130640 1 -- 172.21.15.94:0/1850824524 >> 172.21.15.94:0/1850824524 conn(0x7f9d3c0b47f0 msgr2=0x7f9d3c0b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:19.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d43130640 1 -- 172.21.15.94:0/1850824524 shutdown_connections 2024-04-25T16:51:19.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.172+0000 7f9d43130640 1 -- 172.21.15.94:0/1850824524 wait complete. 2024-04-25T16:51:19.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:19.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.340+0000 7f2932a6d640 1 Processor -- start 2024-04-25T16:51:19.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.340+0000 7f2932a6d640 1 -- start start 2024-04-25T16:51:19.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.344+0000 7f2932a6d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f292c165060 0x7f292c167480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.344+0000 7f2932a6d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f292c156ba0 con 0x7f292c06fbe0 2024-04-25T16:51:19.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.344+0000 7f2932a6d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f292c15fd80 con 0x7f292c156790 2024-04-25T16:51:19.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.344+0000 7f2932a6d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f292c160060 con 0x7f292c165060 2024-04-25T16:51:19.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.344+0000 7f293100a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f292c165060 0x7f292c167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.344+0000 7f293100a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f292c165060 0x7f292c167480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41922/0 (socket says 172.21.15.94:41922) 2024-04-25T16:51:19.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.344+0000 7f293100a640 1 -- 172.21.15.94:0/1044990055 learned_addr learned my addr 172.21.15.94:0/1044990055 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:19.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.344+0000 7f293100a640 1 -- 172.21.15.94:0/1044990055 >> v1:172.21.15.138:6789/0 conn(0x7f292c156790 legacy=0x7f292c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:19.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.344+0000 7f293100a640 1 -- 172.21.15.94:0/1044990055 >> v1:172.21.15.94:6789/0 conn(0x7f292c06fbe0 legacy=0x7f292c06fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.344+0000 7f293100a640 1 -- 172.21.15.94:0/1044990055 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c160340 con 0x7f292c165060 2024-04-25T16:51:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.344+0000 7f293100a640 1 --2- 172.21.15.94:0/1044990055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f292c165060 0x7f292c167480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2924009e50 tx=0x7f292401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ab3ce1a5aab853 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.348+0000 7f292b7fe640 1 -- 172.21.15.94:0/1044990055 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29240a55a0 con 0x7f292c165060 2024-04-25T16:51:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.348+0000 7f292b7fe640 1 -- 172.21.15.94:0/1044990055 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29240a5740 con 0x7f292c165060 2024-04-25T16:51:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.348+0000 7f292b7fe640 1 -- 172.21.15.94:0/1044990055 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29240a5b10 con 0x7f292c165060 2024-04-25T16:51:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.348+0000 7f2932a6d640 1 -- 172.21.15.94:0/1044990055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f292c165060 msgr2=0x7f292c167480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.348+0000 7f2932a6d640 1 --2- 172.21.15.94:0/1044990055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f292c165060 0x7f292c167480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2924009e50 tx=0x7f292401c960 comp rx=0 tx=0).stop 2024-04-25T16:51:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.348+0000 7f2932a6d640 1 -- 172.21.15.94:0/1044990055 shutdown_connections 2024-04-25T16:51:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.348+0000 7f2932a6d640 1 --2- 172.21.15.94:0/1044990055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f292c165060 0x7f292c167480 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.348+0000 7f2932a6d640 1 -- 172.21.15.94:0/1044990055 >> 172.21.15.94:0/1044990055 conn(0x7f292c0b47f0 msgr2=0x7f292c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.348+0000 7f2932a6d640 1 -- 172.21.15.94:0/1044990055 shutdown_connections 2024-04-25T16:51:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.348+0000 7f2932a6d640 1 -- 172.21.15.94:0/1044990055 wait complete. 2024-04-25T16:51:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.348+0000 7f2932a6d640 1 Processor -- start 2024-04-25T16:51:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.348+0000 7f2932a6d640 1 -- start start 2024-04-25T16:51:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f2932a6d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f292c06fbe0 0x7f292c165640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f2932a6d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f292c156790 0x7f292c14b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f2932a6d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f292c165060 0x7f292c14bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f2932a6d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f292c169b50 con 0x7f292c165060 2024-04-25T16:51:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f2932a6d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f292c1699d0 con 0x7f292c156790 2024-04-25T16:51:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f2932a6d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f292c169cd0 con 0x7f292c06fbe0 2024-04-25T16:51:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f293100a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f292c165060 0x7f292c14bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f293100a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f292c165060 0x7f292c14bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41926/0 (socket says 172.21.15.94:41926) 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f293100a640 1 -- 172.21.15.94:0/3283367962 learned_addr learned my addr 172.21.15.94:0/3283367962 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f2930809640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f292c06fbe0 0x7f292c165640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f292bfff640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f292c156790 0x7f292c14b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f293100a640 1 -- 172.21.15.94:0/3283367962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f292c06fbe0 msgr2=0x7f292c165640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f293100a640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f292c06fbe0 0x7f292c165640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f293100a640 1 -- 172.21.15.94:0/3283367962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f292c156790 msgr2=0x7f292c14b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f293100a640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f292c156790 0x7f292c14b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f293100a640 1 -- 172.21.15.94:0/3283367962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c166b50 con 0x7f292c165060 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f292bfff640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f292c156790 0x7f292c14b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f293100a640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f292c165060 0x7f292c14bea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2924009e50 tx=0x7f2924019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f29297fa640 1 -- 172.21.15.94:0/3283367962 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924009510 con 0x7f292c165060 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f29297fa640 1 -- 172.21.15.94:0/3283367962 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29240be070 con 0x7f292c165060 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f29297fa640 1 -- 172.21.15.94:0/3283367962 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29240b88d0 con 0x7f292c165060 2024-04-25T16:51:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f2932a6d640 1 -- 172.21.15.94:0/3283367962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292c1517b0 con 0x7f292c165060 2024-04-25T16:51:19.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f2932a6d640 1 -- 172.21.15.94:0/3283367962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f292c1557d0 con 0x7f292c165060 2024-04-25T16:51:19.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f2932a6d640 1 -- 172.21.15.94:0/3283367962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f292c076830 con 0x7f292c165060 2024-04-25T16:51:19.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f29297fa640 1 -- 172.21.15.94:0/3283367962 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f29240b8a70 con 0x7f292c165060 2024-04-25T16:51:19.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f29297fa640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f291806f400 0x7f2918071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f29297fa640 1 -- 172.21.15.94:0/3283367962 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f292412a640 con 0x7f292c165060 2024-04-25T16:51:19.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f2930809640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f291806f400 0x7f2918071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.352+0000 7f2930809640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f291806f400 0x7f2918071910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f292c0b06d0 tx=0x7f291c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:19.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.360+0000 7f29297fa640 1 -- 172.21.15.94:0/3283367962 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f29240165e0 con 0x7f292c165060 2024-04-25T16:51:19.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.520+0000 7f2932a6d640 1 -- 172.21.15.94:0/3283367962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f292c074440 con 0x7f292c165060 2024-04-25T16:51:19.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.524+0000 7f29297fa640 1 -- 172.21.15.94:0/3283367962 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f29240f71f0 con 0x7f292c165060 2024-04-25T16:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:19.521 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:59.528743+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:19.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.524+0000 7f2932a6d640 1 -- 172.21.15.94:0/3283367962 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f291806f400 msgr2=0x7f2918071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.524+0000 7f2932a6d640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f291806f400 0x7f2918071910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f292c0b06d0 tx=0x7f291c008040 comp rx=0 tx=0).stop 2024-04-25T16:51:19.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.524+0000 7f2932a6d640 1 -- 172.21.15.94:0/3283367962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f292c165060 msgr2=0x7f292c14bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.524+0000 7f2932a6d640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f292c165060 0x7f292c14bea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2924009e50 tx=0x7f2924019700 comp rx=0 tx=0).stop 2024-04-25T16:51:19.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.524+0000 7f2932a6d640 1 -- 172.21.15.94:0/3283367962 shutdown_connections 2024-04-25T16:51:19.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.524+0000 7f2932a6d640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f292c165060 0x7f292c14bea0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.524+0000 7f2932a6d640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f292c156790 0x7f292c14b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.524+0000 7f2932a6d640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f291806f400 0x7f2918071910 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.524+0000 7f2932a6d640 1 --2- 172.21.15.94:0/3283367962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f292c06fbe0 0x7f292c165640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.524+0000 7f2932a6d640 1 -- 172.21.15.94:0/3283367962 >> 172.21.15.94:0/3283367962 conn(0x7f292c0b47f0 msgr2=0x7f292c0b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:19.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.524+0000 7f2932a6d640 1 -- 172.21.15.94:0/3283367962 shutdown_connections 2024-04-25T16:51:19.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.524+0000 7f2932a6d640 1 -- 172.21.15.94:0/3283367962 wait complete. 2024-04-25T16:51:19.542 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T16:51:19.542 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:51:19.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.696+0000 7feb6c023640 1 Processor -- start 2024-04-25T16:51:19.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.696+0000 7feb6c023640 1 -- start start 2024-04-25T16:51:19.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.696+0000 7feb6c023640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb640d2740 0x7feb640b6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.696+0000 7feb6c023640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feb640cdeb0 0x7feb640b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.696+0000 7feb6c023640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb640cec90 con 0x7feb640cdaa0 2024-04-25T16:51:19.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.696+0000 7feb6c023640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feb640ce3f0 con 0x7feb640cdeb0 2024-04-25T16:51:19.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.696+0000 7feb6c023640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feb640ce6d0 con 0x7feb640d2740 2024-04-25T16:51:19.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.696+0000 7feb69dbf640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7feb640cdaa0 0x7feb640d2030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38986/0 (socket says 172.21.15.94:38986) 2024-04-25T16:51:19.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.696+0000 7feb69dbf640 1 -- 172.21.15.94:0/1696117240 learned_addr learned my addr 172.21.15.94:0/1696117240 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:19.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.696+0000 7feb6a5c0640 1 --2- 172.21.15.94:0/1696117240 >> v2:172.21.15.138:3300/0 conn(0x7feb640cdeb0 0x7feb640b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.696+0000 7feb695be640 1 --2- 172.21.15.94:0/1696117240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb640d2740 0x7feb640b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.696+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3991125989 0 0) 0x7feb640cec90 con 0x7feb640cdaa0 2024-04-25T16:51:19.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.696+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb50003530 con 0x7feb640cdaa0 2024-04-25T16:51:19.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4209635203 0 0) 0x7feb50003530 con 0x7feb640cdaa0 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb640d2740 msgr2=0x7feb640b6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 --2- 172.21.15.94:0/1696117240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb640d2740 0x7feb640b6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 >> v2:172.21.15.138:3300/0 conn(0x7feb640cdeb0 msgr2=0x7feb640b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 --2- 172.21.15.94:0/1696117240 >> v2:172.21.15.138:3300/0 conn(0x7feb640cdeb0 0x7feb640b6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb640ce9b0 con 0x7feb640cdaa0 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7feb60001680 con 0x7feb640cdaa0 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 >> v1:172.21.15.94:6789/0 conn(0x7feb640cdaa0 legacy=0x7feb640d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 --2- 172.21.15.94:0/1696117240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb500039c0 0x7feb50003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 --2- 172.21.15.94:0/1696117240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb50004810 0x7feb50004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 --2- 172.21.15.94:0/1696117240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb50005160 0x7feb5000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb640ce3f0 con 0x7feb500039c0 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb640ce6d0 con 0x7feb50004810 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feb5000bf10 con 0x7feb50005160 2024-04-25T16:51:19.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb69dbf640 1 --2- 172.21.15.94:0/1696117240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb500039c0 0x7feb50003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb695be640 1 --2- 172.21.15.94:0/1696117240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb50004810 0x7feb50004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb6a5c0640 1 --2- 172.21.15.94:0/1696117240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb50005160 0x7feb5000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb69dbf640 1 -- 172.21.15.94:0/1696117240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb50005160 msgr2=0x7feb5000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:19.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb69dbf640 1 --2- 172.21.15.94:0/1696117240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb50005160 0x7feb5000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb69dbf640 1 -- 172.21.15.94:0/1696117240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb50004810 msgr2=0x7feb50004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:19.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb69dbf640 1 --2- 172.21.15.94:0/1696117240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb50004810 0x7feb50004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb69dbf640 1 -- 172.21.15.94:0/1696117240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb50003530 con 0x7feb500039c0 2024-04-25T16:51:19.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb695be640 1 --2- 172.21.15.94:0/1696117240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb50004810 0x7feb50004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:19.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb69dbf640 1 --2- 172.21.15.94:0/1696117240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb500039c0 0x7feb50003e30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feb60001060 tx=0x7feb6001cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106771dd2d2b535c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:19.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60007d80 con 0x7feb500039c0 2024-04-25T16:51:19.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb600a9020 con 0x7feb500039c0 2024-04-25T16:51:19.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb68dbd640 1 -- 172.21.15.94:0/1696117240 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb600a5b70 con 0x7feb500039c0 2024-04-25T16:51:19.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb6c023640 1 -- 172.21.15.94:0/1696117240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb500039c0 msgr2=0x7feb50003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb6c023640 1 --2- 172.21.15.94:0/1696117240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb500039c0 0x7feb50003e30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feb60001060 tx=0x7feb6001cc50 comp rx=0 tx=0).stop 2024-04-25T16:51:19.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb6a5c0640 1 -- 172.21.15.94:0/1696117240 reap_dead start 2024-04-25T16:51:19.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb6c023640 1 -- 172.21.15.94:0/1696117240 shutdown_connections 2024-04-25T16:51:19.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb6c023640 1 -- 172.21.15.94:0/1696117240 >> 172.21.15.94:0/1696117240 conn(0x7feb640cfed0 msgr2=0x7feb640d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:19.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb6c023640 1 -- 172.21.15.94:0/1696117240 shutdown_connections 2024-04-25T16:51:19.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.700+0000 7feb6c023640 1 -- 172.21.15.94:0/1696117240 wait complete. 2024-04-25T16:51:19.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb6c023640 1 Processor -- start 2024-04-25T16:51:19.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb6c023640 1 -- start start 2024-04-25T16:51:19.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb6c023640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb500039c0 0x7feb64152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb6c023640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb64151a30 0x7feb641668f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb6c023640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb64166e30 0x7feb640b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb6c023640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb640d2740 con 0x7feb500039c0 2024-04-25T16:51:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb6c023640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb640cf9f0 con 0x7feb64151a30 2024-04-25T16:51:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb6c023640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feb640d28c0 con 0x7feb64166e30 2024-04-25T16:51:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb69dbf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb500039c0 0x7feb64152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb69dbf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb500039c0 0x7feb64152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41944/0 (socket says 172.21.15.94:41944) 2024-04-25T16:51:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb69dbf640 1 -- 172.21.15.94:0/3615614050 learned_addr learned my addr 172.21.15.94:0/3615614050 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb6a5c0640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb64166e30 0x7feb640b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb695be640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb64151a30 0x7feb641668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb69dbf640 1 -- 172.21.15.94:0/3615614050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb64166e30 msgr2=0x7feb640b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb69dbf640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb64166e30 0x7feb640b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb69dbf640 1 -- 172.21.15.94:0/3615614050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb64151a30 msgr2=0x7feb641668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb69dbf640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb64151a30 0x7feb641668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb69dbf640 1 -- 172.21.15.94:0/3615614050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb64167240 con 0x7feb500039c0 2024-04-25T16:51:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb695be640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb64151a30 0x7feb641668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb6a5c0640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb64166e30 0x7feb640b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb69dbf640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb500039c0 0x7feb64152a40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feb60007040 tx=0x7feb50005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.704+0000 7feb5a7fc640 1 -- 172.21.15.94:0/3615614050 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb600a5b50 con 0x7feb500039c0 2024-04-25T16:51:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.708+0000 7feb5a7fc640 1 -- 172.21.15.94:0/3615614050 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb600b3030 con 0x7feb500039c0 2024-04-25T16:51:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.708+0000 7feb6c023640 1 -- 172.21.15.94:0/3615614050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb641533b0 con 0x7feb500039c0 2024-04-25T16:51:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.708+0000 7feb5a7fc640 1 -- 172.21.15.94:0/3615614050 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb600047f0 con 0x7feb500039c0 2024-04-25T16:51:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.708+0000 7feb6c023640 1 -- 172.21.15.94:0/3615614050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb64153970 con 0x7feb500039c0 2024-04-25T16:51:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.708+0000 7feb6c023640 1 -- 172.21.15.94:0/3615614050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6406a730 con 0x7feb500039c0 2024-04-25T16:51:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.708+0000 7feb5a7fc640 1 -- 172.21.15.94:0/3615614050 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feb60004990 con 0x7feb500039c0 2024-04-25T16:51:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.708+0000 7feb5a7fc640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feb4806f400 0x7feb48071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.708+0000 7feb5a7fc640 1 -- 172.21.15.94:0/3615614050 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7feb60128e30 con 0x7feb500039c0 2024-04-25T16:51:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.708+0000 7feb695be640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feb4806f400 0x7feb48071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.708+0000 7feb695be640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feb4806f400 0x7feb48071910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feb54006f90 tx=0x7feb54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:19.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.712+0000 7feb5a7fc640 1 -- 172.21.15.94:0/3615614050 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feb64167240 con 0x7feb500039c0 2024-04-25T16:51:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.872+0000 7feb6c023640 1 -- 172.21.15.94:0/3615614050 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feb640b29b0 con 0x7feb4806f400 2024-04-25T16:51:19.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.880+0000 7feb5a7fc640 1 -- 172.21.15.94:0/3615614050 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332527 (secure 0 0 0) 0x7feb640b29b0 con 0x7feb4806f400 2024-04-25T16:51:19.881 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:19.883 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:51:19.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.884+0000 7feb6c023640 1 -- 172.21.15.94:0/3615614050 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feb4806f400 msgr2=0x7feb48071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.884+0000 7feb6c023640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feb4806f400 0x7feb48071910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feb54006f90 tx=0x7feb54008040 comp rx=0 tx=0).stop 2024-04-25T16:51:19.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.884+0000 7feb6c023640 1 -- 172.21.15.94:0/3615614050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb500039c0 msgr2=0x7feb64152a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:19.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.884+0000 7feb6c023640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb500039c0 0x7feb64152a40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feb60007040 tx=0x7feb50005840 comp rx=0 tx=0).stop 2024-04-25T16:51:19.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.884+0000 7feb6c023640 1 -- 172.21.15.94:0/3615614050 shutdown_connections 2024-04-25T16:51:19.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.884+0000 7feb6c023640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feb4806f400 0x7feb48071910 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.884+0000 7feb6c023640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb64166e30 0x7feb640b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.884+0000 7feb6c023640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb64151a30 0x7feb641668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.884+0000 7feb6c023640 1 --2- 172.21.15.94:0/3615614050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb500039c0 0x7feb64152a40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:19.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.884+0000 7feb6c023640 1 -- 172.21.15.94:0/3615614050 >> 172.21.15.94:0/3615614050 conn(0x7feb640cfed0 msgr2=0x7feb64167d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:19.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.884+0000 7feb6c023640 1 -- 172.21.15.94:0/3615614050 shutdown_connections 2024-04-25T16:51:19.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:19.884+0000 7feb6c023640 1 -- 172.21.15.94:0/3615614050 wait complete. 2024-04-25T16:51:19.899 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-25T16:51:19.670638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240164,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734556,"statfs":{"total":773094113280,"available":772848185344,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006024"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018387+0000","last_change":"2024-04-25T16:50:54.047200+0000","last_active":"2024-04-25T16:51:01.018387+0000","last_peered":"2024-04-25T16:51:01.018387+0000","last_clean":"2024-04-25T16:51:01.018387+0000","last_became_active":"2024-04-25T16:50:54.046862+0000","last_became_peered":"2024-04-25T16:50:54.046862+0000","last_unstale":"2024-04-25T16:51:01.018387+0000","last_undegraded":"2024-04-25T16:51:01.018387+0000","last_fullsized":"2024-04-25T16:51:01.018387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.166309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311306+0000","last_change":"2024-04-25T16:50:55.060286+0000","last_active":"2024-04-25T16:51:00.311306+0000","last_peered":"2024-04-25T16:51:00.311306+0000","last_clean":"2024-04-25T16:51:00.311306+0000","last_became_active":"2024-04-25T16:50:55.059171+0000","last_became_peered":"2024-04-25T16:50:55.059171+0000","last_unstale":"2024-04-25T16:51:00.311306+0000","last_undegraded":"2024-04-25T16:51:00.311306+0000","last_fullsized":"2024-04-25T16:51:00.311306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:08.427466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310185+0000","last_change":"2024-04-25T16:50:59.181612+0000","last_active":"2024-04-25T16:51:00.310185+0000","last_peered":"2024-04-25T16:51:00.310185+0000","last_clean":"2024-04-25T16:51:00.310185+0000","last_became_active":"2024-04-25T16:50:59.180865+0000","last_became_peered":"2024-04-25T16:50:59.180865+0000","last_unstale":"2024-04-25T16:51:00.310185+0000","last_undegraded":"2024-04-25T16:51:00.310185+0000","last_fullsized":"2024-04-25T16:51:00.310185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:59.449855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311587+0000","last_change":"2024-04-25T16:50:54.040413+0000","last_active":"2024-04-25T16:51:00.311587+0000","last_peered":"2024-04-25T16:51:00.311587+0000","last_clean":"2024-04-25T16:51:00.311587+0000","last_became_active":"2024-04-25T16:50:54.040072+0000","last_became_peered":"2024-04-25T16:50:54.040072+0000","last_unstale":"2024-04-25T16:51:00.311587+0000","last_undegraded":"2024-04-25T16:51:00.311587+0000","last_fullsized":"2024-04-25T16:51:00.311587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:24.611706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018056+0000","last_change":"2024-04-25T16:50:55.083877+0000","last_active":"2024-04-25T16:51:01.018056+0000","last_peered":"2024-04-25T16:51:01.018056+0000","last_clean":"2024-04-25T16:51:01.018056+0000","last_became_active":"2024-04-25T16:50:55.083529+0000","last_became_peered":"2024-04-25T16:50:55.083529+0000","last_unstale":"2024-04-25T16:51:01.018056+0000","last_undegraded":"2024-04-25T16:51:01.018056+0000","last_fullsized":"2024-04-25T16:51:01.018056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152476+0000","last_change":"2024-04-25T16:50:59.143706+0000","last_active":"2024-04-25T16:51:00.152476+0000","last_peered":"2024-04-25T16:51:00.152476+0000","last_clean":"2024-04-25T16:51:00.152476+0000","last_became_active":"2024-04-25T16:50:59.142488+0000","last_became_peered":"2024-04-25T16:50:59.142488+0000","last_unstale":"2024-04-25T16:51:00.152476+0000","last_undegraded":"2024-04-25T16:51:00.152476+0000","last_fullsized":"2024-04-25T16:51:00.152476+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:20.203920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020308+0000","last_change":"2024-04-25T16:50:54.061175+0000","last_active":"2024-04-25T16:51:01.020308+0000","last_peered":"2024-04-25T16:51:01.020308+0000","last_clean":"2024-04-25T16:51:01.020308+0000","last_became_active":"2024-04-25T16:50:54.059991+0000","last_became_peered":"2024-04-25T16:50:54.059991+0000","last_unstale":"2024-04-25T16:51:01.020308+0000","last_undegraded":"2024-04-25T16:51:01.020308+0000","last_fullsized":"2024-04-25T16:51:01.020308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018692+0000","last_change":"2024-04-25T16:50:55.087495+0000","last_active":"2024-04-25T16:51:01.018692+0000","last_peered":"2024-04-25T16:51:01.018692+0000","last_clean":"2024-04-25T16:51:01.018692+0000","last_became_active":"2024-04-25T16:50:55.087177+0000","last_became_peered":"2024-04-25T16:50:55.087177+0000","last_unstale":"2024-04-25T16:51:01.018692+0000","last_undegraded":"2024-04-25T16:51:01.018692+0000","last_fullsized":"2024-04-25T16:51:01.018692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154110+0000","last_change":"2024-04-25T16:50:59.153145+0000","last_active":"2024-04-25T16:51:00.154110+0000","last_peered":"2024-04-25T16:51:00.154110+0000","last_clean":"2024-04-25T16:51:00.154110+0000","last_became_active":"2024-04-25T16:50:59.151443+0000","last_became_peered":"2024-04-25T16:50:59.151443+0000","last_unstale":"2024-04-25T16:51:00.154110+0000","last_undegraded":"2024-04-25T16:51:00.154110+0000","last_fullsized":"2024-04-25T16:51:00.154110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:08.117787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019564+0000","last_change":"2024-04-25T16:50:54.064208+0000","last_active":"2024-04-25T16:51:01.019564+0000","last_peered":"2024-04-25T16:51:01.019564+0000","last_clean":"2024-04-25T16:51:01.019564+0000","last_became_active":"2024-04-25T16:50:54.063734+0000","last_became_peered":"2024-04-25T16:50:54.063734+0000","last_unstale":"2024-04-25T16:51:01.019564+0000","last_undegraded":"2024-04-25T16:51:01.019564+0000","last_fullsized":"2024-04-25T16:51:01.019564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018436+0000","last_change":"2024-04-25T16:50:55.087482+0000","last_active":"2024-04-25T16:51:01.018436+0000","last_peered":"2024-04-25T16:51:01.018436+0000","last_clean":"2024-04-25T16:51:01.018436+0000","last_became_active":"2024-04-25T16:50:55.087139+0000","last_became_peered":"2024-04-25T16:50:55.087139+0000","last_unstale":"2024-04-25T16:51:01.018436+0000","last_undegraded":"2024-04-25T16:51:01.018436+0000","last_fullsized":"2024-04-25T16:51:01.018436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.304249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152921+0000","last_change":"2024-04-25T16:50:59.142658+0000","last_active":"2024-04-25T16:51:00.152921+0000","last_peered":"2024-04-25T16:51:00.152921+0000","last_clean":"2024-04-25T16:51:00.152921+0000","last_became_active":"2024-04-25T16:50:59.139343+0000","last_became_peered":"2024-04-25T16:50:59.139343+0000","last_unstale":"2024-04-25T16:51:00.152921+0000","last_undegraded":"2024-04-25T16:51:00.152921+0000","last_fullsized":"2024-04-25T16:51:00.152921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:45.390516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149327+0000","last_change":"2024-04-25T16:50:55.049255+0000","last_active":"2024-04-25T16:51:00.149327+0000","last_peered":"2024-04-25T16:51:00.149327+0000","last_clean":"2024-04-25T16:51:00.149327+0000","last_became_active":"2024-04-25T16:50:55.047635+0000","last_became_peered":"2024-04-25T16:50:55.047635+0000","last_unstale":"2024-04-25T16:51:00.149327+0000","last_undegraded":"2024-04-25T16:51:00.149327+0000","last_fullsized":"2024-04-25T16:51:00.149327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019735+0000","last_change":"2024-04-25T16:50:54.056721+0000","last_active":"2024-04-25T16:51:01.019735+0000","last_peered":"2024-04-25T16:51:01.019735+0000","last_clean":"2024-04-25T16:51:01.019735+0000","last_became_active":"2024-04-25T16:50:54.055845+0000","last_became_peered":"2024-04-25T16:50:54.055845+0000","last_unstale":"2024-04-25T16:51:01.019735+0000","last_undegraded":"2024-04-25T16:51:01.019735+0000","last_fullsized":"2024-04-25T16:51:01.019735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.077660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153494+0000","last_change":"2024-04-25T16:50:59.152269+0000","last_active":"2024-04-25T16:51:00.153494+0000","last_peered":"2024-04-25T16:51:00.153494+0000","last_clean":"2024-04-25T16:51:00.153494+0000","last_became_active":"2024-04-25T16:50:59.150778+0000","last_became_peered":"2024-04-25T16:50:59.150778+0000","last_unstale":"2024-04-25T16:51:00.153494+0000","last_undegraded":"2024-04-25T16:51:00.153494+0000","last_fullsized":"2024-04-25T16:51:00.153494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:12.513733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310585+0000","last_change":"2024-04-25T16:50:55.072126+0000","last_active":"2024-04-25T16:51:00.310585+0000","last_peered":"2024-04-25T16:51:00.310585+0000","last_clean":"2024-04-25T16:51:00.310585+0000","last_became_active":"2024-04-25T16:50:55.071191+0000","last_became_peered":"2024-04-25T16:50:55.071191+0000","last_unstale":"2024-04-25T16:51:00.310585+0000","last_undegraded":"2024-04-25T16:51:00.310585+0000","last_fullsized":"2024-04-25T16:51:00.310585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015930+0000","last_change":"2024-04-25T16:50:54.033554+0000","last_active":"2024-04-25T16:51:01.015930+0000","last_peered":"2024-04-25T16:51:01.015930+0000","last_clean":"2024-04-25T16:51:01.015930+0000","last_became_active":"2024-04-25T16:50:54.032467+0000","last_became_peered":"2024-04-25T16:50:54.032467+0000","last_unstale":"2024-04-25T16:51:01.015930+0000","last_undegraded":"2024-04-25T16:51:01.015930+0000","last_fullsized":"2024-04-25T16:51:01.015930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.077660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153884+0000","last_change":"2024-04-25T16:50:59.157126+0000","last_active":"2024-04-25T16:51:00.153884+0000","last_peered":"2024-04-25T16:51:00.153884+0000","last_clean":"2024-04-25T16:51:00.153884+0000","last_became_active":"2024-04-25T16:50:59.153352+0000","last_became_peered":"2024-04-25T16:50:59.153352+0000","last_unstale":"2024-04-25T16:51:00.153884+0000","last_undegraded":"2024-04-25T16:51:00.153884+0000","last_fullsized":"2024-04-25T16:51:00.153884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:18.428310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310685+0000","last_change":"2024-04-25T16:50:55.059831+0000","last_active":"2024-04-25T16:51:00.310685+0000","last_peered":"2024-04-25T16:51:00.310685+0000","last_clean":"2024-04-25T16:51:00.310685+0000","last_became_active":"2024-04-25T16:50:55.058958+0000","last_became_peered":"2024-04-25T16:50:55.058958+0000","last_unstale":"2024-04-25T16:51:00.310685+0000","last_undegraded":"2024-04-25T16:51:00.310685+0000","last_fullsized":"2024-04-25T16:51:00.310685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015662+0000","last_change":"2024-04-25T16:50:54.045690+0000","last_active":"2024-04-25T16:51:01.015662+0000","last_peered":"2024-04-25T16:51:01.015662+0000","last_clean":"2024-04-25T16:51:01.015662+0000","last_became_active":"2024-04-25T16:50:54.045057+0000","last_became_peered":"2024-04-25T16:50:54.045057+0000","last_unstale":"2024-04-25T16:51:01.015662+0000","last_undegraded":"2024-04-25T16:51:01.015662+0000","last_fullsized":"2024-04-25T16:51:01.015662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:27.893510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016104+0000","last_change":"2024-04-25T16:50:59.163126+0000","last_active":"2024-04-25T16:51:01.016104+0000","last_peered":"2024-04-25T16:51:01.016104+0000","last_clean":"2024-04-25T16:51:01.016104+0000","last_became_active":"2024-04-25T16:50:59.162606+0000","last_became_peered":"2024-04-25T16:50:59.162606+0000","last_unstale":"2024-04-25T16:51:01.016104+0000","last_undegraded":"2024-04-25T16:51:01.016104+0000","last_fullsized":"2024-04-25T16:51:01.016104+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.992576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153472+0000","last_change":"2024-04-25T16:50:55.079857+0000","last_active":"2024-04-25T16:51:00.153472+0000","last_peered":"2024-04-25T16:51:00.153472+0000","last_clean":"2024-04-25T16:51:00.153472+0000","last_became_active":"2024-04-25T16:50:55.070021+0000","last_became_peered":"2024-04-25T16:50:55.070021+0000","last_unstale":"2024-04-25T16:51:00.153472+0000","last_undegraded":"2024-04-25T16:51:00.153472+0000","last_fullsized":"2024-04-25T16:51:00.153472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:00.753427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017068+0000","last_change":"2024-04-25T16:50:54.050226+0000","last_active":"2024-04-25T16:51:01.017068+0000","last_peered":"2024-04-25T16:51:01.017068+0000","last_clean":"2024-04-25T16:51:01.017068+0000","last_became_active":"2024-04-25T16:50:54.049451+0000","last_became_peered":"2024-04-25T16:50:54.049451+0000","last_unstale":"2024-04-25T16:51:01.017068+0000","last_undegraded":"2024-04-25T16:51:01.017068+0000","last_fullsized":"2024-04-25T16:51:01.017068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:26.942742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154683+0000","last_change":"2024-04-25T16:50:59.152071+0000","last_active":"2024-04-25T16:51:00.154683+0000","last_peered":"2024-04-25T16:51:00.154683+0000","last_clean":"2024-04-25T16:51:00.154683+0000","last_became_active":"2024-04-25T16:50:59.149014+0000","last_became_peered":"2024-04-25T16:50:59.149014+0000","last_unstale":"2024-04-25T16:51:00.154683+0000","last_undegraded":"2024-04-25T16:51:00.154683+0000","last_fullsized":"2024-04-25T16:51:00.154683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:00.447099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018096+0000","last_change":"2024-04-25T16:50:55.057085+0000","last_active":"2024-04-25T16:51:01.018096+0000","last_peered":"2024-04-25T16:51:01.018096+0000","last_clean":"2024-04-25T16:51:01.018096+0000","last_became_active":"2024-04-25T16:50:55.056254+0000","last_became_peered":"2024-04-25T16:50:55.056254+0000","last_unstale":"2024-04-25T16:51:01.018096+0000","last_undegraded":"2024-04-25T16:51:01.018096+0000","last_fullsized":"2024-04-25T16:51:01.018096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151405+0000","last_change":"2024-04-25T16:50:54.047574+0000","last_active":"2024-04-25T16:51:00.151405+0000","last_peered":"2024-04-25T16:51:00.151405+0000","last_clean":"2024-04-25T16:51:00.151405+0000","last_became_active":"2024-04-25T16:50:54.047164+0000","last_became_peered":"2024-04-25T16:50:54.047164+0000","last_unstale":"2024-04-25T16:51:00.151405+0000","last_undegraded":"2024-04-25T16:51:00.151405+0000","last_fullsized":"2024-04-25T16:51:00.151405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152691+0000","last_change":"2024-04-25T16:50:59.181342+0000","last_active":"2024-04-25T16:51:00.152691+0000","last_peered":"2024-04-25T16:51:00.152691+0000","last_clean":"2024-04-25T16:51:00.152691+0000","last_became_active":"2024-04-25T16:50:59.180010+0000","last_became_peered":"2024-04-25T16:50:59.180010+0000","last_unstale":"2024-04-25T16:51:00.152691+0000","last_undegraded":"2024-04-25T16:51:00.152691+0000","last_fullsized":"2024-04-25T16:51:00.152691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:09.707352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.312597+0000","last_change":"2024-04-25T16:50:55.093718+0000","last_active":"2024-04-25T16:51:00.312597+0000","last_peered":"2024-04-25T16:51:00.312597+0000","last_clean":"2024-04-25T16:51:00.312597+0000","last_became_active":"2024-04-25T16:50:55.093376+0000","last_became_peered":"2024-04-25T16:50:55.093376+0000","last_unstale":"2024-04-25T16:51:00.312597+0000","last_undegraded":"2024-04-25T16:51:00.312597+0000","last_fullsized":"2024-04-25T16:51:00.312597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:04.031520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016325+0000","last_change":"2024-04-25T16:50:54.040703+0000","last_active":"2024-04-25T16:51:01.016325+0000","last_peered":"2024-04-25T16:51:01.016325+0000","last_clean":"2024-04-25T16:51:01.016325+0000","last_became_active":"2024-04-25T16:50:54.039961+0000","last_became_peered":"2024-04-25T16:50:54.039961+0000","last_unstale":"2024-04-25T16:51:01.016325+0000","last_undegraded":"2024-04-25T16:51:01.016325+0000","last_fullsized":"2024-04-25T16:51:01.016325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310069+0000","last_change":"2024-04-25T16:50:59.161678+0000","last_active":"2024-04-25T16:51:00.310069+0000","last_peered":"2024-04-25T16:51:00.310069+0000","last_clean":"2024-04-25T16:51:00.310069+0000","last_became_active":"2024-04-25T16:50:59.161064+0000","last_became_peered":"2024-04-25T16:50:59.161064+0000","last_unstale":"2024-04-25T16:51:00.310069+0000","last_undegraded":"2024-04-25T16:51:00.310069+0000","last_fullsized":"2024-04-25T16:51:00.310069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:19.854569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015648+0000","last_change":"2024-04-25T16:50:55.069738+0000","last_active":"2024-04-25T16:51:01.015648+0000","last_peered":"2024-04-25T16:51:01.015648+0000","last_clean":"2024-04-25T16:51:01.015648+0000","last_became_active":"2024-04-25T16:50:55.068864+0000","last_became_peered":"2024-04-25T16:50:55.068864+0000","last_unstale":"2024-04-25T16:51:01.015648+0000","last_undegraded":"2024-04-25T16:51:01.015648+0000","last_fullsized":"2024-04-25T16:51:01.015648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020503+0000","last_change":"2024-04-25T16:50:54.061758+0000","last_active":"2024-04-25T16:51:01.020503+0000","last_peered":"2024-04-25T16:51:01.020503+0000","last_clean":"2024-04-25T16:51:01.020503+0000","last_became_active":"2024-04-25T16:50:54.061267+0000","last_became_peered":"2024-04-25T16:50:54.061267+0000","last_unstale":"2024-04-25T16:51:01.020503+0000","last_undegraded":"2024-04-25T16:51:01.020503+0000","last_fullsized":"2024-04-25T16:51:01.020503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:21.043343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017543+0000","last_change":"2024-04-25T16:50:59.174520+0000","last_active":"2024-04-25T16:51:01.017543+0000","last_peered":"2024-04-25T16:51:01.017543+0000","last_clean":"2024-04-25T16:51:01.017543+0000","last_became_active":"2024-04-25T16:50:59.173812+0000","last_became_peered":"2024-04-25T16:50:59.173812+0000","last_unstale":"2024-04-25T16:51:01.017543+0000","last_undegraded":"2024-04-25T16:51:01.017543+0000","last_fullsized":"2024-04-25T16:51:01.017543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:29.939373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018748+0000","last_change":"2024-04-25T16:50:55.090616+0000","last_active":"2024-04-25T16:51:01.018748+0000","last_peered":"2024-04-25T16:51:01.018748+0000","last_clean":"2024-04-25T16:51:01.018748+0000","last_became_active":"2024-04-25T16:50:55.090287+0000","last_became_peered":"2024-04-25T16:50:55.090287+0000","last_unstale":"2024-04-25T16:51:01.018748+0000","last_undegraded":"2024-04-25T16:51:01.018748+0000","last_fullsized":"2024-04-25T16:51:01.018748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:04.031520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015756+0000","last_change":"2024-04-25T16:50:54.023906+0000","last_active":"2024-04-25T16:51:01.015756+0000","last_peered":"2024-04-25T16:51:01.015756+0000","last_clean":"2024-04-25T16:51:01.015756+0000","last_became_active":"2024-04-25T16:50:54.022198+0000","last_became_peered":"2024-04-25T16:50:54.022198+0000","last_unstale":"2024-04-25T16:51:01.015756+0000","last_undegraded":"2024-04-25T16:51:01.015756+0000","last_fullsized":"2024-04-25T16:51:01.015756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015375+0000","last_change":"2024-04-25T16:50:59.153325+0000","last_active":"2024-04-25T16:51:01.015375+0000","last_peered":"2024-04-25T16:51:01.015375+0000","last_clean":"2024-04-25T16:51:01.015375+0000","last_became_active":"2024-04-25T16:50:59.152431+0000","last_became_peered":"2024-04-25T16:50:59.152431+0000","last_unstale":"2024-04-25T16:51:01.015375+0000","last_undegraded":"2024-04-25T16:51:01.015375+0000","last_fullsized":"2024-04-25T16:51:01.015375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.819554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154818+0000","last_change":"2024-04-25T16:50:55.058098+0000","last_active":"2024-04-25T16:51:00.154818+0000","last_peered":"2024-04-25T16:51:00.154818+0000","last_clean":"2024-04-25T16:51:00.154818+0000","last_became_active":"2024-04-25T16:50:55.057233+0000","last_became_peered":"2024-04-25T16:50:55.057233+0000","last_unstale":"2024-04-25T16:51:00.154818+0000","last_undegraded":"2024-04-25T16:51:00.154818+0000","last_fullsized":"2024-04-25T16:51:00.154818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:36.088026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019426+0000","last_change":"2024-04-25T16:50:54.059732+0000","last_active":"2024-04-25T16:51:01.019426+0000","last_peered":"2024-04-25T16:51:01.019426+0000","last_clean":"2024-04-25T16:51:01.019426+0000","last_became_active":"2024-04-25T16:50:54.058805+0000","last_became_peered":"2024-04-25T16:50:54.058805+0000","last_unstale":"2024-04-25T16:51:01.019426+0000","last_undegraded":"2024-04-25T16:51:01.019426+0000","last_fullsized":"2024-04-25T16:51:01.019426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015551+0000","last_change":"2024-04-25T16:50:59.157551+0000","last_active":"2024-04-25T16:51:01.015551+0000","last_peered":"2024-04-25T16:51:01.015551+0000","last_clean":"2024-04-25T16:51:01.015551+0000","last_became_active":"2024-04-25T16:50:59.156077+0000","last_became_peered":"2024-04-25T16:50:59.156077+0000","last_unstale":"2024-04-25T16:51:01.015551+0000","last_undegraded":"2024-04-25T16:51:01.015551+0000","last_fullsized":"2024-04-25T16:51:01.015551+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:29.939373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015015+0000","last_change":"2024-04-25T16:50:55.072516+0000","last_active":"2024-04-25T16:51:01.015015+0000","last_peered":"2024-04-25T16:51:01.015015+0000","last_clean":"2024-04-25T16:51:01.015015+0000","last_became_active":"2024-04-25T16:50:55.071677+0000","last_became_peered":"2024-04-25T16:50:55.071677+0000","last_unstale":"2024-04-25T16:51:01.015015+0000","last_undegraded":"2024-04-25T16:51:01.015015+0000","last_fullsized":"2024-04-25T16:51:01.015015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148828+0000","last_change":"2024-04-25T16:50:54.026443+0000","last_active":"2024-04-25T16:51:00.148828+0000","last_peered":"2024-04-25T16:51:00.148828+0000","last_clean":"2024-04-25T16:51:00.148828+0000","last_became_active":"2024-04-25T16:50:54.024224+0000","last_became_peered":"2024-04-25T16:50:54.024224+0000","last_unstale":"2024-04-25T16:51:00.148828+0000","last_undegraded":"2024-04-25T16:51:00.148828+0000","last_fullsized":"2024-04-25T16:51:00.148828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016170+0000","last_change":"2024-04-25T16:50:59.149300+0000","last_active":"2024-04-25T16:51:01.016170+0000","last_peered":"2024-04-25T16:51:01.016170+0000","last_clean":"2024-04-25T16:51:01.016170+0000","last_became_active":"2024-04-25T16:50:59.147799+0000","last_became_peered":"2024-04-25T16:50:59.147799+0000","last_unstale":"2024-04-25T16:51:01.016170+0000","last_undegraded":"2024-04-25T16:51:01.016170+0000","last_fullsized":"2024-04-25T16:51:01.016170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:05:04.839576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016900+0000","last_change":"2024-04-25T16:50:55.048595+0000","last_active":"2024-04-25T16:51:01.016900+0000","last_peered":"2024-04-25T16:51:01.016900+0000","last_clean":"2024-04-25T16:51:01.016900+0000","last_became_active":"2024-04-25T16:50:55.047706+0000","last_became_peered":"2024-04-25T16:50:55.047706+0000","last_unstale":"2024-04-25T16:51:01.016900+0000","last_undegraded":"2024-04-25T16:51:01.016900+0000","last_fullsized":"2024-04-25T16:51:01.016900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:56.480058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017350+0000","last_change":"2024-04-25T16:50:54.053397+0000","last_active":"2024-04-25T16:51:01.017350+0000","last_peered":"2024-04-25T16:51:01.017350+0000","last_clean":"2024-04-25T16:51:01.017350+0000","last_became_active":"2024-04-25T16:50:54.052633+0000","last_became_peered":"2024-04-25T16:50:54.052633+0000","last_unstale":"2024-04-25T16:51:01.017350+0000","last_undegraded":"2024-04-25T16:51:01.017350+0000","last_fullsized":"2024-04-25T16:51:01.017350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018189+0000","last_change":"2024-04-25T16:50:59.162703+0000","last_active":"2024-04-25T16:51:01.018189+0000","last_peered":"2024-04-25T16:51:01.018189+0000","last_clean":"2024-04-25T16:51:01.018189+0000","last_became_active":"2024-04-25T16:50:59.161980+0000","last_became_peered":"2024-04-25T16:50:59.161980+0000","last_unstale":"2024-04-25T16:51:01.018189+0000","last_undegraded":"2024-04-25T16:51:01.018189+0000","last_fullsized":"2024-04-25T16:51:01.018189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:05:04.839576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310673+0000","last_change":"2024-04-25T16:50:55.090720+0000","last_active":"2024-04-25T16:51:00.310673+0000","last_peered":"2024-04-25T16:51:00.310673+0000","last_clean":"2024-04-25T16:51:00.310673+0000","last_became_active":"2024-04-25T16:50:55.090360+0000","last_became_peered":"2024-04-25T16:50:55.090360+0000","last_unstale":"2024-04-25T16:51:00.310673+0000","last_undegraded":"2024-04-25T16:51:00.310673+0000","last_fullsized":"2024-04-25T16:51:00.310673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018278+0000","last_change":"2024-04-25T16:50:54.050879+0000","last_active":"2024-04-25T16:51:01.018278+0000","last_peered":"2024-04-25T16:51:01.018278+0000","last_clean":"2024-04-25T16:51:01.018278+0000","last_became_active":"2024-04-25T16:50:54.049784+0000","last_became_peered":"2024-04-25T16:50:54.049784+0000","last_unstale":"2024-04-25T16:51:01.018278+0000","last_undegraded":"2024-04-25T16:51:01.018278+0000","last_fullsized":"2024-04-25T16:51:01.018278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":118,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017460+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:51:01.017460+0000","last_peered":"2024-04-25T16:51:01.017460+0000","last_clean":"2024-04-25T16:51:01.017460+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:51:01.017460+0000","last_undegraded":"2024-04-25T16:51:01.017460+0000","last_fullsized":"2024-04-25T16:51:01.017460+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150640+0000","last_change":"2024-04-25T16:50:59.175343+0000","last_active":"2024-04-25T16:51:00.150640+0000","last_peered":"2024-04-25T16:51:00.150640+0000","last_clean":"2024-04-25T16:51:00.150640+0000","last_became_active":"2024-04-25T16:50:59.173979+0000","last_became_peered":"2024-04-25T16:50:59.173979+0000","last_unstale":"2024-04-25T16:51:00.150640+0000","last_undegraded":"2024-04-25T16:51:00.150640+0000","last_fullsized":"2024-04-25T16:51:00.150640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:07.790324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019964+0000","last_change":"2024-04-25T16:50:55.092868+0000","last_active":"2024-04-25T16:51:01.019964+0000","last_peered":"2024-04-25T16:51:01.019964+0000","last_clean":"2024-04-25T16:51:01.019964+0000","last_became_active":"2024-04-25T16:50:55.092569+0000","last_became_peered":"2024-04-25T16:50:55.092569+0000","last_unstale":"2024-04-25T16:51:01.019964+0000","last_undegraded":"2024-04-25T16:51:01.019964+0000","last_fullsized":"2024-04-25T16:51:01.019964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:03.704057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149644+0000","last_change":"2024-04-25T16:50:54.056537+0000","last_active":"2024-04-25T16:51:00.149644+0000","last_peered":"2024-04-25T16:51:00.149644+0000","last_clean":"2024-04-25T16:51:00.149644+0000","last_became_active":"2024-04-25T16:50:54.056219+0000","last_became_peered":"2024-04-25T16:50:54.056219+0000","last_unstale":"2024-04-25T16:51:00.149644+0000","last_undegraded":"2024-04-25T16:51:00.149644+0000","last_fullsized":"2024-04-25T16:51:00.149644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149259+0000","last_change":"2024-04-25T16:50:59.168724+0000","last_active":"2024-04-25T16:51:00.149259+0000","last_peered":"2024-04-25T16:51:00.149259+0000","last_clean":"2024-04-25T16:51:00.149259+0000","last_became_active":"2024-04-25T16:50:59.167823+0000","last_became_peered":"2024-04-25T16:50:59.167823+0000","last_unstale":"2024-04-25T16:51:00.149259+0000","last_undegraded":"2024-04-25T16:51:00.149259+0000","last_fullsized":"2024-04-25T16:51:00.149259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:00.447099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018846+0000","last_change":"2024-04-25T16:50:55.091727+0000","last_active":"2024-04-25T16:51:01.018846+0000","last_peered":"2024-04-25T16:51:01.018846+0000","last_clean":"2024-04-25T16:51:01.018846+0000","last_became_active":"2024-04-25T16:50:55.091409+0000","last_became_peered":"2024-04-25T16:50:55.091409+0000","last_unstale":"2024-04-25T16:51:01.018846+0000","last_undegraded":"2024-04-25T16:51:01.018846+0000","last_fullsized":"2024-04-25T16:51:01.018846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:14.342043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015818+0000","last_change":"2024-04-25T16:50:54.051928+0000","last_active":"2024-04-25T16:51:01.015818+0000","last_peered":"2024-04-25T16:51:01.015818+0000","last_clean":"2024-04-25T16:51:01.015818+0000","last_became_active":"2024-04-25T16:50:54.045379+0000","last_became_peered":"2024-04-25T16:50:54.045379+0000","last_unstale":"2024-04-25T16:51:01.015818+0000","last_undegraded":"2024-04-25T16:51:01.015818+0000","last_fullsized":"2024-04-25T16:51:01.015818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017284+0000","last_change":"2024-04-25T16:50:59.149171+0000","last_active":"2024-04-25T16:51:01.017284+0000","last_peered":"2024-04-25T16:51:01.017284+0000","last_clean":"2024-04-25T16:51:01.017284+0000","last_became_active":"2024-04-25T16:50:59.148753+0000","last_became_peered":"2024-04-25T16:50:59.148753+0000","last_unstale":"2024-04-25T16:51:01.017284+0000","last_undegraded":"2024-04-25T16:51:01.017284+0000","last_fullsized":"2024-04-25T16:51:01.017284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.819554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.021346+0000","last_change":"2024-04-25T16:50:55.093340+0000","last_active":"2024-04-25T16:51:01.021346+0000","last_peered":"2024-04-25T16:51:01.021346+0000","last_clean":"2024-04-25T16:51:01.021346+0000","last_became_active":"2024-04-25T16:50:55.092870+0000","last_became_peered":"2024-04-25T16:50:55.092870+0000","last_unstale":"2024-04-25T16:51:01.021346+0000","last_undegraded":"2024-04-25T16:51:01.021346+0000","last_fullsized":"2024-04-25T16:51:01.021346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:25.853106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150507+0000","last_change":"2024-04-25T16:50:54.041300+0000","last_active":"2024-04-25T16:51:00.150507+0000","last_peered":"2024-04-25T16:51:00.150507+0000","last_clean":"2024-04-25T16:51:00.150507+0000","last_became_active":"2024-04-25T16:50:54.040987+0000","last_became_peered":"2024-04-25T16:50:54.040987+0000","last_unstale":"2024-04-25T16:51:00.150507+0000","last_undegraded":"2024-04-25T16:51:00.150507+0000","last_fullsized":"2024-04-25T16:51:00.150507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016995+0000","last_change":"2024-04-25T16:50:59.180254+0000","last_active":"2024-04-25T16:51:01.016995+0000","last_peered":"2024-04-25T16:51:01.016995+0000","last_clean":"2024-04-25T16:51:01.016995+0000","last_became_active":"2024-04-25T16:50:59.179432+0000","last_became_peered":"2024-04-25T16:50:59.179432+0000","last_unstale":"2024-04-25T16:51:01.016995+0000","last_undegraded":"2024-04-25T16:51:01.016995+0000","last_fullsized":"2024-04-25T16:51:01.016995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:44.752448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020598+0000","last_change":"2024-04-25T16:50:55.085874+0000","last_active":"2024-04-25T16:51:01.020598+0000","last_peered":"2024-04-25T16:51:01.020598+0000","last_clean":"2024-04-25T16:51:01.020598+0000","last_became_active":"2024-04-25T16:50:55.085347+0000","last_became_peered":"2024-04-25T16:50:55.085347+0000","last_unstale":"2024-04-25T16:51:01.020598+0000","last_undegraded":"2024-04-25T16:51:01.020598+0000","last_fullsized":"2024-04-25T16:51:01.020598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:14.906309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152960+0000","last_change":"2024-04-25T16:50:54.037711+0000","last_active":"2024-04-25T16:51:00.152960+0000","last_peered":"2024-04-25T16:51:00.152960+0000","last_clean":"2024-04-25T16:51:00.152960+0000","last_became_active":"2024-04-25T16:50:54.036847+0000","last_became_peered":"2024-04-25T16:50:54.036847+0000","last_unstale":"2024-04-25T16:51:00.152960+0000","last_undegraded":"2024-04-25T16:51:00.152960+0000","last_fullsized":"2024-04-25T16:51:00.152960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152479+0000","last_change":"2024-04-25T16:50:59.165076+0000","last_active":"2024-04-25T16:51:00.152479+0000","last_peered":"2024-04-25T16:51:00.152479+0000","last_clean":"2024-04-25T16:51:00.152479+0000","last_became_active":"2024-04-25T16:50:59.159200+0000","last_became_peered":"2024-04-25T16:50:59.159200+0000","last_unstale":"2024-04-25T16:51:00.152479+0000","last_undegraded":"2024-04-25T16:51:00.152479+0000","last_fullsized":"2024-04-25T16:51:00.152479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:11.722626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017585+0000","last_change":"2024-04-25T16:50:55.085792+0000","last_active":"2024-04-25T16:51:01.017585+0000","last_peered":"2024-04-25T16:51:01.017585+0000","last_clean":"2024-04-25T16:51:01.017585+0000","last_became_active":"2024-04-25T16:50:55.085212+0000","last_became_peered":"2024-04-25T16:50:55.085212+0000","last_unstale":"2024-04-25T16:51:01.017585+0000","last_undegraded":"2024-04-25T16:51:01.017585+0000","last_fullsized":"2024-04-25T16:51:01.017585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:08.427466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148299+0000","last_change":"2024-04-25T16:50:54.021199+0000","last_active":"2024-04-25T16:51:00.148299+0000","last_peered":"2024-04-25T16:51:00.148299+0000","last_clean":"2024-04-25T16:51:00.148299+0000","last_became_active":"2024-04-25T16:50:54.020716+0000","last_became_peered":"2024-04-25T16:50:54.020716+0000","last_unstale":"2024-04-25T16:51:00.148299+0000","last_undegraded":"2024-04-25T16:51:00.148299+0000","last_fullsized":"2024-04-25T16:51:00.148299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:27.893510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017880+0000","last_change":"2024-04-25T16:50:59.160368+0000","last_active":"2024-04-25T16:51:01.017880+0000","last_peered":"2024-04-25T16:51:01.017880+0000","last_clean":"2024-04-25T16:51:01.017880+0000","last_became_active":"2024-04-25T16:50:59.159232+0000","last_became_peered":"2024-04-25T16:50:59.159232+0000","last_unstale":"2024-04-25T16:51:01.017880+0000","last_undegraded":"2024-04-25T16:51:01.017880+0000","last_fullsized":"2024-04-25T16:51:01.017880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.992576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148948+0000","last_change":"2024-04-25T16:50:55.059427+0000","last_active":"2024-04-25T16:51:00.148948+0000","last_peered":"2024-04-25T16:51:00.148948+0000","last_clean":"2024-04-25T16:51:00.148948+0000","last_became_active":"2024-04-25T16:50:55.058611+0000","last_became_peered":"2024-04-25T16:50:55.058611+0000","last_unstale":"2024-04-25T16:51:00.148948+0000","last_undegraded":"2024-04-25T16:51:00.148948+0000","last_fullsized":"2024-04-25T16:51:00.148948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149289+0000","last_change":"2024-04-25T16:50:54.044858+0000","last_active":"2024-04-25T16:51:00.149289+0000","last_peered":"2024-04-25T16:51:00.149289+0000","last_clean":"2024-04-25T16:51:00.149289+0000","last_became_active":"2024-04-25T16:50:54.043345+0000","last_became_peered":"2024-04-25T16:50:54.043345+0000","last_unstale":"2024-04-25T16:51:00.149289+0000","last_undegraded":"2024-04-25T16:51:00.149289+0000","last_fullsized":"2024-04-25T16:51:00.149289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.312170+0000","last_change":"2024-04-25T16:50:59.173258+0000","last_active":"2024-04-25T16:51:00.312170+0000","last_peered":"2024-04-25T16:51:00.312170+0000","last_clean":"2024-04-25T16:51:00.312170+0000","last_became_active":"2024-04-25T16:50:59.172637+0000","last_became_peered":"2024-04-25T16:50:59.172637+0000","last_unstale":"2024-04-25T16:51:00.312170+0000","last_undegraded":"2024-04-25T16:51:00.312170+0000","last_fullsized":"2024-04-25T16:51:00.312170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:46.618416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019246+0000","last_change":"2024-04-25T16:50:55.085682+0000","last_active":"2024-04-25T16:51:01.019246+0000","last_peered":"2024-04-25T16:51:01.019246+0000","last_clean":"2024-04-25T16:51:01.019246+0000","last_became_active":"2024-04-25T16:50:55.084855+0000","last_became_peered":"2024-04-25T16:50:55.084855+0000","last_unstale":"2024-04-25T16:51:01.019246+0000","last_undegraded":"2024-04-25T16:51:01.019246+0000","last_fullsized":"2024-04-25T16:51:01.019246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:42.081728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019411+0000","last_change":"2024-04-25T16:50:54.059696+0000","last_active":"2024-04-25T16:51:01.019411+0000","last_peered":"2024-04-25T16:51:01.019411+0000","last_clean":"2024-04-25T16:51:01.019411+0000","last_became_active":"2024-04-25T16:50:54.058979+0000","last_became_peered":"2024-04-25T16:50:54.058979+0000","last_unstale":"2024-04-25T16:51:01.019411+0000","last_undegraded":"2024-04-25T16:51:01.019411+0000","last_fullsized":"2024-04-25T16:51:01.019411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311564+0000","last_change":"2024-04-25T16:50:59.163291+0000","last_active":"2024-04-25T16:51:00.311564+0000","last_peered":"2024-04-25T16:51:00.311564+0000","last_clean":"2024-04-25T16:51:00.311564+0000","last_became_active":"2024-04-25T16:50:59.162316+0000","last_became_peered":"2024-04-25T16:50:59.162316+0000","last_unstale":"2024-04-25T16:51:00.311564+0000","last_undegraded":"2024-04-25T16:51:00.311564+0000","last_fullsized":"2024-04-25T16:51:00.311564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:41.776182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018471+0000","last_change":"2024-04-25T16:50:55.067672+0000","last_active":"2024-04-25T16:51:01.018471+0000","last_peered":"2024-04-25T16:51:01.018471+0000","last_clean":"2024-04-25T16:51:01.018471+0000","last_became_active":"2024-04-25T16:50:55.066870+0000","last_became_peered":"2024-04-25T16:50:55.066870+0000","last_unstale":"2024-04-25T16:51:01.018471+0000","last_undegraded":"2024-04-25T16:51:01.018471+0000","last_fullsized":"2024-04-25T16:51:01.018471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:28:31.672027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017648+0000","last_change":"2024-04-25T16:50:54.036862+0000","last_active":"2024-04-25T16:51:01.017648+0000","last_peered":"2024-04-25T16:51:01.017648+0000","last_clean":"2024-04-25T16:51:01.017648+0000","last_became_active":"2024-04-25T16:50:54.036549+0000","last_became_peered":"2024-04-25T16:50:54.036549+0000","last_unstale":"2024-04-25T16:51:01.017648+0000","last_undegraded":"2024-04-25T16:51:01.017648+0000","last_fullsized":"2024-04-25T16:51:01.017648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:24.611706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018388+0000","last_change":"2024-04-25T16:50:59.175285+0000","last_active":"2024-04-25T16:51:01.018388+0000","last_peered":"2024-04-25T16:51:01.018388+0000","last_clean":"2024-04-25T16:51:01.018388+0000","last_became_active":"2024-04-25T16:50:59.174301+0000","last_became_peered":"2024-04-25T16:50:59.174301+0000","last_unstale":"2024-04-25T16:51:01.018388+0000","last_undegraded":"2024-04-25T16:51:01.018388+0000","last_fullsized":"2024-04-25T16:51:01.018388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:25.742371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019931+0000","last_change":"2024-04-25T16:50:55.076173+0000","last_active":"2024-04-25T16:51:01.019931+0000","last_peered":"2024-04-25T16:51:01.019931+0000","last_clean":"2024-04-25T16:51:01.019931+0000","last_became_active":"2024-04-25T16:50:55.075360+0000","last_became_peered":"2024-04-25T16:50:55.075360+0000","last_unstale":"2024-04-25T16:51:01.019931+0000","last_undegraded":"2024-04-25T16:51:01.019931+0000","last_fullsized":"2024-04-25T16:51:01.019931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:04.031520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:10.281685+0000","last_change":"2024-04-25T16:50:54.035115+0000","last_active":"2024-04-25T16:51:10.281685+0000","last_peered":"2024-04-25T16:51:10.281685+0000","last_clean":"2024-04-25T16:51:10.281685+0000","last_became_active":"2024-04-25T16:50:54.034629+0000","last_became_peered":"2024-04-25T16:50:54.034629+0000","last_unstale":"2024-04-25T16:51:10.281685+0000","last_undegraded":"2024-04-25T16:51:10.281685+0000","last_fullsized":"2024-04-25T16:51:10.281685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.077660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015930+0000","last_change":"2024-04-25T16:50:59.159337+0000","last_active":"2024-04-25T16:51:01.015930+0000","last_peered":"2024-04-25T16:51:01.015930+0000","last_clean":"2024-04-25T16:51:01.015930+0000","last_became_active":"2024-04-25T16:50:59.156207+0000","last_became_peered":"2024-04-25T16:50:59.156207+0000","last_unstale":"2024-04-25T16:51:01.015930+0000","last_undegraded":"2024-04-25T16:51:01.015930+0000","last_fullsized":"2024-04-25T16:51:01.015930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:39.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154043+0000","last_change":"2024-04-25T16:50:55.066237+0000","last_active":"2024-04-25T16:51:00.154043+0000","last_peered":"2024-04-25T16:51:00.154043+0000","last_clean":"2024-04-25T16:51:00.154043+0000","last_became_active":"2024-04-25T16:50:55.063200+0000","last_became_peered":"2024-04-25T16:50:55.063200+0000","last_unstale":"2024-04-25T16:51:00.154043+0000","last_undegraded":"2024-04-25T16:51:00.154043+0000","last_fullsized":"2024-04-25T16:51:00.154043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:22.053709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015709+0000","last_change":"2024-04-25T16:50:54.055594+0000","last_active":"2024-04-25T16:51:01.015709+0000","last_peered":"2024-04-25T16:51:01.015709+0000","last_clean":"2024-04-25T16:51:01.015709+0000","last_became_active":"2024-04-25T16:50:54.055027+0000","last_became_peered":"2024-04-25T16:50:54.055027+0000","last_unstale":"2024-04-25T16:51:01.015709+0000","last_undegraded":"2024-04-25T16:51:01.015709+0000","last_fullsized":"2024-04-25T16:51:01.015709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154662+0000","last_change":"2024-04-25T16:50:59.160116+0000","last_active":"2024-04-25T16:51:00.154662+0000","last_peered":"2024-04-25T16:51:00.154662+0000","last_clean":"2024-04-25T16:51:00.154662+0000","last_became_active":"2024-04-25T16:50:59.157923+0000","last_became_peered":"2024-04-25T16:50:59.157923+0000","last_unstale":"2024-04-25T16:51:00.154662+0000","last_undegraded":"2024-04-25T16:51:00.154662+0000","last_fullsized":"2024-04-25T16:51:00.154662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:46.167995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018271+0000","last_change":"2024-04-25T16:50:55.083898+0000","last_active":"2024-04-25T16:51:01.018271+0000","last_peered":"2024-04-25T16:51:01.018271+0000","last_clean":"2024-04-25T16:51:01.018271+0000","last_became_active":"2024-04-25T16:50:55.083533+0000","last_became_peered":"2024-04-25T16:50:55.083533+0000","last_unstale":"2024-04-25T16:51:01.018271+0000","last_undegraded":"2024-04-25T16:51:01.018271+0000","last_fullsized":"2024-04-25T16:51:01.018271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019088+0000","last_change":"2024-04-25T16:50:54.056906+0000","last_active":"2024-04-25T16:51:01.019088+0000","last_peered":"2024-04-25T16:51:01.019088+0000","last_clean":"2024-04-25T16:51:01.019088+0000","last_became_active":"2024-04-25T16:50:54.056102+0000","last_became_peered":"2024-04-25T16:50:54.056102+0000","last_unstale":"2024-04-25T16:51:01.019088+0000","last_undegraded":"2024-04-25T16:51:01.019088+0000","last_fullsized":"2024-04-25T16:51:01.019088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310494+0000","last_change":"2024-04-25T16:50:59.129868+0000","last_active":"2024-04-25T16:51:00.310494+0000","last_peered":"2024-04-25T16:51:00.310494+0000","last_clean":"2024-04-25T16:51:00.310494+0000","last_became_active":"2024-04-25T16:50:59.129447+0000","last_became_peered":"2024-04-25T16:50:59.129447+0000","last_unstale":"2024-04-25T16:51:00.310494+0000","last_undegraded":"2024-04-25T16:51:00.310494+0000","last_fullsized":"2024-04-25T16:51:00.310494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:07.826745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018797+0000","last_change":"2024-04-25T16:50:55.093338+0000","last_active":"2024-04-25T16:51:01.018797+0000","last_peered":"2024-04-25T16:51:01.018797+0000","last_clean":"2024-04-25T16:51:01.018797+0000","last_became_active":"2024-04-25T16:50:55.092985+0000","last_became_peered":"2024-04-25T16:50:55.092985+0000","last_unstale":"2024-04-25T16:51:01.018797+0000","last_undegraded":"2024-04-25T16:51:01.018797+0000","last_fullsized":"2024-04-25T16:51:01.018797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:03.704057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151183+0000","last_change":"2024-04-25T16:50:54.053862+0000","last_active":"2024-04-25T16:51:00.151183+0000","last_peered":"2024-04-25T16:51:00.151183+0000","last_clean":"2024-04-25T16:51:00.151183+0000","last_became_active":"2024-04-25T16:50:54.053156+0000","last_became_peered":"2024-04-25T16:50:54.053156+0000","last_unstale":"2024-04-25T16:51:00.151183+0000","last_undegraded":"2024-04-25T16:51:00.151183+0000","last_fullsized":"2024-04-25T16:51:00.151183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310744+0000","last_change":"2024-04-25T16:50:59.181080+0000","last_active":"2024-04-25T16:51:00.310744+0000","last_peered":"2024-04-25T16:51:00.310744+0000","last_clean":"2024-04-25T16:51:00.310744+0000","last_became_active":"2024-04-25T16:50:59.180638+0000","last_became_peered":"2024-04-25T16:50:59.180638+0000","last_unstale":"2024-04-25T16:51:00.310744+0000","last_undegraded":"2024-04-25T16:51:00.310744+0000","last_fullsized":"2024-04-25T16:51:00.310744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:47.904606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017640+0000","last_change":"2024-04-25T16:50:59.173382+0000","last_active":"2024-04-25T16:51:01.017640+0000","last_peered":"2024-04-25T16:51:01.017640+0000","last_clean":"2024-04-25T16:51:01.017640+0000","last_became_active":"2024-04-25T16:50:59.172921+0000","last_became_peered":"2024-04-25T16:50:59.172921+0000","last_unstale":"2024-04-25T16:51:01.017640+0000","last_undegraded":"2024-04-25T16:51:01.017640+0000","last_fullsized":"2024-04-25T16:51:01.017640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:39.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015213+0000","last_change":"2024-04-25T16:50:55.065015+0000","last_active":"2024-04-25T16:51:01.015213+0000","last_peered":"2024-04-25T16:51:01.015213+0000","last_clean":"2024-04-25T16:51:01.015213+0000","last_became_active":"2024-04-25T16:50:55.063038+0000","last_became_peered":"2024-04-25T16:50:55.063038+0000","last_unstale":"2024-04-25T16:51:01.015213+0000","last_undegraded":"2024-04-25T16:51:01.015213+0000","last_fullsized":"2024-04-25T16:51:01.015213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017330+0000","last_change":"2024-04-25T16:50:54.054787+0000","last_active":"2024-04-25T16:51:01.017330+0000","last_peered":"2024-04-25T16:51:01.017330+0000","last_clean":"2024-04-25T16:51:01.017330+0000","last_became_active":"2024-04-25T16:50:54.051984+0000","last_became_peered":"2024-04-25T16:50:54.051984+0000","last_unstale":"2024-04-25T16:51:01.017330+0000","last_undegraded":"2024-04-25T16:51:01.017330+0000","last_fullsized":"2024-04-25T16:51:01.017330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.166309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151032+0000","last_change":"2024-04-25T16:50:59.154675+0000","last_active":"2024-04-25T16:51:00.151032+0000","last_peered":"2024-04-25T16:51:00.151032+0000","last_clean":"2024-04-25T16:51:00.151032+0000","last_became_active":"2024-04-25T16:50:59.152969+0000","last_became_peered":"2024-04-25T16:50:59.152969+0000","last_unstale":"2024-04-25T16:51:00.151032+0000","last_undegraded":"2024-04-25T16:51:00.151032+0000","last_fullsized":"2024-04-25T16:51:00.151032+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.819554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310256+0000","last_change":"2024-04-25T16:50:55.069932+0000","last_active":"2024-04-25T16:51:00.310256+0000","last_peered":"2024-04-25T16:51:00.310256+0000","last_clean":"2024-04-25T16:51:00.310256+0000","last_became_active":"2024-04-25T16:50:55.060902+0000","last_became_peered":"2024-04-25T16:50:55.060902+0000","last_unstale":"2024-04-25T16:51:00.310256+0000","last_undegraded":"2024-04-25T16:51:00.310256+0000","last_fullsized":"2024-04-25T16:51:00.310256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310569+0000","last_change":"2024-04-25T16:50:54.043300+0000","last_active":"2024-04-25T16:51:00.310569+0000","last_peered":"2024-04-25T16:51:00.310569+0000","last_clean":"2024-04-25T16:51:00.310569+0000","last_became_active":"2024-04-25T16:50:54.042909+0000","last_became_peered":"2024-04-25T16:50:54.042909+0000","last_unstale":"2024-04-25T16:51:00.310569+0000","last_undegraded":"2024-04-25T16:51:00.310569+0000","last_fullsized":"2024-04-25T16:51:00.310569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:22.485560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149622+0000","last_change":"2024-04-25T16:50:54.044711+0000","last_active":"2024-04-25T16:51:00.149622+0000","last_peered":"2024-04-25T16:51:00.149622+0000","last_clean":"2024-04-25T16:51:00.149622+0000","last_became_active":"2024-04-25T16:50:54.043597+0000","last_became_peered":"2024-04-25T16:50:54.043597+0000","last_unstale":"2024-04-25T16:51:00.149622+0000","last_undegraded":"2024-04-25T16:51:00.149622+0000","last_fullsized":"2024-04-25T16:51:00.149622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017542+0000","last_change":"2024-04-25T16:50:55.061228+0000","last_active":"2024-04-25T16:51:01.017542+0000","last_peered":"2024-04-25T16:51:01.017542+0000","last_clean":"2024-04-25T16:51:01.017542+0000","last_became_active":"2024-04-25T16:50:55.060211+0000","last_became_peered":"2024-04-25T16:50:55.060211+0000","last_unstale":"2024-04-25T16:51:01.017542+0000","last_undegraded":"2024-04-25T16:51:01.017542+0000","last_fullsized":"2024-04-25T16:51:01.017542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149479+0000","last_change":"2024-04-25T16:50:59.151109+0000","last_active":"2024-04-25T16:51:00.149479+0000","last_peered":"2024-04-25T16:51:00.149479+0000","last_clean":"2024-04-25T16:51:00.149479+0000","last_became_active":"2024-04-25T16:50:59.150443+0000","last_became_peered":"2024-04-25T16:50:59.150443+0000","last_unstale":"2024-04-25T16:51:00.149479+0000","last_undegraded":"2024-04-25T16:51:00.149479+0000","last_fullsized":"2024-04-25T16:51:00.149479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:45.414606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310730+0000","last_change":"2024-04-25T16:50:54.047737+0000","last_active":"2024-04-25T16:51:00.310730+0000","last_peered":"2024-04-25T16:51:00.310730+0000","last_clean":"2024-04-25T16:51:00.310730+0000","last_became_active":"2024-04-25T16:50:54.047417+0000","last_became_peered":"2024-04-25T16:50:54.047417+0000","last_unstale":"2024-04-25T16:51:00.310730+0000","last_undegraded":"2024-04-25T16:51:00.310730+0000","last_fullsized":"2024-04-25T16:51:00.310730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.166309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153448+0000","last_change":"2024-04-25T16:50:55.064168+0000","last_active":"2024-04-25T16:51:00.153448+0000","last_peered":"2024-04-25T16:51:00.153448+0000","last_clean":"2024-04-25T16:51:00.153448+0000","last_became_active":"2024-04-25T16:50:55.062345+0000","last_became_peered":"2024-04-25T16:50:55.062345+0000","last_unstale":"2024-04-25T16:51:00.153448+0000","last_undegraded":"2024-04-25T16:51:00.153448+0000","last_fullsized":"2024-04-25T16:51:00.153448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:09.018645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311446+0000","last_change":"2024-04-25T16:50:59.143004+0000","last_active":"2024-04-25T16:51:00.311446+0000","last_peered":"2024-04-25T16:51:00.311446+0000","last_clean":"2024-04-25T16:51:00.311446+0000","last_became_active":"2024-04-25T16:50:59.142240+0000","last_became_peered":"2024-04-25T16:50:59.142240+0000","last_unstale":"2024-04-25T16:51:00.311446+0000","last_undegraded":"2024-04-25T16:51:00.311446+0000","last_fullsized":"2024-04-25T16:51:00.311446+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:19.230775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148672+0000","last_change":"2024-04-25T16:50:55.066558+0000","last_active":"2024-04-25T16:51:00.148672+0000","last_peered":"2024-04-25T16:51:00.148672+0000","last_clean":"2024-04-25T16:51:00.148672+0000","last_became_active":"2024-04-25T16:50:55.065201+0000","last_became_peered":"2024-04-25T16:50:55.065201+0000","last_unstale":"2024-04-25T16:51:00.148672+0000","last_undegraded":"2024-04-25T16:51:00.148672+0000","last_fullsized":"2024-04-25T16:51:00.148672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:12.518641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148555+0000","last_change":"2024-04-25T16:50:55.089086+0000","last_active":"2024-04-25T16:51:00.148555+0000","last_peered":"2024-04-25T16:51:00.148555+0000","last_clean":"2024-04-25T16:51:00.148555+0000","last_became_active":"2024-04-25T16:50:55.088788+0000","last_became_peered":"2024-04-25T16:50:55.088788+0000","last_unstale":"2024-04-25T16:51:00.148555+0000","last_undegraded":"2024-04-25T16:51:00.148555+0000","last_fullsized":"2024-04-25T16:51:00.148555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:22.053709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019230+0000","last_change":"2024-04-25T16:50:55.092706+0000","last_active":"2024-04-25T16:51:01.019230+0000","last_peered":"2024-04-25T16:51:01.019230+0000","last_clean":"2024-04-25T16:51:01.019230+0000","last_became_active":"2024-04-25T16:50:55.092411+0000","last_became_peered":"2024-04-25T16:50:55.092411+0000","last_unstale":"2024-04-25T16:51:01.019230+0000","last_undegraded":"2024-04-25T16:51:01.019230+0000","last_fullsized":"2024-04-25T16:51:01.019230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:56.360832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017429+0000","last_change":"2024-04-25T16:50:55.053386+0000","last_active":"2024-04-25T16:51:01.017429+0000","last_peered":"2024-04-25T16:51:01.017429+0000","last_clean":"2024-04-25T16:51:01.017429+0000","last_became_active":"2024-04-25T16:50:55.052555+0000","last_became_peered":"2024-04-25T16:50:55.052555+0000","last_unstale":"2024-04-25T16:51:01.017429+0000","last_undegraded":"2024-04-25T16:51:01.017429+0000","last_fullsized":"2024-04-25T16:51:01.017429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311617+0000","last_change":"2024-04-25T16:50:55.072013+0000","last_active":"2024-04-25T16:51:00.311617+0000","last_peered":"2024-04-25T16:51:00.311617+0000","last_clean":"2024-04-25T16:51:00.311617+0000","last_became_active":"2024-04-25T16:50:55.071089+0000","last_became_peered":"2024-04-25T16:50:55.071089+0000","last_unstale":"2024-04-25T16:51:00.311617+0000","last_undegraded":"2024-04-25T16:51:00.311617+0000","last_fullsized":"2024-04-25T16:51:00.311617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.304249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015430+0000","last_change":"2024-04-25T16:50:55.063809+0000","last_active":"2024-04-25T16:51:01.015430+0000","last_peered":"2024-04-25T16:51:01.015430+0000","last_clean":"2024-04-25T16:51:01.015430+0000","last_became_active":"2024-04-25T16:50:55.062913+0000","last_became_peered":"2024-04-25T16:50:55.062913+0000","last_unstale":"2024-04-25T16:51:01.015430+0000","last_undegraded":"2024-04-25T16:51:01.015430+0000","last_fullsized":"2024-04-25T16:51:01.015430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151698+0000","last_change":"2024-04-25T16:50:55.066745+0000","last_active":"2024-04-25T16:51:00.151698+0000","last_peered":"2024-04-25T16:51:00.151698+0000","last_clean":"2024-04-25T16:51:00.151698+0000","last_became_active":"2024-04-25T16:50:55.065075+0000","last_became_peered":"2024-04-25T16:50:55.065075+0000","last_unstale":"2024-04-25T16:51:00.151698+0000","last_undegraded":"2024-04-25T16:51:00.151698+0000","last_fullsized":"2024-04-25T16:51:00.151698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.312312+0000","last_change":"2024-04-25T16:50:55.081262+0000","last_active":"2024-04-25T16:51:00.312312+0000","last_peered":"2024-04-25T16:51:00.312312+0000","last_clean":"2024-04-25T16:51:00.312312+0000","last_became_active":"2024-04-25T16:50:55.080803+0000","last_became_peered":"2024-04-25T16:50:55.080803+0000","last_unstale":"2024-04-25T16:51:00.312312+0000","last_undegraded":"2024-04-25T16:51:00.312312+0000","last_fullsized":"2024-04-25T16:51:00.312312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:14.342043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017299+0000","last_change":"2024-04-25T16:50:55.090717+0000","last_active":"2024-04-25T16:51:01.017299+0000","last_peered":"2024-04-25T16:51:01.017299+0000","last_clean":"2024-04-25T16:51:01.017299+0000","last_became_active":"2024-04-25T16:50:55.090388+0000","last_became_peered":"2024-04-25T16:50:55.090388+0000","last_unstale":"2024-04-25T16:51:01.017299+0000","last_undegraded":"2024-04-25T16:51:01.017299+0000","last_fullsized":"2024-04-25T16:51:01.017299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310335+0000","last_change":"2024-04-25T16:50:55.072773+0000","last_active":"2024-04-25T16:51:00.310335+0000","last_peered":"2024-04-25T16:51:00.310335+0000","last_clean":"2024-04-25T16:51:00.310335+0000","last_became_active":"2024-04-25T16:50:55.071914+0000","last_became_peered":"2024-04-25T16:50:55.071914+0000","last_unstale":"2024-04-25T16:51:00.310335+0000","last_undegraded":"2024-04-25T16:51:00.310335+0000","last_fullsized":"2024-04-25T16:51:00.310335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018758+0000","last_change":"2024-04-25T16:50:55.093501+0000","last_active":"2024-04-25T16:51:01.018758+0000","last_peered":"2024-04-25T16:51:01.018758+0000","last_clean":"2024-04-25T16:51:01.018758+0000","last_became_active":"2024-04-25T16:50:55.093168+0000","last_became_peered":"2024-04-25T16:50:55.093168+0000","last_unstale":"2024-04-25T16:51:01.018758+0000","last_undegraded":"2024-04-25T16:51:01.018758+0000","last_fullsized":"2024-04-25T16:51:01.018758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:42.081728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017137+0000","last_change":"2024-04-25T16:50:55.066877+0000","last_active":"2024-04-25T16:51:01.017137+0000","last_peered":"2024-04-25T16:51:01.017137+0000","last_clean":"2024-04-25T16:51:01.017137+0000","last_became_active":"2024-04-25T16:50:55.066129+0000","last_became_peered":"2024-04-25T16:50:55.066129+0000","last_unstale":"2024-04-25T16:51:01.017137+0000","last_undegraded":"2024-04-25T16:51:01.017137+0000","last_fullsized":"2024-04-25T16:51:01.017137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:12.518641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150236+0000","last_change":"2024-04-25T16:50:55.052920+0000","last_active":"2024-04-25T16:51:00.150236+0000","last_peered":"2024-04-25T16:51:00.150236+0000","last_clean":"2024-04-25T16:51:00.150236+0000","last_became_active":"2024-04-25T16:50:55.047735+0000","last_became_peered":"2024-04-25T16:50:55.047735+0000","last_unstale":"2024-04-25T16:51:00.150236+0000","last_undegraded":"2024-04-25T16:51:00.150236+0000","last_fullsized":"2024-04-25T16:51:00.150236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020445+0000","last_change":"2024-04-25T16:50:55.090598+0000","last_active":"2024-04-25T16:51:01.020445+0000","last_peered":"2024-04-25T16:51:01.020445+0000","last_clean":"2024-04-25T16:51:01.020445+0000","last_became_active":"2024-04-25T16:50:55.090102+0000","last_became_peered":"2024-04-25T16:50:55.090102+0000","last_unstale":"2024-04-25T16:51:01.020445+0000","last_undegraded":"2024-04-25T16:51:01.020445+0000","last_fullsized":"2024-04-25T16:51:01.020445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:35.322477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150315+0000","last_change":"2024-04-25T16:50:55.081066+0000","last_active":"2024-04-25T16:51:00.150315+0000","last_peered":"2024-04-25T16:51:00.150315+0000","last_clean":"2024-04-25T16:51:00.150315+0000","last_became_active":"2024-04-25T16:50:55.069516+0000","last_became_peered":"2024-04-25T16:50:55.069516+0000","last_unstale":"2024-04-25T16:51:00.150315+0000","last_undegraded":"2024-04-25T16:51:00.150315+0000","last_fullsized":"2024-04-25T16:51:00.150315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:13.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019657+0000","last_change":"2024-04-25T16:50:55.093971+0000","last_active":"2024-04-25T16:51:01.019657+0000","last_peered":"2024-04-25T16:51:01.019657+0000","last_clean":"2024-04-25T16:51:01.019657+0000","last_became_active":"2024-04-25T16:50:55.093421+0000","last_became_peered":"2024-04-25T16:50:55.093421+0000","last_unstale":"2024-04-25T16:51:01.019657+0000","last_undegraded":"2024-04-25T16:51:01.019657+0000","last_fullsized":"2024-04-25T16:51:01.019657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:10.733287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016967+0000","last_change":"2024-04-25T16:50:55.075149+0000","last_active":"2024-04-25T16:51:01.016967+0000","last_peered":"2024-04-25T16:51:01.016967+0000","last_clean":"2024-04-25T16:51:01.016967+0000","last_became_active":"2024-04-25T16:50:55.074215+0000","last_became_peered":"2024-04-25T16:50:55.074215+0000","last_unstale":"2024-04-25T16:51:01.016967+0000","last_undegraded":"2024-04-25T16:51:01.016967+0000","last_fullsized":"2024-04-25T16:51:01.016967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015476+0000","last_change":"2024-04-25T16:50:55.072156+0000","last_active":"2024-04-25T16:51:01.015476+0000","last_peered":"2024-04-25T16:51:01.015476+0000","last_clean":"2024-04-25T16:51:01.015476+0000","last_became_active":"2024-04-25T16:50:55.071295+0000","last_became_peered":"2024-04-25T16:50:55.071295+0000","last_unstale":"2024-04-25T16:51:01.015476+0000","last_undegraded":"2024-04-25T16:51:01.015476+0000","last_fullsized":"2024-04-25T16:51:01.015476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018336+0000","last_change":"2024-04-25T16:50:55.062605+0000","last_active":"2024-04-25T16:51:01.018336+0000","last_peered":"2024-04-25T16:51:01.018336+0000","last_clean":"2024-04-25T16:51:01.018336+0000","last_became_active":"2024-04-25T16:50:55.061970+0000","last_became_peered":"2024-04-25T16:50:55.061970+0000","last_unstale":"2024-04-25T16:51:01.018336+0000","last_undegraded":"2024-04-25T16:51:01.018336+0000","last_fullsized":"2024-04-25T16:51:01.018336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:41.328339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310463+0000","last_change":"2024-04-25T16:50:55.062534+0000","last_active":"2024-04-25T16:51:00.310463+0000","last_peered":"2024-04-25T16:51:00.310463+0000","last_clean":"2024-04-25T16:51:00.310463+0000","last_became_active":"2024-04-25T16:50:55.061601+0000","last_became_peered":"2024-04-25T16:50:55.061601+0000","last_unstale":"2024-04-25T16:51:00.310463+0000","last_undegraded":"2024-04-25T16:51:00.310463+0000","last_fullsized":"2024-04-25T16:51:00.310463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154054+0000","last_change":"2024-04-25T16:50:55.067043+0000","last_active":"2024-04-25T16:51:00.154054+0000","last_peered":"2024-04-25T16:51:00.154054+0000","last_clean":"2024-04-25T16:51:00.154054+0000","last_became_active":"2024-04-25T16:50:55.066155+0000","last_became_peered":"2024-04-25T16:50:55.066155+0000","last_unstale":"2024-04-25T16:51:00.154054+0000","last_undegraded":"2024-04-25T16:51:00.154054+0000","last_fullsized":"2024-04-25T16:51:00.154054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:08.701059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150829+0000","last_change":"2024-04-25T16:50:55.084655+0000","last_active":"2024-04-25T16:51:00.150829+0000","last_peered":"2024-04-25T16:51:00.150829+0000","last_clean":"2024-04-25T16:51:00.150829+0000","last_became_active":"2024-04-25T16:50:55.083647+0000","last_became_peered":"2024-04-25T16:50:55.083647+0000","last_unstale":"2024-04-25T16:51:00.150829+0000","last_undegraded":"2024-04-25T16:51:00.150829+0000","last_fullsized":"2024-04-25T16:51:00.150829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151431+0000","last_change":"2024-04-25T16:50:55.085352+0000","last_active":"2024-04-25T16:51:00.151431+0000","last_peered":"2024-04-25T16:51:00.151431+0000","last_clean":"2024-04-25T16:51:00.151431+0000","last_became_active":"2024-04-25T16:50:55.083135+0000","last_became_peered":"2024-04-25T16:50:55.083135+0000","last_unstale":"2024-04-25T16:51:00.151431+0000","last_undegraded":"2024-04-25T16:51:00.151431+0000","last_fullsized":"2024-04-25T16:51:00.151431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.021207+0000","last_change":"2024-04-25T16:50:55.090981+0000","last_active":"2024-04-25T16:51:01.021207+0000","last_peered":"2024-04-25T16:51:01.021207+0000","last_clean":"2024-04-25T16:51:01.021207+0000","last_became_active":"2024-04-25T16:50:55.090507+0000","last_became_peered":"2024-04-25T16:50:55.090507+0000","last_unstale":"2024-04-25T16:51:01.021207+0000","last_undegraded":"2024-04-25T16:51:01.021207+0000","last_fullsized":"2024-04-25T16:51:01.021207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:10.733287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.309934+0000","last_change":"2024-04-25T16:50:55.090738+0000","last_active":"2024-04-25T16:51:00.309934+0000","last_peered":"2024-04-25T16:51:00.309934+0000","last_clean":"2024-04-25T16:51:00.309934+0000","last_became_active":"2024-04-25T16:50:55.090298+0000","last_became_peered":"2024-04-25T16:50:55.090298+0000","last_unstale":"2024-04-25T16:51:00.309934+0000","last_undegraded":"2024-04-25T16:51:00.309934+0000","last_fullsized":"2024-04-25T16:51:00.309934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:12.518641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017349+0000","last_change":"2024-04-25T16:50:55.083767+0000","last_active":"2024-04-25T16:51:01.017349+0000","last_peered":"2024-04-25T16:51:01.017349+0000","last_clean":"2024-04-25T16:51:01.017349+0000","last_became_active":"2024-04-25T16:50:55.083468+0000","last_became_peered":"2024-04-25T16:50:55.083468+0000","last_unstale":"2024-04-25T16:51:01.017349+0000","last_undegraded":"2024-04-25T16:51:01.017349+0000","last_fullsized":"2024-04-25T16:51:01.017349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.304249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018383+0000","last_change":"2024-04-25T16:50:55.083971+0000","last_active":"2024-04-25T16:51:01.018383+0000","last_peered":"2024-04-25T16:51:01.018383+0000","last_clean":"2024-04-25T16:51:01.018383+0000","last_became_active":"2024-04-25T16:50:55.083688+0000","last_became_peered":"2024-04-25T16:50:55.083688+0000","last_unstale":"2024-04-25T16:51:01.018383+0000","last_undegraded":"2024-04-25T16:51:01.018383+0000","last_fullsized":"2024-04-25T16:51:01.018383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:08.427466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018722+0000","last_change":"2024-04-25T16:50:55.090861+0000","last_active":"2024-04-25T16:51:01.018722+0000","last_peered":"2024-04-25T16:51:01.018722+0000","last_clean":"2024-04-25T16:51:01.018722+0000","last_became_active":"2024-04-25T16:50:55.089863+0000","last_became_peered":"2024-04-25T16:50:55.089863+0000","last_unstale":"2024-04-25T16:51:01.018722+0000","last_undegraded":"2024-04-25T16:51:01.018722+0000","last_fullsized":"2024-04-25T16:51:01.018722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:14.342043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018454+0000","last_change":"2024-04-25T16:50:55.091645+0000","last_active":"2024-04-25T16:51:01.018454+0000","last_peered":"2024-04-25T16:51:01.018454+0000","last_clean":"2024-04-25T16:51:01.018454+0000","last_became_active":"2024-04-25T16:50:55.091308+0000","last_became_peered":"2024-04-25T16:50:55.091308+0000","last_unstale":"2024-04-25T16:51:01.018454+0000","last_undegraded":"2024-04-25T16:51:01.018454+0000","last_fullsized":"2024-04-25T16:51:01.018454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017510+0000","last_change":"2024-04-25T16:50:55.074041+0000","last_active":"2024-04-25T16:51:01.017510+0000","last_peered":"2024-04-25T16:51:01.017510+0000","last_clean":"2024-04-25T16:51:01.017510+0000","last_became_active":"2024-04-25T16:50:55.073142+0000","last_became_peered":"2024-04-25T16:50:55.073142+0000","last_unstale":"2024-04-25T16:51:01.017510+0000","last_undegraded":"2024-04-25T16:51:01.017510+0000","last_fullsized":"2024-04-25T16:51:01.017510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016248+0000","last_change":"2024-04-25T16:50:55.065163+0000","last_active":"2024-04-25T16:51:01.016248+0000","last_peered":"2024-04-25T16:51:01.016248+0000","last_clean":"2024-04-25T16:51:01.016248+0000","last_became_active":"2024-04-25T16:50:55.064257+0000","last_became_peered":"2024-04-25T16:50:55.064257+0000","last_unstale":"2024-04-25T16:51:01.016248+0000","last_undegraded":"2024-04-25T16:51:01.016248+0000","last_fullsized":"2024-04-25T16:51:01.016248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020950+0000","last_change":"2024-04-25T16:50:55.082932+0000","last_active":"2024-04-25T16:51:01.020950+0000","last_peered":"2024-04-25T16:51:01.020950+0000","last_clean":"2024-04-25T16:51:01.020950+0000","last_became_active":"2024-04-25T16:50:55.082589+0000","last_became_peered":"2024-04-25T16:50:55.082589+0000","last_unstale":"2024-04-25T16:51:01.020950+0000","last_undegraded":"2024-04-25T16:51:01.020950+0000","last_fullsized":"2024-04-25T16:51:01.020950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:56.360832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335656,"statfs":{"total":96636764160,"available":96599711744,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:51:19.905 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:51:20.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24beaaa640 1 Processor -- start 2024-04-25T16:51:20.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24beaaa640 1 -- start start 2024-04-25T16:51:20.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24beaaa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f24b80cdcb0 0x7f24b8072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24beaaa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f24b80b7df0 0x7f24b8161c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24beaaa640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24b8073090 con 0x7f24b8072c80 2024-04-25T16:51:20.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24beaaa640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f24b80d2120 con 0x7f24b80cdcb0 2024-04-25T16:51:20.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24beaaa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f24b80d2400 con 0x7f24b80b7df0 2024-04-25T16:51:20.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24bc846640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f24b80cdcb0 0x7f24b8072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24b7fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f24b8072c80 0x7f24b8161580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38996/0 (socket says 172.21.15.94:38996) 2024-04-25T16:51:20.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24b7fff640 1 -- 172.21.15.94:0/1425014610 learned_addr learned my addr 172.21.15.94:0/1425014610 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:20.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24bd047640 1 --2- 172.21.15.94:0/1425014610 >> v2:172.21.15.138:3300/0 conn(0x7f24b80b7df0 0x7f24b8161c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24bc846640 1 -- 172.21.15.94:0/1425014610 >> v2:172.21.15.138:3300/0 conn(0x7f24b80b7df0 msgr2=0x7f24b8161c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:20.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24bc846640 1 --2- 172.21.15.94:0/1425014610 >> v2:172.21.15.138:3300/0 conn(0x7f24b80b7df0 0x7f24b8161c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24bc846640 1 -- 172.21.15.94:0/1425014610 >> v1:172.21.15.94:6789/0 conn(0x7f24b8072c80 legacy=0x7f24b8161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24b77fe640 1 -- 172.21.15.94:0/1425014610 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2863808170 0 0) 0x7f24ac001120 con 0x7f24b8072c80 2024-04-25T16:51:20.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.096+0000 7f24bc846640 1 -- 172.21.15.94:0/1425014610 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b80d26e0 con 0x7f24b80cdcb0 2024-04-25T16:51:20.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.100+0000 7f24bc846640 1 --2- 172.21.15.94:0/1425014610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80cdcb0 0x7f24b8072740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24b80d29c0 tx=0x7f24a801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4a0727f9124881 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:20.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.100+0000 7f24b77fe640 1 -- 172.21.15.94:0/1425014610 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80b1070 con 0x7f24b80cdcb0 2024-04-25T16:51:20.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.100+0000 7f24b77fe640 1 -- 172.21.15.94:0/1425014610 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a80ac600 con 0x7f24b80cdcb0 2024-04-25T16:51:20.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.100+0000 7f24b77fe640 1 -- 172.21.15.94:0/1425014610 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80ac9d0 con 0x7f24b80cdcb0 2024-04-25T16:51:20.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.100+0000 7f24beaaa640 1 -- 172.21.15.94:0/1425014610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80cdcb0 msgr2=0x7f24b8072740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.100+0000 7f24beaaa640 1 --2- 172.21.15.94:0/1425014610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80cdcb0 0x7f24b8072740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24b80d29c0 tx=0x7f24a801c3c0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.100+0000 7f24beaaa640 1 -- 172.21.15.94:0/1425014610 shutdown_connections 2024-04-25T16:51:20.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.100+0000 7f24beaaa640 1 --2- 172.21.15.94:0/1425014610 >> v2:172.21.15.138:3300/0 conn(0x7f24b80b7df0 0x7f24b8161c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.100+0000 7f24beaaa640 1 --2- 172.21.15.94:0/1425014610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80cdcb0 0x7f24b8072740 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.100+0000 7f24beaaa640 1 -- 172.21.15.94:0/1425014610 >> 172.21.15.94:0/1425014610 conn(0x7f24b80d3850 msgr2=0x7f24b80d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:20.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.100+0000 7f24beaaa640 1 -- 172.21.15.94:0/1425014610 shutdown_connections 2024-04-25T16:51:20.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24beaaa640 1 -- 172.21.15.94:0/1425014610 wait complete. 2024-04-25T16:51:20.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24beaaa640 1 Processor -- start 2024-04-25T16:51:20.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24beaaa640 1 -- start start 2024-04-25T16:51:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24beaaa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b8072c80 0x7f24b815fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24beaaa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b80b7df0 0x7f24b81597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24beaaa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24b80cdcb0 0x7f24b815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24beaaa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24b80b6680 con 0x7f24b8072c80 2024-04-25T16:51:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24beaaa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24b80b6500 con 0x7f24b80b7df0 2024-04-25T16:51:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24beaaa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f24b80ce0c0 con 0x7f24b80cdcb0 2024-04-25T16:51:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24bc846640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b8072c80 0x7f24b815fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24bc846640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b8072c80 0x7f24b815fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41962/0 (socket says 172.21.15.94:41962) 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24bc846640 1 -- 172.21.15.94:0/2073666906 learned_addr learned my addr 172.21.15.94:0/2073666906 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24b7fff640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b80b7df0 0x7f24b81597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24bd047640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24b80cdcb0 0x7f24b815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24bc846640 1 -- 172.21.15.94:0/2073666906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24b80cdcb0 msgr2=0x7f24b815c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24bc846640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24b80cdcb0 0x7f24b815c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24bc846640 1 -- 172.21.15.94:0/2073666906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b80b7df0 msgr2=0x7f24b81597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24bc846640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b80b7df0 0x7f24b81597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24bc846640 1 -- 172.21.15.94:0/2073666906 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b8160210 con 0x7f24b8072c80 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24b7fff640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b80b7df0 0x7f24b81597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.104+0000 7f24bc846640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b8072c80 0x7f24b815fcd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f24a8018fd0 tx=0x7f24a801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.108+0000 7f24b57fa640 1 -- 172.21.15.94:0/2073666906 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80b1040 con 0x7f24b8072c80 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.108+0000 7f24b57fa640 1 -- 172.21.15.94:0/2073666906 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a80bc070 con 0x7f24b8072c80 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.108+0000 7f24beaaa640 1 -- 172.21.15.94:0/2073666906 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b8152a90 con 0x7f24b8072c80 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.108+0000 7f24b57fa640 1 -- 172.21.15.94:0/2073666906 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80b7670 con 0x7f24b8072c80 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.108+0000 7f24beaaa640 1 -- 172.21.15.94:0/2073666906 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b8153050 con 0x7f24b8072c80 2024-04-25T16:51:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.108+0000 7f24beaaa640 1 -- 172.21.15.94:0/2073666906 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b806a730 con 0x7f24b8072c80 2024-04-25T16:51:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.108+0000 7f24b57fa640 1 -- 172.21.15.94:0/2073666906 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f24a80b7810 con 0x7f24b8072c80 2024-04-25T16:51:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.108+0000 7f24b57fa640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f24a406f400 0x7f24a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.108+0000 7f24b57fa640 1 -- 172.21.15.94:0/2073666906 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f24a8129430 con 0x7f24b8072c80 2024-04-25T16:51:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.108+0000 7f24b7fff640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f24a406f400 0x7f24a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.108+0000 7f24b7fff640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f24a406f400 0x7f24a4071910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24b80b7b40 tx=0x7f24ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.112+0000 7f24b57fa640 1 -- 172.21.15.94:0/2073666906 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f24a80c5020 con 0x7f24b8072c80 2024-04-25T16:51:20.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.272+0000 7f24beaaa640 1 -- 172.21.15.94:0/2073666906 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f24b81525d0 con 0x7f24a406f400 2024-04-25T16:51:20.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.280+0000 7f24b57fa640 1 -- 172.21.15.94:0/2073666906 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332527 (secure 0 0 0) 0x7f24ac001c70 con 0x7f24a406f400 2024-04-25T16:51:20.279 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:51:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.284+0000 7f24beaaa640 1 -- 172.21.15.94:0/2073666906 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f24a406f400 msgr2=0x7f24a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.284+0000 7f24beaaa640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f24a406f400 0x7f24a4071910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24b80b7b40 tx=0x7f24ac009040 comp rx=0 tx=0).stop 2024-04-25T16:51:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.284+0000 7f24beaaa640 1 -- 172.21.15.94:0/2073666906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b8072c80 msgr2=0x7f24b815fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.284+0000 7f24beaaa640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b8072c80 0x7f24b815fcd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f24a8018fd0 tx=0x7f24a801ac00 comp rx=0 tx=0).stop 2024-04-25T16:51:20.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.284+0000 7f24beaaa640 1 -- 172.21.15.94:0/2073666906 shutdown_connections 2024-04-25T16:51:20.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.284+0000 7f24beaaa640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f24a406f400 0x7f24a4071910 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.284+0000 7f24beaaa640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24b80cdcb0 0x7f24b815c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.284+0000 7f24beaaa640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b80b7df0 0x7f24b81597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.284+0000 7f24beaaa640 1 --2- 172.21.15.94:0/2073666906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b8072c80 0x7f24b815fcd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.288+0000 7f24beaaa640 1 -- 172.21.15.94:0/2073666906 >> 172.21.15.94:0/2073666906 conn(0x7f24b80d3850 msgr2=0x7f24b8156d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:20.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.288+0000 7f24beaaa640 1 -- 172.21.15.94:0/2073666906 shutdown_connections 2024-04-25T16:51:20.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.288+0000 7f24beaaa640 1 -- 172.21.15.94:0/2073666906 wait complete. 2024-04-25T16:51:20.303 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-25T16:51:19.670638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240164,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734556,"statfs":{"total":773094113280,"available":772848185344,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006024"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018387+0000","last_change":"2024-04-25T16:50:54.047200+0000","last_active":"2024-04-25T16:51:01.018387+0000","last_peered":"2024-04-25T16:51:01.018387+0000","last_clean":"2024-04-25T16:51:01.018387+0000","last_became_active":"2024-04-25T16:50:54.046862+0000","last_became_peered":"2024-04-25T16:50:54.046862+0000","last_unstale":"2024-04-25T16:51:01.018387+0000","last_undegraded":"2024-04-25T16:51:01.018387+0000","last_fullsized":"2024-04-25T16:51:01.018387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.166309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311306+0000","last_change":"2024-04-25T16:50:55.060286+0000","last_active":"2024-04-25T16:51:00.311306+0000","last_peered":"2024-04-25T16:51:00.311306+0000","last_clean":"2024-04-25T16:51:00.311306+0000","last_became_active":"2024-04-25T16:50:55.059171+0000","last_became_peered":"2024-04-25T16:50:55.059171+0000","last_unstale":"2024-04-25T16:51:00.311306+0000","last_undegraded":"2024-04-25T16:51:00.311306+0000","last_fullsized":"2024-04-25T16:51:00.311306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:08.427466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310185+0000","last_change":"2024-04-25T16:50:59.181612+0000","last_active":"2024-04-25T16:51:00.310185+0000","last_peered":"2024-04-25T16:51:00.310185+0000","last_clean":"2024-04-25T16:51:00.310185+0000","last_became_active":"2024-04-25T16:50:59.180865+0000","last_became_peered":"2024-04-25T16:50:59.180865+0000","last_unstale":"2024-04-25T16:51:00.310185+0000","last_undegraded":"2024-04-25T16:51:00.310185+0000","last_fullsized":"2024-04-25T16:51:00.310185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:59.449855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311587+0000","last_change":"2024-04-25T16:50:54.040413+0000","last_active":"2024-04-25T16:51:00.311587+0000","last_peered":"2024-04-25T16:51:00.311587+0000","last_clean":"2024-04-25T16:51:00.311587+0000","last_became_active":"2024-04-25T16:50:54.040072+0000","last_became_peered":"2024-04-25T16:50:54.040072+0000","last_unstale":"2024-04-25T16:51:00.311587+0000","last_undegraded":"2024-04-25T16:51:00.311587+0000","last_fullsized":"2024-04-25T16:51:00.311587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:24.611706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018056+0000","last_change":"2024-04-25T16:50:55.083877+0000","last_active":"2024-04-25T16:51:01.018056+0000","last_peered":"2024-04-25T16:51:01.018056+0000","last_clean":"2024-04-25T16:51:01.018056+0000","last_became_active":"2024-04-25T16:50:55.083529+0000","last_became_peered":"2024-04-25T16:50:55.083529+0000","last_unstale":"2024-04-25T16:51:01.018056+0000","last_undegraded":"2024-04-25T16:51:01.018056+0000","last_fullsized":"2024-04-25T16:51:01.018056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152476+0000","last_change":"2024-04-25T16:50:59.143706+0000","last_active":"2024-04-25T16:51:00.152476+0000","last_peered":"2024-04-25T16:51:00.152476+0000","last_clean":"2024-04-25T16:51:00.152476+0000","last_became_active":"2024-04-25T16:50:59.142488+0000","last_became_peered":"2024-04-25T16:50:59.142488+0000","last_unstale":"2024-04-25T16:51:00.152476+0000","last_undegraded":"2024-04-25T16:51:00.152476+0000","last_fullsized":"2024-04-25T16:51:00.152476+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:20.203920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020308+0000","last_change":"2024-04-25T16:50:54.061175+0000","last_active":"2024-04-25T16:51:01.020308+0000","last_peered":"2024-04-25T16:51:01.020308+0000","last_clean":"2024-04-25T16:51:01.020308+0000","last_became_active":"2024-04-25T16:50:54.059991+0000","last_became_peered":"2024-04-25T16:50:54.059991+0000","last_unstale":"2024-04-25T16:51:01.020308+0000","last_undegraded":"2024-04-25T16:51:01.020308+0000","last_fullsized":"2024-04-25T16:51:01.020308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018692+0000","last_change":"2024-04-25T16:50:55.087495+0000","last_active":"2024-04-25T16:51:01.018692+0000","last_peered":"2024-04-25T16:51:01.018692+0000","last_clean":"2024-04-25T16:51:01.018692+0000","last_became_active":"2024-04-25T16:50:55.087177+0000","last_became_peered":"2024-04-25T16:50:55.087177+0000","last_unstale":"2024-04-25T16:51:01.018692+0000","last_undegraded":"2024-04-25T16:51:01.018692+0000","last_fullsized":"2024-04-25T16:51:01.018692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154110+0000","last_change":"2024-04-25T16:50:59.153145+0000","last_active":"2024-04-25T16:51:00.154110+0000","last_peered":"2024-04-25T16:51:00.154110+0000","last_clean":"2024-04-25T16:51:00.154110+0000","last_became_active":"2024-04-25T16:50:59.151443+0000","last_became_peered":"2024-04-25T16:50:59.151443+0000","last_unstale":"2024-04-25T16:51:00.154110+0000","last_undegraded":"2024-04-25T16:51:00.154110+0000","last_fullsized":"2024-04-25T16:51:00.154110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:08.117787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019564+0000","last_change":"2024-04-25T16:50:54.064208+0000","last_active":"2024-04-25T16:51:01.019564+0000","last_peered":"2024-04-25T16:51:01.019564+0000","last_clean":"2024-04-25T16:51:01.019564+0000","last_became_active":"2024-04-25T16:50:54.063734+0000","last_became_peered":"2024-04-25T16:50:54.063734+0000","last_unstale":"2024-04-25T16:51:01.019564+0000","last_undegraded":"2024-04-25T16:51:01.019564+0000","last_fullsized":"2024-04-25T16:51:01.019564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018436+0000","last_change":"2024-04-25T16:50:55.087482+0000","last_active":"2024-04-25T16:51:01.018436+0000","last_peered":"2024-04-25T16:51:01.018436+0000","last_clean":"2024-04-25T16:51:01.018436+0000","last_became_active":"2024-04-25T16:50:55.087139+0000","last_became_peered":"2024-04-25T16:50:55.087139+0000","last_unstale":"2024-04-25T16:51:01.018436+0000","last_undegraded":"2024-04-25T16:51:01.018436+0000","last_fullsized":"2024-04-25T16:51:01.018436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.304249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152921+0000","last_change":"2024-04-25T16:50:59.142658+0000","last_active":"2024-04-25T16:51:00.152921+0000","last_peered":"2024-04-25T16:51:00.152921+0000","last_clean":"2024-04-25T16:51:00.152921+0000","last_became_active":"2024-04-25T16:50:59.139343+0000","last_became_peered":"2024-04-25T16:50:59.139343+0000","last_unstale":"2024-04-25T16:51:00.152921+0000","last_undegraded":"2024-04-25T16:51:00.152921+0000","last_fullsized":"2024-04-25T16:51:00.152921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:45.390516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149327+0000","last_change":"2024-04-25T16:50:55.049255+0000","last_active":"2024-04-25T16:51:00.149327+0000","last_peered":"2024-04-25T16:51:00.149327+0000","last_clean":"2024-04-25T16:51:00.149327+0000","last_became_active":"2024-04-25T16:50:55.047635+0000","last_became_peered":"2024-04-25T16:50:55.047635+0000","last_unstale":"2024-04-25T16:51:00.149327+0000","last_undegraded":"2024-04-25T16:51:00.149327+0000","last_fullsized":"2024-04-25T16:51:00.149327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019735+0000","last_change":"2024-04-25T16:50:54.056721+0000","last_active":"2024-04-25T16:51:01.019735+0000","last_peered":"2024-04-25T16:51:01.019735+0000","last_clean":"2024-04-25T16:51:01.019735+0000","last_became_active":"2024-04-25T16:50:54.055845+0000","last_became_peered":"2024-04-25T16:50:54.055845+0000","last_unstale":"2024-04-25T16:51:01.019735+0000","last_undegraded":"2024-04-25T16:51:01.019735+0000","last_fullsized":"2024-04-25T16:51:01.019735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.077660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153494+0000","last_change":"2024-04-25T16:50:59.152269+0000","last_active":"2024-04-25T16:51:00.153494+0000","last_peered":"2024-04-25T16:51:00.153494+0000","last_clean":"2024-04-25T16:51:00.153494+0000","last_became_active":"2024-04-25T16:50:59.150778+0000","last_became_peered":"2024-04-25T16:50:59.150778+0000","last_unstale":"2024-04-25T16:51:00.153494+0000","last_undegraded":"2024-04-25T16:51:00.153494+0000","last_fullsized":"2024-04-25T16:51:00.153494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:12.513733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310585+0000","last_change":"2024-04-25T16:50:55.072126+0000","last_active":"2024-04-25T16:51:00.310585+0000","last_peered":"2024-04-25T16:51:00.310585+0000","last_clean":"2024-04-25T16:51:00.310585+0000","last_became_active":"2024-04-25T16:50:55.071191+0000","last_became_peered":"2024-04-25T16:50:55.071191+0000","last_unstale":"2024-04-25T16:51:00.310585+0000","last_undegraded":"2024-04-25T16:51:00.310585+0000","last_fullsized":"2024-04-25T16:51:00.310585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015930+0000","last_change":"2024-04-25T16:50:54.033554+0000","last_active":"2024-04-25T16:51:01.015930+0000","last_peered":"2024-04-25T16:51:01.015930+0000","last_clean":"2024-04-25T16:51:01.015930+0000","last_became_active":"2024-04-25T16:50:54.032467+0000","last_became_peered":"2024-04-25T16:50:54.032467+0000","last_unstale":"2024-04-25T16:51:01.015930+0000","last_undegraded":"2024-04-25T16:51:01.015930+0000","last_fullsized":"2024-04-25T16:51:01.015930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.077660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153884+0000","last_change":"2024-04-25T16:50:59.157126+0000","last_active":"2024-04-25T16:51:00.153884+0000","last_peered":"2024-04-25T16:51:00.153884+0000","last_clean":"2024-04-25T16:51:00.153884+0000","last_became_active":"2024-04-25T16:50:59.153352+0000","last_became_peered":"2024-04-25T16:50:59.153352+0000","last_unstale":"2024-04-25T16:51:00.153884+0000","last_undegraded":"2024-04-25T16:51:00.153884+0000","last_fullsized":"2024-04-25T16:51:00.153884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:18.428310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310685+0000","last_change":"2024-04-25T16:50:55.059831+0000","last_active":"2024-04-25T16:51:00.310685+0000","last_peered":"2024-04-25T16:51:00.310685+0000","last_clean":"2024-04-25T16:51:00.310685+0000","last_became_active":"2024-04-25T16:50:55.058958+0000","last_became_peered":"2024-04-25T16:50:55.058958+0000","last_unstale":"2024-04-25T16:51:00.310685+0000","last_undegraded":"2024-04-25T16:51:00.310685+0000","last_fullsized":"2024-04-25T16:51:00.310685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015662+0000","last_change":"2024-04-25T16:50:54.045690+0000","last_active":"2024-04-25T16:51:01.015662+0000","last_peered":"2024-04-25T16:51:01.015662+0000","last_clean":"2024-04-25T16:51:01.015662+0000","last_became_active":"2024-04-25T16:50:54.045057+0000","last_became_peered":"2024-04-25T16:50:54.045057+0000","last_unstale":"2024-04-25T16:51:01.015662+0000","last_undegraded":"2024-04-25T16:51:01.015662+0000","last_fullsized":"2024-04-25T16:51:01.015662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:27.893510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016104+0000","last_change":"2024-04-25T16:50:59.163126+0000","last_active":"2024-04-25T16:51:01.016104+0000","last_peered":"2024-04-25T16:51:01.016104+0000","last_clean":"2024-04-25T16:51:01.016104+0000","last_became_active":"2024-04-25T16:50:59.162606+0000","last_became_peered":"2024-04-25T16:50:59.162606+0000","last_unstale":"2024-04-25T16:51:01.016104+0000","last_undegraded":"2024-04-25T16:51:01.016104+0000","last_fullsized":"2024-04-25T16:51:01.016104+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.992576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153472+0000","last_change":"2024-04-25T16:50:55.079857+0000","last_active":"2024-04-25T16:51:00.153472+0000","last_peered":"2024-04-25T16:51:00.153472+0000","last_clean":"2024-04-25T16:51:00.153472+0000","last_became_active":"2024-04-25T16:50:55.070021+0000","last_became_peered":"2024-04-25T16:50:55.070021+0000","last_unstale":"2024-04-25T16:51:00.153472+0000","last_undegraded":"2024-04-25T16:51:00.153472+0000","last_fullsized":"2024-04-25T16:51:00.153472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:00.753427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017068+0000","last_change":"2024-04-25T16:50:54.050226+0000","last_active":"2024-04-25T16:51:01.017068+0000","last_peered":"2024-04-25T16:51:01.017068+0000","last_clean":"2024-04-25T16:51:01.017068+0000","last_became_active":"2024-04-25T16:50:54.049451+0000","last_became_peered":"2024-04-25T16:50:54.049451+0000","last_unstale":"2024-04-25T16:51:01.017068+0000","last_undegraded":"2024-04-25T16:51:01.017068+0000","last_fullsized":"2024-04-25T16:51:01.017068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:26.942742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154683+0000","last_change":"2024-04-25T16:50:59.152071+0000","last_active":"2024-04-25T16:51:00.154683+0000","last_peered":"2024-04-25T16:51:00.154683+0000","last_clean":"2024-04-25T16:51:00.154683+0000","last_became_active":"2024-04-25T16:50:59.149014+0000","last_became_peered":"2024-04-25T16:50:59.149014+0000","last_unstale":"2024-04-25T16:51:00.154683+0000","last_undegraded":"2024-04-25T16:51:00.154683+0000","last_fullsized":"2024-04-25T16:51:00.154683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:00.447099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018096+0000","last_change":"2024-04-25T16:50:55.057085+0000","last_active":"2024-04-25T16:51:01.018096+0000","last_peered":"2024-04-25T16:51:01.018096+0000","last_clean":"2024-04-25T16:51:01.018096+0000","last_became_active":"2024-04-25T16:50:55.056254+0000","last_became_peered":"2024-04-25T16:50:55.056254+0000","last_unstale":"2024-04-25T16:51:01.018096+0000","last_undegraded":"2024-04-25T16:51:01.018096+0000","last_fullsized":"2024-04-25T16:51:01.018096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151405+0000","last_change":"2024-04-25T16:50:54.047574+0000","last_active":"2024-04-25T16:51:00.151405+0000","last_peered":"2024-04-25T16:51:00.151405+0000","last_clean":"2024-04-25T16:51:00.151405+0000","last_became_active":"2024-04-25T16:50:54.047164+0000","last_became_peered":"2024-04-25T16:50:54.047164+0000","last_unstale":"2024-04-25T16:51:00.151405+0000","last_undegraded":"2024-04-25T16:51:00.151405+0000","last_fullsized":"2024-04-25T16:51:00.151405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152691+0000","last_change":"2024-04-25T16:50:59.181342+0000","last_active":"2024-04-25T16:51:00.152691+0000","last_peered":"2024-04-25T16:51:00.152691+0000","last_clean":"2024-04-25T16:51:00.152691+0000","last_became_active":"2024-04-25T16:50:59.180010+0000","last_became_peered":"2024-04-25T16:50:59.180010+0000","last_unstale":"2024-04-25T16:51:00.152691+0000","last_undegraded":"2024-04-25T16:51:00.152691+0000","last_fullsized":"2024-04-25T16:51:00.152691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:09.707352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.312597+0000","last_change":"2024-04-25T16:50:55.093718+0000","last_active":"2024-04-25T16:51:00.312597+0000","last_peered":"2024-04-25T16:51:00.312597+0000","last_clean":"2024-04-25T16:51:00.312597+0000","last_became_active":"2024-04-25T16:50:55.093376+0000","last_became_peered":"2024-04-25T16:50:55.093376+0000","last_unstale":"2024-04-25T16:51:00.312597+0000","last_undegraded":"2024-04-25T16:51:00.312597+0000","last_fullsized":"2024-04-25T16:51:00.312597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:04.031520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016325+0000","last_change":"2024-04-25T16:50:54.040703+0000","last_active":"2024-04-25T16:51:01.016325+0000","last_peered":"2024-04-25T16:51:01.016325+0000","last_clean":"2024-04-25T16:51:01.016325+0000","last_became_active":"2024-04-25T16:50:54.039961+0000","last_became_peered":"2024-04-25T16:50:54.039961+0000","last_unstale":"2024-04-25T16:51:01.016325+0000","last_undegraded":"2024-04-25T16:51:01.016325+0000","last_fullsized":"2024-04-25T16:51:01.016325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310069+0000","last_change":"2024-04-25T16:50:59.161678+0000","last_active":"2024-04-25T16:51:00.310069+0000","last_peered":"2024-04-25T16:51:00.310069+0000","last_clean":"2024-04-25T16:51:00.310069+0000","last_became_active":"2024-04-25T16:50:59.161064+0000","last_became_peered":"2024-04-25T16:50:59.161064+0000","last_unstale":"2024-04-25T16:51:00.310069+0000","last_undegraded":"2024-04-25T16:51:00.310069+0000","last_fullsized":"2024-04-25T16:51:00.310069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:19.854569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015648+0000","last_change":"2024-04-25T16:50:55.069738+0000","last_active":"2024-04-25T16:51:01.015648+0000","last_peered":"2024-04-25T16:51:01.015648+0000","last_clean":"2024-04-25T16:51:01.015648+0000","last_became_active":"2024-04-25T16:50:55.068864+0000","last_became_peered":"2024-04-25T16:50:55.068864+0000","last_unstale":"2024-04-25T16:51:01.015648+0000","last_undegraded":"2024-04-25T16:51:01.015648+0000","last_fullsized":"2024-04-25T16:51:01.015648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020503+0000","last_change":"2024-04-25T16:50:54.061758+0000","last_active":"2024-04-25T16:51:01.020503+0000","last_peered":"2024-04-25T16:51:01.020503+0000","last_clean":"2024-04-25T16:51:01.020503+0000","last_became_active":"2024-04-25T16:50:54.061267+0000","last_became_peered":"2024-04-25T16:50:54.061267+0000","last_unstale":"2024-04-25T16:51:01.020503+0000","last_undegraded":"2024-04-25T16:51:01.020503+0000","last_fullsized":"2024-04-25T16:51:01.020503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:21.043343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017543+0000","last_change":"2024-04-25T16:50:59.174520+0000","last_active":"2024-04-25T16:51:01.017543+0000","last_peered":"2024-04-25T16:51:01.017543+0000","last_clean":"2024-04-25T16:51:01.017543+0000","last_became_active":"2024-04-25T16:50:59.173812+0000","last_became_peered":"2024-04-25T16:50:59.173812+0000","last_unstale":"2024-04-25T16:51:01.017543+0000","last_undegraded":"2024-04-25T16:51:01.017543+0000","last_fullsized":"2024-04-25T16:51:01.017543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:29.939373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018748+0000","last_change":"2024-04-25T16:50:55.090616+0000","last_active":"2024-04-25T16:51:01.018748+0000","last_peered":"2024-04-25T16:51:01.018748+0000","last_clean":"2024-04-25T16:51:01.018748+0000","last_became_active":"2024-04-25T16:50:55.090287+0000","last_became_peered":"2024-04-25T16:50:55.090287+0000","last_unstale":"2024-04-25T16:51:01.018748+0000","last_undegraded":"2024-04-25T16:51:01.018748+0000","last_fullsized":"2024-04-25T16:51:01.018748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:04.031520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015756+0000","last_change":"2024-04-25T16:50:54.023906+0000","last_active":"2024-04-25T16:51:01.015756+0000","last_peered":"2024-04-25T16:51:01.015756+0000","last_clean":"2024-04-25T16:51:01.015756+0000","last_became_active":"2024-04-25T16:50:54.022198+0000","last_became_peered":"2024-04-25T16:50:54.022198+0000","last_unstale":"2024-04-25T16:51:01.015756+0000","last_undegraded":"2024-04-25T16:51:01.015756+0000","last_fullsized":"2024-04-25T16:51:01.015756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015375+0000","last_change":"2024-04-25T16:50:59.153325+0000","last_active":"2024-04-25T16:51:01.015375+0000","last_peered":"2024-04-25T16:51:01.015375+0000","last_clean":"2024-04-25T16:51:01.015375+0000","last_became_active":"2024-04-25T16:50:59.152431+0000","last_became_peered":"2024-04-25T16:50:59.152431+0000","last_unstale":"2024-04-25T16:51:01.015375+0000","last_undegraded":"2024-04-25T16:51:01.015375+0000","last_fullsized":"2024-04-25T16:51:01.015375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.819554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154818+0000","last_change":"2024-04-25T16:50:55.058098+0000","last_active":"2024-04-25T16:51:00.154818+0000","last_peered":"2024-04-25T16:51:00.154818+0000","last_clean":"2024-04-25T16:51:00.154818+0000","last_became_active":"2024-04-25T16:50:55.057233+0000","last_became_peered":"2024-04-25T16:50:55.057233+0000","last_unstale":"2024-04-25T16:51:00.154818+0000","last_undegraded":"2024-04-25T16:51:00.154818+0000","last_fullsized":"2024-04-25T16:51:00.154818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:36.088026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019426+0000","last_change":"2024-04-25T16:50:54.059732+0000","last_active":"2024-04-25T16:51:01.019426+0000","last_peered":"2024-04-25T16:51:01.019426+0000","last_clean":"2024-04-25T16:51:01.019426+0000","last_became_active":"2024-04-25T16:50:54.058805+0000","last_became_peered":"2024-04-25T16:50:54.058805+0000","last_unstale":"2024-04-25T16:51:01.019426+0000","last_undegraded":"2024-04-25T16:51:01.019426+0000","last_fullsized":"2024-04-25T16:51:01.019426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015551+0000","last_change":"2024-04-25T16:50:59.157551+0000","last_active":"2024-04-25T16:51:01.015551+0000","last_peered":"2024-04-25T16:51:01.015551+0000","last_clean":"2024-04-25T16:51:01.015551+0000","last_became_active":"2024-04-25T16:50:59.156077+0000","last_became_peered":"2024-04-25T16:50:59.156077+0000","last_unstale":"2024-04-25T16:51:01.015551+0000","last_undegraded":"2024-04-25T16:51:01.015551+0000","last_fullsized":"2024-04-25T16:51:01.015551+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:29.939373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015015+0000","last_change":"2024-04-25T16:50:55.072516+0000","last_active":"2024-04-25T16:51:01.015015+0000","last_peered":"2024-04-25T16:51:01.015015+0000","last_clean":"2024-04-25T16:51:01.015015+0000","last_became_active":"2024-04-25T16:50:55.071677+0000","last_became_peered":"2024-04-25T16:50:55.071677+0000","last_unstale":"2024-04-25T16:51:01.015015+0000","last_undegraded":"2024-04-25T16:51:01.015015+0000","last_fullsized":"2024-04-25T16:51:01.015015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148828+0000","last_change":"2024-04-25T16:50:54.026443+0000","last_active":"2024-04-25T16:51:00.148828+0000","last_peered":"2024-04-25T16:51:00.148828+0000","last_clean":"2024-04-25T16:51:00.148828+0000","last_became_active":"2024-04-25T16:50:54.024224+0000","last_became_peered":"2024-04-25T16:50:54.024224+0000","last_unstale":"2024-04-25T16:51:00.148828+0000","last_undegraded":"2024-04-25T16:51:00.148828+0000","last_fullsized":"2024-04-25T16:51:00.148828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016170+0000","last_change":"2024-04-25T16:50:59.149300+0000","last_active":"2024-04-25T16:51:01.016170+0000","last_peered":"2024-04-25T16:51:01.016170+0000","last_clean":"2024-04-25T16:51:01.016170+0000","last_became_active":"2024-04-25T16:50:59.147799+0000","last_became_peered":"2024-04-25T16:50:59.147799+0000","last_unstale":"2024-04-25T16:51:01.016170+0000","last_undegraded":"2024-04-25T16:51:01.016170+0000","last_fullsized":"2024-04-25T16:51:01.016170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:05:04.839576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016900+0000","last_change":"2024-04-25T16:50:55.048595+0000","last_active":"2024-04-25T16:51:01.016900+0000","last_peered":"2024-04-25T16:51:01.016900+0000","last_clean":"2024-04-25T16:51:01.016900+0000","last_became_active":"2024-04-25T16:50:55.047706+0000","last_became_peered":"2024-04-25T16:50:55.047706+0000","last_unstale":"2024-04-25T16:51:01.016900+0000","last_undegraded":"2024-04-25T16:51:01.016900+0000","last_fullsized":"2024-04-25T16:51:01.016900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:56.480058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017350+0000","last_change":"2024-04-25T16:50:54.053397+0000","last_active":"2024-04-25T16:51:01.017350+0000","last_peered":"2024-04-25T16:51:01.017350+0000","last_clean":"2024-04-25T16:51:01.017350+0000","last_became_active":"2024-04-25T16:50:54.052633+0000","last_became_peered":"2024-04-25T16:50:54.052633+0000","last_unstale":"2024-04-25T16:51:01.017350+0000","last_undegraded":"2024-04-25T16:51:01.017350+0000","last_fullsized":"2024-04-25T16:51:01.017350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018189+0000","last_change":"2024-04-25T16:50:59.162703+0000","last_active":"2024-04-25T16:51:01.018189+0000","last_peered":"2024-04-25T16:51:01.018189+0000","last_clean":"2024-04-25T16:51:01.018189+0000","last_became_active":"2024-04-25T16:50:59.161980+0000","last_became_peered":"2024-04-25T16:50:59.161980+0000","last_unstale":"2024-04-25T16:51:01.018189+0000","last_undegraded":"2024-04-25T16:51:01.018189+0000","last_fullsized":"2024-04-25T16:51:01.018189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:05:04.839576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310673+0000","last_change":"2024-04-25T16:50:55.090720+0000","last_active":"2024-04-25T16:51:00.310673+0000","last_peered":"2024-04-25T16:51:00.310673+0000","last_clean":"2024-04-25T16:51:00.310673+0000","last_became_active":"2024-04-25T16:50:55.090360+0000","last_became_peered":"2024-04-25T16:50:55.090360+0000","last_unstale":"2024-04-25T16:51:00.310673+0000","last_undegraded":"2024-04-25T16:51:00.310673+0000","last_fullsized":"2024-04-25T16:51:00.310673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018278+0000","last_change":"2024-04-25T16:50:54.050879+0000","last_active":"2024-04-25T16:51:01.018278+0000","last_peered":"2024-04-25T16:51:01.018278+0000","last_clean":"2024-04-25T16:51:01.018278+0000","last_became_active":"2024-04-25T16:50:54.049784+0000","last_became_peered":"2024-04-25T16:50:54.049784+0000","last_unstale":"2024-04-25T16:51:01.018278+0000","last_undegraded":"2024-04-25T16:51:01.018278+0000","last_fullsized":"2024-04-25T16:51:01.018278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":118,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017460+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:51:01.017460+0000","last_peered":"2024-04-25T16:51:01.017460+0000","last_clean":"2024-04-25T16:51:01.017460+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:51:01.017460+0000","last_undegraded":"2024-04-25T16:51:01.017460+0000","last_fullsized":"2024-04-25T16:51:01.017460+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150640+0000","last_change":"2024-04-25T16:50:59.175343+0000","last_active":"2024-04-25T16:51:00.150640+0000","last_peered":"2024-04-25T16:51:00.150640+0000","last_clean":"2024-04-25T16:51:00.150640+0000","last_became_active":"2024-04-25T16:50:59.173979+0000","last_became_peered":"2024-04-25T16:50:59.173979+0000","last_unstale":"2024-04-25T16:51:00.150640+0000","last_undegraded":"2024-04-25T16:51:00.150640+0000","last_fullsized":"2024-04-25T16:51:00.150640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:07.790324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019964+0000","last_change":"2024-04-25T16:50:55.092868+0000","last_active":"2024-04-25T16:51:01.019964+0000","last_peered":"2024-04-25T16:51:01.019964+0000","last_clean":"2024-04-25T16:51:01.019964+0000","last_became_active":"2024-04-25T16:50:55.092569+0000","last_became_peered":"2024-04-25T16:50:55.092569+0000","last_unstale":"2024-04-25T16:51:01.019964+0000","last_undegraded":"2024-04-25T16:51:01.019964+0000","last_fullsized":"2024-04-25T16:51:01.019964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:03.704057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149644+0000","last_change":"2024-04-25T16:50:54.056537+0000","last_active":"2024-04-25T16:51:00.149644+0000","last_peered":"2024-04-25T16:51:00.149644+0000","last_clean":"2024-04-25T16:51:00.149644+0000","last_became_active":"2024-04-25T16:50:54.056219+0000","last_became_peered":"2024-04-25T16:50:54.056219+0000","last_unstale":"2024-04-25T16:51:00.149644+0000","last_undegraded":"2024-04-25T16:51:00.149644+0000","last_fullsized":"2024-04-25T16:51:00.149644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149259+0000","last_change":"2024-04-25T16:50:59.168724+0000","last_active":"2024-04-25T16:51:00.149259+0000","last_peered":"2024-04-25T16:51:00.149259+0000","last_clean":"2024-04-25T16:51:00.149259+0000","last_became_active":"2024-04-25T16:50:59.167823+0000","last_became_peered":"2024-04-25T16:50:59.167823+0000","last_unstale":"2024-04-25T16:51:00.149259+0000","last_undegraded":"2024-04-25T16:51:00.149259+0000","last_fullsized":"2024-04-25T16:51:00.149259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:00.447099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018846+0000","last_change":"2024-04-25T16:50:55.091727+0000","last_active":"2024-04-25T16:51:01.018846+0000","last_peered":"2024-04-25T16:51:01.018846+0000","last_clean":"2024-04-25T16:51:01.018846+0000","last_became_active":"2024-04-25T16:50:55.091409+0000","last_became_peered":"2024-04-25T16:50:55.091409+0000","last_unstale":"2024-04-25T16:51:01.018846+0000","last_undegraded":"2024-04-25T16:51:01.018846+0000","last_fullsized":"2024-04-25T16:51:01.018846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:14.342043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015818+0000","last_change":"2024-04-25T16:50:54.051928+0000","last_active":"2024-04-25T16:51:01.015818+0000","last_peered":"2024-04-25T16:51:01.015818+0000","last_clean":"2024-04-25T16:51:01.015818+0000","last_became_active":"2024-04-25T16:50:54.045379+0000","last_became_peered":"2024-04-25T16:50:54.045379+0000","last_unstale":"2024-04-25T16:51:01.015818+0000","last_undegraded":"2024-04-25T16:51:01.015818+0000","last_fullsized":"2024-04-25T16:51:01.015818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017284+0000","last_change":"2024-04-25T16:50:59.149171+0000","last_active":"2024-04-25T16:51:01.017284+0000","last_peered":"2024-04-25T16:51:01.017284+0000","last_clean":"2024-04-25T16:51:01.017284+0000","last_became_active":"2024-04-25T16:50:59.148753+0000","last_became_peered":"2024-04-25T16:50:59.148753+0000","last_unstale":"2024-04-25T16:51:01.017284+0000","last_undegraded":"2024-04-25T16:51:01.017284+0000","last_fullsized":"2024-04-25T16:51:01.017284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.819554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.021346+0000","last_change":"2024-04-25T16:50:55.093340+0000","last_active":"2024-04-25T16:51:01.021346+0000","last_peered":"2024-04-25T16:51:01.021346+0000","last_clean":"2024-04-25T16:51:01.021346+0000","last_became_active":"2024-04-25T16:50:55.092870+0000","last_became_peered":"2024-04-25T16:50:55.092870+0000","last_unstale":"2024-04-25T16:51:01.021346+0000","last_undegraded":"2024-04-25T16:51:01.021346+0000","last_fullsized":"2024-04-25T16:51:01.021346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:25.853106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150507+0000","last_change":"2024-04-25T16:50:54.041300+0000","last_active":"2024-04-25T16:51:00.150507+0000","last_peered":"2024-04-25T16:51:00.150507+0000","last_clean":"2024-04-25T16:51:00.150507+0000","last_became_active":"2024-04-25T16:50:54.040987+0000","last_became_peered":"2024-04-25T16:50:54.040987+0000","last_unstale":"2024-04-25T16:51:00.150507+0000","last_undegraded":"2024-04-25T16:51:00.150507+0000","last_fullsized":"2024-04-25T16:51:00.150507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016995+0000","last_change":"2024-04-25T16:50:59.180254+0000","last_active":"2024-04-25T16:51:01.016995+0000","last_peered":"2024-04-25T16:51:01.016995+0000","last_clean":"2024-04-25T16:51:01.016995+0000","last_became_active":"2024-04-25T16:50:59.179432+0000","last_became_peered":"2024-04-25T16:50:59.179432+0000","last_unstale":"2024-04-25T16:51:01.016995+0000","last_undegraded":"2024-04-25T16:51:01.016995+0000","last_fullsized":"2024-04-25T16:51:01.016995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:44.752448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020598+0000","last_change":"2024-04-25T16:50:55.085874+0000","last_active":"2024-04-25T16:51:01.020598+0000","last_peered":"2024-04-25T16:51:01.020598+0000","last_clean":"2024-04-25T16:51:01.020598+0000","last_became_active":"2024-04-25T16:50:55.085347+0000","last_became_peered":"2024-04-25T16:50:55.085347+0000","last_unstale":"2024-04-25T16:51:01.020598+0000","last_undegraded":"2024-04-25T16:51:01.020598+0000","last_fullsized":"2024-04-25T16:51:01.020598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:14.906309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152960+0000","last_change":"2024-04-25T16:50:54.037711+0000","last_active":"2024-04-25T16:51:00.152960+0000","last_peered":"2024-04-25T16:51:00.152960+0000","last_clean":"2024-04-25T16:51:00.152960+0000","last_became_active":"2024-04-25T16:50:54.036847+0000","last_became_peered":"2024-04-25T16:50:54.036847+0000","last_unstale":"2024-04-25T16:51:00.152960+0000","last_undegraded":"2024-04-25T16:51:00.152960+0000","last_fullsized":"2024-04-25T16:51:00.152960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152479+0000","last_change":"2024-04-25T16:50:59.165076+0000","last_active":"2024-04-25T16:51:00.152479+0000","last_peered":"2024-04-25T16:51:00.152479+0000","last_clean":"2024-04-25T16:51:00.152479+0000","last_became_active":"2024-04-25T16:50:59.159200+0000","last_became_peered":"2024-04-25T16:50:59.159200+0000","last_unstale":"2024-04-25T16:51:00.152479+0000","last_undegraded":"2024-04-25T16:51:00.152479+0000","last_fullsized":"2024-04-25T16:51:00.152479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:11.722626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017585+0000","last_change":"2024-04-25T16:50:55.085792+0000","last_active":"2024-04-25T16:51:01.017585+0000","last_peered":"2024-04-25T16:51:01.017585+0000","last_clean":"2024-04-25T16:51:01.017585+0000","last_became_active":"2024-04-25T16:50:55.085212+0000","last_became_peered":"2024-04-25T16:50:55.085212+0000","last_unstale":"2024-04-25T16:51:01.017585+0000","last_undegraded":"2024-04-25T16:51:01.017585+0000","last_fullsized":"2024-04-25T16:51:01.017585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:08.427466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148299+0000","last_change":"2024-04-25T16:50:54.021199+0000","last_active":"2024-04-25T16:51:00.148299+0000","last_peered":"2024-04-25T16:51:00.148299+0000","last_clean":"2024-04-25T16:51:00.148299+0000","last_became_active":"2024-04-25T16:50:54.020716+0000","last_became_peered":"2024-04-25T16:50:54.020716+0000","last_unstale":"2024-04-25T16:51:00.148299+0000","last_undegraded":"2024-04-25T16:51:00.148299+0000","last_fullsized":"2024-04-25T16:51:00.148299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:27.893510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017880+0000","last_change":"2024-04-25T16:50:59.160368+0000","last_active":"2024-04-25T16:51:01.017880+0000","last_peered":"2024-04-25T16:51:01.017880+0000","last_clean":"2024-04-25T16:51:01.017880+0000","last_became_active":"2024-04-25T16:50:59.159232+0000","last_became_peered":"2024-04-25T16:50:59.159232+0000","last_unstale":"2024-04-25T16:51:01.017880+0000","last_undegraded":"2024-04-25T16:51:01.017880+0000","last_fullsized":"2024-04-25T16:51:01.017880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.992576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148948+0000","last_change":"2024-04-25T16:50:55.059427+0000","last_active":"2024-04-25T16:51:00.148948+0000","last_peered":"2024-04-25T16:51:00.148948+0000","last_clean":"2024-04-25T16:51:00.148948+0000","last_became_active":"2024-04-25T16:50:55.058611+0000","last_became_peered":"2024-04-25T16:50:55.058611+0000","last_unstale":"2024-04-25T16:51:00.148948+0000","last_undegraded":"2024-04-25T16:51:00.148948+0000","last_fullsized":"2024-04-25T16:51:00.148948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149289+0000","last_change":"2024-04-25T16:50:54.044858+0000","last_active":"2024-04-25T16:51:00.149289+0000","last_peered":"2024-04-25T16:51:00.149289+0000","last_clean":"2024-04-25T16:51:00.149289+0000","last_became_active":"2024-04-25T16:50:54.043345+0000","last_became_peered":"2024-04-25T16:50:54.043345+0000","last_unstale":"2024-04-25T16:51:00.149289+0000","last_undegraded":"2024-04-25T16:51:00.149289+0000","last_fullsized":"2024-04-25T16:51:00.149289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.312170+0000","last_change":"2024-04-25T16:50:59.173258+0000","last_active":"2024-04-25T16:51:00.312170+0000","last_peered":"2024-04-25T16:51:00.312170+0000","last_clean":"2024-04-25T16:51:00.312170+0000","last_became_active":"2024-04-25T16:50:59.172637+0000","last_became_peered":"2024-04-25T16:50:59.172637+0000","last_unstale":"2024-04-25T16:51:00.312170+0000","last_undegraded":"2024-04-25T16:51:00.312170+0000","last_fullsized":"2024-04-25T16:51:00.312170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:46.618416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019246+0000","last_change":"2024-04-25T16:50:55.085682+0000","last_active":"2024-04-25T16:51:01.019246+0000","last_peered":"2024-04-25T16:51:01.019246+0000","last_clean":"2024-04-25T16:51:01.019246+0000","last_became_active":"2024-04-25T16:50:55.084855+0000","last_became_peered":"2024-04-25T16:50:55.084855+0000","last_unstale":"2024-04-25T16:51:01.019246+0000","last_undegraded":"2024-04-25T16:51:01.019246+0000","last_fullsized":"2024-04-25T16:51:01.019246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:42.081728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019411+0000","last_change":"2024-04-25T16:50:54.059696+0000","last_active":"2024-04-25T16:51:01.019411+0000","last_peered":"2024-04-25T16:51:01.019411+0000","last_clean":"2024-04-25T16:51:01.019411+0000","last_became_active":"2024-04-25T16:50:54.058979+0000","last_became_peered":"2024-04-25T16:50:54.058979+0000","last_unstale":"2024-04-25T16:51:01.019411+0000","last_undegraded":"2024-04-25T16:51:01.019411+0000","last_fullsized":"2024-04-25T16:51:01.019411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311564+0000","last_change":"2024-04-25T16:50:59.163291+0000","last_active":"2024-04-25T16:51:00.311564+0000","last_peered":"2024-04-25T16:51:00.311564+0000","last_clean":"2024-04-25T16:51:00.311564+0000","last_became_active":"2024-04-25T16:50:59.162316+0000","last_became_peered":"2024-04-25T16:50:59.162316+0000","last_unstale":"2024-04-25T16:51:00.311564+0000","last_undegraded":"2024-04-25T16:51:00.311564+0000","last_fullsized":"2024-04-25T16:51:00.311564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:41.776182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018471+0000","last_change":"2024-04-25T16:50:55.067672+0000","last_active":"2024-04-25T16:51:01.018471+0000","last_peered":"2024-04-25T16:51:01.018471+0000","last_clean":"2024-04-25T16:51:01.018471+0000","last_became_active":"2024-04-25T16:50:55.066870+0000","last_became_peered":"2024-04-25T16:50:55.066870+0000","last_unstale":"2024-04-25T16:51:01.018471+0000","last_undegraded":"2024-04-25T16:51:01.018471+0000","last_fullsized":"2024-04-25T16:51:01.018471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:28:31.672027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017648+0000","last_change":"2024-04-25T16:50:54.036862+0000","last_active":"2024-04-25T16:51:01.017648+0000","last_peered":"2024-04-25T16:51:01.017648+0000","last_clean":"2024-04-25T16:51:01.017648+0000","last_became_active":"2024-04-25T16:50:54.036549+0000","last_became_peered":"2024-04-25T16:50:54.036549+0000","last_unstale":"2024-04-25T16:51:01.017648+0000","last_undegraded":"2024-04-25T16:51:01.017648+0000","last_fullsized":"2024-04-25T16:51:01.017648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:24.611706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018388+0000","last_change":"2024-04-25T16:50:59.175285+0000","last_active":"2024-04-25T16:51:01.018388+0000","last_peered":"2024-04-25T16:51:01.018388+0000","last_clean":"2024-04-25T16:51:01.018388+0000","last_became_active":"2024-04-25T16:50:59.174301+0000","last_became_peered":"2024-04-25T16:50:59.174301+0000","last_unstale":"2024-04-25T16:51:01.018388+0000","last_undegraded":"2024-04-25T16:51:01.018388+0000","last_fullsized":"2024-04-25T16:51:01.018388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:25.742371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019931+0000","last_change":"2024-04-25T16:50:55.076173+0000","last_active":"2024-04-25T16:51:01.019931+0000","last_peered":"2024-04-25T16:51:01.019931+0000","last_clean":"2024-04-25T16:51:01.019931+0000","last_became_active":"2024-04-25T16:50:55.075360+0000","last_became_peered":"2024-04-25T16:50:55.075360+0000","last_unstale":"2024-04-25T16:51:01.019931+0000","last_undegraded":"2024-04-25T16:51:01.019931+0000","last_fullsized":"2024-04-25T16:51:01.019931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:04.031520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:10.281685+0000","last_change":"2024-04-25T16:50:54.035115+0000","last_active":"2024-04-25T16:51:10.281685+0000","last_peered":"2024-04-25T16:51:10.281685+0000","last_clean":"2024-04-25T16:51:10.281685+0000","last_became_active":"2024-04-25T16:50:54.034629+0000","last_became_peered":"2024-04-25T16:50:54.034629+0000","last_unstale":"2024-04-25T16:51:10.281685+0000","last_undegraded":"2024-04-25T16:51:10.281685+0000","last_fullsized":"2024-04-25T16:51:10.281685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.077660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015930+0000","last_change":"2024-04-25T16:50:59.159337+0000","last_active":"2024-04-25T16:51:01.015930+0000","last_peered":"2024-04-25T16:51:01.015930+0000","last_clean":"2024-04-25T16:51:01.015930+0000","last_became_active":"2024-04-25T16:50:59.156207+0000","last_became_peered":"2024-04-25T16:50:59.156207+0000","last_unstale":"2024-04-25T16:51:01.015930+0000","last_undegraded":"2024-04-25T16:51:01.015930+0000","last_fullsized":"2024-04-25T16:51:01.015930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:39.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154043+0000","last_change":"2024-04-25T16:50:55.066237+0000","last_active":"2024-04-25T16:51:00.154043+0000","last_peered":"2024-04-25T16:51:00.154043+0000","last_clean":"2024-04-25T16:51:00.154043+0000","last_became_active":"2024-04-25T16:50:55.063200+0000","last_became_peered":"2024-04-25T16:50:55.063200+0000","last_unstale":"2024-04-25T16:51:00.154043+0000","last_undegraded":"2024-04-25T16:51:00.154043+0000","last_fullsized":"2024-04-25T16:51:00.154043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:22.053709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015709+0000","last_change":"2024-04-25T16:50:54.055594+0000","last_active":"2024-04-25T16:51:01.015709+0000","last_peered":"2024-04-25T16:51:01.015709+0000","last_clean":"2024-04-25T16:51:01.015709+0000","last_became_active":"2024-04-25T16:50:54.055027+0000","last_became_peered":"2024-04-25T16:50:54.055027+0000","last_unstale":"2024-04-25T16:51:01.015709+0000","last_undegraded":"2024-04-25T16:51:01.015709+0000","last_fullsized":"2024-04-25T16:51:01.015709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154662+0000","last_change":"2024-04-25T16:50:59.160116+0000","last_active":"2024-04-25T16:51:00.154662+0000","last_peered":"2024-04-25T16:51:00.154662+0000","last_clean":"2024-04-25T16:51:00.154662+0000","last_became_active":"2024-04-25T16:50:59.157923+0000","last_became_peered":"2024-04-25T16:50:59.157923+0000","last_unstale":"2024-04-25T16:51:00.154662+0000","last_undegraded":"2024-04-25T16:51:00.154662+0000","last_fullsized":"2024-04-25T16:51:00.154662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:46.167995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018271+0000","last_change":"2024-04-25T16:50:55.083898+0000","last_active":"2024-04-25T16:51:01.018271+0000","last_peered":"2024-04-25T16:51:01.018271+0000","last_clean":"2024-04-25T16:51:01.018271+0000","last_became_active":"2024-04-25T16:50:55.083533+0000","last_became_peered":"2024-04-25T16:50:55.083533+0000","last_unstale":"2024-04-25T16:51:01.018271+0000","last_undegraded":"2024-04-25T16:51:01.018271+0000","last_fullsized":"2024-04-25T16:51:01.018271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019088+0000","last_change":"2024-04-25T16:50:54.056906+0000","last_active":"2024-04-25T16:51:01.019088+0000","last_peered":"2024-04-25T16:51:01.019088+0000","last_clean":"2024-04-25T16:51:01.019088+0000","last_became_active":"2024-04-25T16:50:54.056102+0000","last_became_peered":"2024-04-25T16:50:54.056102+0000","last_unstale":"2024-04-25T16:51:01.019088+0000","last_undegraded":"2024-04-25T16:51:01.019088+0000","last_fullsized":"2024-04-25T16:51:01.019088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310494+0000","last_change":"2024-04-25T16:50:59.129868+0000","last_active":"2024-04-25T16:51:00.310494+0000","last_peered":"2024-04-25T16:51:00.310494+0000","last_clean":"2024-04-25T16:51:00.310494+0000","last_became_active":"2024-04-25T16:50:59.129447+0000","last_became_peered":"2024-04-25T16:50:59.129447+0000","last_unstale":"2024-04-25T16:51:00.310494+0000","last_undegraded":"2024-04-25T16:51:00.310494+0000","last_fullsized":"2024-04-25T16:51:00.310494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:07.826745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018797+0000","last_change":"2024-04-25T16:50:55.093338+0000","last_active":"2024-04-25T16:51:01.018797+0000","last_peered":"2024-04-25T16:51:01.018797+0000","last_clean":"2024-04-25T16:51:01.018797+0000","last_became_active":"2024-04-25T16:50:55.092985+0000","last_became_peered":"2024-04-25T16:50:55.092985+0000","last_unstale":"2024-04-25T16:51:01.018797+0000","last_undegraded":"2024-04-25T16:51:01.018797+0000","last_fullsized":"2024-04-25T16:51:01.018797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:03.704057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151183+0000","last_change":"2024-04-25T16:50:54.053862+0000","last_active":"2024-04-25T16:51:00.151183+0000","last_peered":"2024-04-25T16:51:00.151183+0000","last_clean":"2024-04-25T16:51:00.151183+0000","last_became_active":"2024-04-25T16:50:54.053156+0000","last_became_peered":"2024-04-25T16:50:54.053156+0000","last_unstale":"2024-04-25T16:51:00.151183+0000","last_undegraded":"2024-04-25T16:51:00.151183+0000","last_fullsized":"2024-04-25T16:51:00.151183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310744+0000","last_change":"2024-04-25T16:50:59.181080+0000","last_active":"2024-04-25T16:51:00.310744+0000","last_peered":"2024-04-25T16:51:00.310744+0000","last_clean":"2024-04-25T16:51:00.310744+0000","last_became_active":"2024-04-25T16:50:59.180638+0000","last_became_peered":"2024-04-25T16:50:59.180638+0000","last_unstale":"2024-04-25T16:51:00.310744+0000","last_undegraded":"2024-04-25T16:51:00.310744+0000","last_fullsized":"2024-04-25T16:51:00.310744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:47.904606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017640+0000","last_change":"2024-04-25T16:50:59.173382+0000","last_active":"2024-04-25T16:51:01.017640+0000","last_peered":"2024-04-25T16:51:01.017640+0000","last_clean":"2024-04-25T16:51:01.017640+0000","last_became_active":"2024-04-25T16:50:59.172921+0000","last_became_peered":"2024-04-25T16:50:59.172921+0000","last_unstale":"2024-04-25T16:51:01.017640+0000","last_undegraded":"2024-04-25T16:51:01.017640+0000","last_fullsized":"2024-04-25T16:51:01.017640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:39.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015213+0000","last_change":"2024-04-25T16:50:55.065015+0000","last_active":"2024-04-25T16:51:01.015213+0000","last_peered":"2024-04-25T16:51:01.015213+0000","last_clean":"2024-04-25T16:51:01.015213+0000","last_became_active":"2024-04-25T16:50:55.063038+0000","last_became_peered":"2024-04-25T16:50:55.063038+0000","last_unstale":"2024-04-25T16:51:01.015213+0000","last_undegraded":"2024-04-25T16:51:01.015213+0000","last_fullsized":"2024-04-25T16:51:01.015213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017330+0000","last_change":"2024-04-25T16:50:54.054787+0000","last_active":"2024-04-25T16:51:01.017330+0000","last_peered":"2024-04-25T16:51:01.017330+0000","last_clean":"2024-04-25T16:51:01.017330+0000","last_became_active":"2024-04-25T16:50:54.051984+0000","last_became_peered":"2024-04-25T16:50:54.051984+0000","last_unstale":"2024-04-25T16:51:01.017330+0000","last_undegraded":"2024-04-25T16:51:01.017330+0000","last_fullsized":"2024-04-25T16:51:01.017330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.166309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151032+0000","last_change":"2024-04-25T16:50:59.154675+0000","last_active":"2024-04-25T16:51:00.151032+0000","last_peered":"2024-04-25T16:51:00.151032+0000","last_clean":"2024-04-25T16:51:00.151032+0000","last_became_active":"2024-04-25T16:50:59.152969+0000","last_became_peered":"2024-04-25T16:50:59.152969+0000","last_unstale":"2024-04-25T16:51:00.151032+0000","last_undegraded":"2024-04-25T16:51:00.151032+0000","last_fullsized":"2024-04-25T16:51:00.151032+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.819554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310256+0000","last_change":"2024-04-25T16:50:55.069932+0000","last_active":"2024-04-25T16:51:00.310256+0000","last_peered":"2024-04-25T16:51:00.310256+0000","last_clean":"2024-04-25T16:51:00.310256+0000","last_became_active":"2024-04-25T16:50:55.060902+0000","last_became_peered":"2024-04-25T16:50:55.060902+0000","last_unstale":"2024-04-25T16:51:00.310256+0000","last_undegraded":"2024-04-25T16:51:00.310256+0000","last_fullsized":"2024-04-25T16:51:00.310256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310569+0000","last_change":"2024-04-25T16:50:54.043300+0000","last_active":"2024-04-25T16:51:00.310569+0000","last_peered":"2024-04-25T16:51:00.310569+0000","last_clean":"2024-04-25T16:51:00.310569+0000","last_became_active":"2024-04-25T16:50:54.042909+0000","last_became_peered":"2024-04-25T16:50:54.042909+0000","last_unstale":"2024-04-25T16:51:00.310569+0000","last_undegraded":"2024-04-25T16:51:00.310569+0000","last_fullsized":"2024-04-25T16:51:00.310569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:22.485560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149622+0000","last_change":"2024-04-25T16:50:54.044711+0000","last_active":"2024-04-25T16:51:00.149622+0000","last_peered":"2024-04-25T16:51:00.149622+0000","last_clean":"2024-04-25T16:51:00.149622+0000","last_became_active":"2024-04-25T16:50:54.043597+0000","last_became_peered":"2024-04-25T16:50:54.043597+0000","last_unstale":"2024-04-25T16:51:00.149622+0000","last_undegraded":"2024-04-25T16:51:00.149622+0000","last_fullsized":"2024-04-25T16:51:00.149622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017542+0000","last_change":"2024-04-25T16:50:55.061228+0000","last_active":"2024-04-25T16:51:01.017542+0000","last_peered":"2024-04-25T16:51:01.017542+0000","last_clean":"2024-04-25T16:51:01.017542+0000","last_became_active":"2024-04-25T16:50:55.060211+0000","last_became_peered":"2024-04-25T16:50:55.060211+0000","last_unstale":"2024-04-25T16:51:01.017542+0000","last_undegraded":"2024-04-25T16:51:01.017542+0000","last_fullsized":"2024-04-25T16:51:01.017542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149479+0000","last_change":"2024-04-25T16:50:59.151109+0000","last_active":"2024-04-25T16:51:00.149479+0000","last_peered":"2024-04-25T16:51:00.149479+0000","last_clean":"2024-04-25T16:51:00.149479+0000","last_became_active":"2024-04-25T16:50:59.150443+0000","last_became_peered":"2024-04-25T16:50:59.150443+0000","last_unstale":"2024-04-25T16:51:00.149479+0000","last_undegraded":"2024-04-25T16:51:00.149479+0000","last_fullsized":"2024-04-25T16:51:00.149479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:45.414606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310730+0000","last_change":"2024-04-25T16:50:54.047737+0000","last_active":"2024-04-25T16:51:00.310730+0000","last_peered":"2024-04-25T16:51:00.310730+0000","last_clean":"2024-04-25T16:51:00.310730+0000","last_became_active":"2024-04-25T16:50:54.047417+0000","last_became_peered":"2024-04-25T16:50:54.047417+0000","last_unstale":"2024-04-25T16:51:00.310730+0000","last_undegraded":"2024-04-25T16:51:00.310730+0000","last_fullsized":"2024-04-25T16:51:00.310730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.166309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153448+0000","last_change":"2024-04-25T16:50:55.064168+0000","last_active":"2024-04-25T16:51:00.153448+0000","last_peered":"2024-04-25T16:51:00.153448+0000","last_clean":"2024-04-25T16:51:00.153448+0000","last_became_active":"2024-04-25T16:50:55.062345+0000","last_became_peered":"2024-04-25T16:50:55.062345+0000","last_unstale":"2024-04-25T16:51:00.153448+0000","last_undegraded":"2024-04-25T16:51:00.153448+0000","last_fullsized":"2024-04-25T16:51:00.153448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:09.018645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311446+0000","last_change":"2024-04-25T16:50:59.143004+0000","last_active":"2024-04-25T16:51:00.311446+0000","last_peered":"2024-04-25T16:51:00.311446+0000","last_clean":"2024-04-25T16:51:00.311446+0000","last_became_active":"2024-04-25T16:50:59.142240+0000","last_became_peered":"2024-04-25T16:50:59.142240+0000","last_unstale":"2024-04-25T16:51:00.311446+0000","last_undegraded":"2024-04-25T16:51:00.311446+0000","last_fullsized":"2024-04-25T16:51:00.311446+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:19.230775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148672+0000","last_change":"2024-04-25T16:50:55.066558+0000","last_active":"2024-04-25T16:51:00.148672+0000","last_peered":"2024-04-25T16:51:00.148672+0000","last_clean":"2024-04-25T16:51:00.148672+0000","last_became_active":"2024-04-25T16:50:55.065201+0000","last_became_peered":"2024-04-25T16:50:55.065201+0000","last_unstale":"2024-04-25T16:51:00.148672+0000","last_undegraded":"2024-04-25T16:51:00.148672+0000","last_fullsized":"2024-04-25T16:51:00.148672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:12.518641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148555+0000","last_change":"2024-04-25T16:50:55.089086+0000","last_active":"2024-04-25T16:51:00.148555+0000","last_peered":"2024-04-25T16:51:00.148555+0000","last_clean":"2024-04-25T16:51:00.148555+0000","last_became_active":"2024-04-25T16:50:55.088788+0000","last_became_peered":"2024-04-25T16:50:55.088788+0000","last_unstale":"2024-04-25T16:51:00.148555+0000","last_undegraded":"2024-04-25T16:51:00.148555+0000","last_fullsized":"2024-04-25T16:51:00.148555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:22.053709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019230+0000","last_change":"2024-04-25T16:50:55.092706+0000","last_active":"2024-04-25T16:51:01.019230+0000","last_peered":"2024-04-25T16:51:01.019230+0000","last_clean":"2024-04-25T16:51:01.019230+0000","last_became_active":"2024-04-25T16:50:55.092411+0000","last_became_peered":"2024-04-25T16:50:55.092411+0000","last_unstale":"2024-04-25T16:51:01.019230+0000","last_undegraded":"2024-04-25T16:51:01.019230+0000","last_fullsized":"2024-04-25T16:51:01.019230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:56.360832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017429+0000","last_change":"2024-04-25T16:50:55.053386+0000","last_active":"2024-04-25T16:51:01.017429+0000","last_peered":"2024-04-25T16:51:01.017429+0000","last_clean":"2024-04-25T16:51:01.017429+0000","last_became_active":"2024-04-25T16:50:55.052555+0000","last_became_peered":"2024-04-25T16:50:55.052555+0000","last_unstale":"2024-04-25T16:51:01.017429+0000","last_undegraded":"2024-04-25T16:51:01.017429+0000","last_fullsized":"2024-04-25T16:51:01.017429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311617+0000","last_change":"2024-04-25T16:50:55.072013+0000","last_active":"2024-04-25T16:51:00.311617+0000","last_peered":"2024-04-25T16:51:00.311617+0000","last_clean":"2024-04-25T16:51:00.311617+0000","last_became_active":"2024-04-25T16:50:55.071089+0000","last_became_peered":"2024-04-25T16:50:55.071089+0000","last_unstale":"2024-04-25T16:51:00.311617+0000","last_undegraded":"2024-04-25T16:51:00.311617+0000","last_fullsized":"2024-04-25T16:51:00.311617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.304249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015430+0000","last_change":"2024-04-25T16:50:55.063809+0000","last_active":"2024-04-25T16:51:01.015430+0000","last_peered":"2024-04-25T16:51:01.015430+0000","last_clean":"2024-04-25T16:51:01.015430+0000","last_became_active":"2024-04-25T16:50:55.062913+0000","last_became_peered":"2024-04-25T16:50:55.062913+0000","last_unstale":"2024-04-25T16:51:01.015430+0000","last_undegraded":"2024-04-25T16:51:01.015430+0000","last_fullsized":"2024-04-25T16:51:01.015430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151698+0000","last_change":"2024-04-25T16:50:55.066745+0000","last_active":"2024-04-25T16:51:00.151698+0000","last_peered":"2024-04-25T16:51:00.151698+0000","last_clean":"2024-04-25T16:51:00.151698+0000","last_became_active":"2024-04-25T16:50:55.065075+0000","last_became_peered":"2024-04-25T16:50:55.065075+0000","last_unstale":"2024-04-25T16:51:00.151698+0000","last_undegraded":"2024-04-25T16:51:00.151698+0000","last_fullsized":"2024-04-25T16:51:00.151698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.312312+0000","last_change":"2024-04-25T16:50:55.081262+0000","last_active":"2024-04-25T16:51:00.312312+0000","last_peered":"2024-04-25T16:51:00.312312+0000","last_clean":"2024-04-25T16:51:00.312312+0000","last_became_active":"2024-04-25T16:50:55.080803+0000","last_became_peered":"2024-04-25T16:50:55.080803+0000","last_unstale":"2024-04-25T16:51:00.312312+0000","last_undegraded":"2024-04-25T16:51:00.312312+0000","last_fullsized":"2024-04-25T16:51:00.312312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:14.342043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017299+0000","last_change":"2024-04-25T16:50:55.090717+0000","last_active":"2024-04-25T16:51:01.017299+0000","last_peered":"2024-04-25T16:51:01.017299+0000","last_clean":"2024-04-25T16:51:01.017299+0000","last_became_active":"2024-04-25T16:50:55.090388+0000","last_became_peered":"2024-04-25T16:50:55.090388+0000","last_unstale":"2024-04-25T16:51:01.017299+0000","last_undegraded":"2024-04-25T16:51:01.017299+0000","last_fullsized":"2024-04-25T16:51:01.017299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310335+0000","last_change":"2024-04-25T16:50:55.072773+0000","last_active":"2024-04-25T16:51:00.310335+0000","last_peered":"2024-04-25T16:51:00.310335+0000","last_clean":"2024-04-25T16:51:00.310335+0000","last_became_active":"2024-04-25T16:50:55.071914+0000","last_became_peered":"2024-04-25T16:50:55.071914+0000","last_unstale":"2024-04-25T16:51:00.310335+0000","last_undegraded":"2024-04-25T16:51:00.310335+0000","last_fullsized":"2024-04-25T16:51:00.310335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018758+0000","last_change":"2024-04-25T16:50:55.093501+0000","last_active":"2024-04-25T16:51:01.018758+0000","last_peered":"2024-04-25T16:51:01.018758+0000","last_clean":"2024-04-25T16:51:01.018758+0000","last_became_active":"2024-04-25T16:50:55.093168+0000","last_became_peered":"2024-04-25T16:50:55.093168+0000","last_unstale":"2024-04-25T16:51:01.018758+0000","last_undegraded":"2024-04-25T16:51:01.018758+0000","last_fullsized":"2024-04-25T16:51:01.018758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:42.081728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017137+0000","last_change":"2024-04-25T16:50:55.066877+0000","last_active":"2024-04-25T16:51:01.017137+0000","last_peered":"2024-04-25T16:51:01.017137+0000","last_clean":"2024-04-25T16:51:01.017137+0000","last_became_active":"2024-04-25T16:50:55.066129+0000","last_became_peered":"2024-04-25T16:50:55.066129+0000","last_unstale":"2024-04-25T16:51:01.017137+0000","last_undegraded":"2024-04-25T16:51:01.017137+0000","last_fullsized":"2024-04-25T16:51:01.017137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:12.518641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150236+0000","last_change":"2024-04-25T16:50:55.052920+0000","last_active":"2024-04-25T16:51:00.150236+0000","last_peered":"2024-04-25T16:51:00.150236+0000","last_clean":"2024-04-25T16:51:00.150236+0000","last_became_active":"2024-04-25T16:50:55.047735+0000","last_became_peered":"2024-04-25T16:50:55.047735+0000","last_unstale":"2024-04-25T16:51:00.150236+0000","last_undegraded":"2024-04-25T16:51:00.150236+0000","last_fullsized":"2024-04-25T16:51:00.150236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020445+0000","last_change":"2024-04-25T16:50:55.090598+0000","last_active":"2024-04-25T16:51:01.020445+0000","last_peered":"2024-04-25T16:51:01.020445+0000","last_clean":"2024-04-25T16:51:01.020445+0000","last_became_active":"2024-04-25T16:50:55.090102+0000","last_became_peered":"2024-04-25T16:50:55.090102+0000","last_unstale":"2024-04-25T16:51:01.020445+0000","last_undegraded":"2024-04-25T16:51:01.020445+0000","last_fullsized":"2024-04-25T16:51:01.020445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:35.322477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150315+0000","last_change":"2024-04-25T16:50:55.081066+0000","last_active":"2024-04-25T16:51:00.150315+0000","last_peered":"2024-04-25T16:51:00.150315+0000","last_clean":"2024-04-25T16:51:00.150315+0000","last_became_active":"2024-04-25T16:50:55.069516+0000","last_became_peered":"2024-04-25T16:50:55.069516+0000","last_unstale":"2024-04-25T16:51:00.150315+0000","last_undegraded":"2024-04-25T16:51:00.150315+0000","last_fullsized":"2024-04-25T16:51:00.150315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:13.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019657+0000","last_change":"2024-04-25T16:50:55.093971+0000","last_active":"2024-04-25T16:51:01.019657+0000","last_peered":"2024-04-25T16:51:01.019657+0000","last_clean":"2024-04-25T16:51:01.019657+0000","last_became_active":"2024-04-25T16:50:55.093421+0000","last_became_peered":"2024-04-25T16:50:55.093421+0000","last_unstale":"2024-04-25T16:51:01.019657+0000","last_undegraded":"2024-04-25T16:51:01.019657+0000","last_fullsized":"2024-04-25T16:51:01.019657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:10.733287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016967+0000","last_change":"2024-04-25T16:50:55.075149+0000","last_active":"2024-04-25T16:51:01.016967+0000","last_peered":"2024-04-25T16:51:01.016967+0000","last_clean":"2024-04-25T16:51:01.016967+0000","last_became_active":"2024-04-25T16:50:55.074215+0000","last_became_peered":"2024-04-25T16:50:55.074215+0000","last_unstale":"2024-04-25T16:51:01.016967+0000","last_undegraded":"2024-04-25T16:51:01.016967+0000","last_fullsized":"2024-04-25T16:51:01.016967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015476+0000","last_change":"2024-04-25T16:50:55.072156+0000","last_active":"2024-04-25T16:51:01.015476+0000","last_peered":"2024-04-25T16:51:01.015476+0000","last_clean":"2024-04-25T16:51:01.015476+0000","last_became_active":"2024-04-25T16:50:55.071295+0000","last_became_peered":"2024-04-25T16:50:55.071295+0000","last_unstale":"2024-04-25T16:51:01.015476+0000","last_undegraded":"2024-04-25T16:51:01.015476+0000","last_fullsized":"2024-04-25T16:51:01.015476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018336+0000","last_change":"2024-04-25T16:50:55.062605+0000","last_active":"2024-04-25T16:51:01.018336+0000","last_peered":"2024-04-25T16:51:01.018336+0000","last_clean":"2024-04-25T16:51:01.018336+0000","last_became_active":"2024-04-25T16:50:55.061970+0000","last_became_peered":"2024-04-25T16:50:55.061970+0000","last_unstale":"2024-04-25T16:51:01.018336+0000","last_undegraded":"2024-04-25T16:51:01.018336+0000","last_fullsized":"2024-04-25T16:51:01.018336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:41.328339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310463+0000","last_change":"2024-04-25T16:50:55.062534+0000","last_active":"2024-04-25T16:51:00.310463+0000","last_peered":"2024-04-25T16:51:00.310463+0000","last_clean":"2024-04-25T16:51:00.310463+0000","last_became_active":"2024-04-25T16:50:55.061601+0000","last_became_peered":"2024-04-25T16:50:55.061601+0000","last_unstale":"2024-04-25T16:51:00.310463+0000","last_undegraded":"2024-04-25T16:51:00.310463+0000","last_fullsized":"2024-04-25T16:51:00.310463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154054+0000","last_change":"2024-04-25T16:50:55.067043+0000","last_active":"2024-04-25T16:51:00.154054+0000","last_peered":"2024-04-25T16:51:00.154054+0000","last_clean":"2024-04-25T16:51:00.154054+0000","last_became_active":"2024-04-25T16:50:55.066155+0000","last_became_peered":"2024-04-25T16:50:55.066155+0000","last_unstale":"2024-04-25T16:51:00.154054+0000","last_undegraded":"2024-04-25T16:51:00.154054+0000","last_fullsized":"2024-04-25T16:51:00.154054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:08.701059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150829+0000","last_change":"2024-04-25T16:50:55.084655+0000","last_active":"2024-04-25T16:51:00.150829+0000","last_peered":"2024-04-25T16:51:00.150829+0000","last_clean":"2024-04-25T16:51:00.150829+0000","last_became_active":"2024-04-25T16:50:55.083647+0000","last_became_peered":"2024-04-25T16:50:55.083647+0000","last_unstale":"2024-04-25T16:51:00.150829+0000","last_undegraded":"2024-04-25T16:51:00.150829+0000","last_fullsized":"2024-04-25T16:51:00.150829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151431+0000","last_change":"2024-04-25T16:50:55.085352+0000","last_active":"2024-04-25T16:51:00.151431+0000","last_peered":"2024-04-25T16:51:00.151431+0000","last_clean":"2024-04-25T16:51:00.151431+0000","last_became_active":"2024-04-25T16:50:55.083135+0000","last_became_peered":"2024-04-25T16:50:55.083135+0000","last_unstale":"2024-04-25T16:51:00.151431+0000","last_undegraded":"2024-04-25T16:51:00.151431+0000","last_fullsized":"2024-04-25T16:51:00.151431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.021207+0000","last_change":"2024-04-25T16:50:55.090981+0000","last_active":"2024-04-25T16:51:01.021207+0000","last_peered":"2024-04-25T16:51:01.021207+0000","last_clean":"2024-04-25T16:51:01.021207+0000","last_became_active":"2024-04-25T16:50:55.090507+0000","last_became_peered":"2024-04-25T16:50:55.090507+0000","last_unstale":"2024-04-25T16:51:01.021207+0000","last_undegraded":"2024-04-25T16:51:01.021207+0000","last_fullsized":"2024-04-25T16:51:01.021207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:10.733287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.309934+0000","last_change":"2024-04-25T16:50:55.090738+0000","last_active":"2024-04-25T16:51:00.309934+0000","last_peered":"2024-04-25T16:51:00.309934+0000","last_clean":"2024-04-25T16:51:00.309934+0000","last_became_active":"2024-04-25T16:50:55.090298+0000","last_became_peered":"2024-04-25T16:50:55.090298+0000","last_unstale":"2024-04-25T16:51:00.309934+0000","last_undegraded":"2024-04-25T16:51:00.309934+0000","last_fullsized":"2024-04-25T16:51:00.309934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:12.518641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017349+0000","last_change":"2024-04-25T16:50:55.083767+0000","last_active":"2024-04-25T16:51:01.017349+0000","last_peered":"2024-04-25T16:51:01.017349+0000","last_clean":"2024-04-25T16:51:01.017349+0000","last_became_active":"2024-04-25T16:50:55.083468+0000","last_became_peered":"2024-04-25T16:50:55.083468+0000","last_unstale":"2024-04-25T16:51:01.017349+0000","last_undegraded":"2024-04-25T16:51:01.017349+0000","last_fullsized":"2024-04-25T16:51:01.017349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.304249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018383+0000","last_change":"2024-04-25T16:50:55.083971+0000","last_active":"2024-04-25T16:51:01.018383+0000","last_peered":"2024-04-25T16:51:01.018383+0000","last_clean":"2024-04-25T16:51:01.018383+0000","last_became_active":"2024-04-25T16:50:55.083688+0000","last_became_peered":"2024-04-25T16:50:55.083688+0000","last_unstale":"2024-04-25T16:51:01.018383+0000","last_undegraded":"2024-04-25T16:51:01.018383+0000","last_fullsized":"2024-04-25T16:51:01.018383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:08.427466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018722+0000","last_change":"2024-04-25T16:50:55.090861+0000","last_active":"2024-04-25T16:51:01.018722+0000","last_peered":"2024-04-25T16:51:01.018722+0000","last_clean":"2024-04-25T16:51:01.018722+0000","last_became_active":"2024-04-25T16:50:55.089863+0000","last_became_peered":"2024-04-25T16:50:55.089863+0000","last_unstale":"2024-04-25T16:51:01.018722+0000","last_undegraded":"2024-04-25T16:51:01.018722+0000","last_fullsized":"2024-04-25T16:51:01.018722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:14.342043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018454+0000","last_change":"2024-04-25T16:50:55.091645+0000","last_active":"2024-04-25T16:51:01.018454+0000","last_peered":"2024-04-25T16:51:01.018454+0000","last_clean":"2024-04-25T16:51:01.018454+0000","last_became_active":"2024-04-25T16:50:55.091308+0000","last_became_peered":"2024-04-25T16:50:55.091308+0000","last_unstale":"2024-04-25T16:51:01.018454+0000","last_undegraded":"2024-04-25T16:51:01.018454+0000","last_fullsized":"2024-04-25T16:51:01.018454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017510+0000","last_change":"2024-04-25T16:50:55.074041+0000","last_active":"2024-04-25T16:51:01.017510+0000","last_peered":"2024-04-25T16:51:01.017510+0000","last_clean":"2024-04-25T16:51:01.017510+0000","last_became_active":"2024-04-25T16:50:55.073142+0000","last_became_peered":"2024-04-25T16:50:55.073142+0000","last_unstale":"2024-04-25T16:51:01.017510+0000","last_undegraded":"2024-04-25T16:51:01.017510+0000","last_fullsized":"2024-04-25T16:51:01.017510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016248+0000","last_change":"2024-04-25T16:50:55.065163+0000","last_active":"2024-04-25T16:51:01.016248+0000","last_peered":"2024-04-25T16:51:01.016248+0000","last_clean":"2024-04-25T16:51:01.016248+0000","last_became_active":"2024-04-25T16:50:55.064257+0000","last_became_peered":"2024-04-25T16:50:55.064257+0000","last_unstale":"2024-04-25T16:51:01.016248+0000","last_undegraded":"2024-04-25T16:51:01.016248+0000","last_fullsized":"2024-04-25T16:51:01.016248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020950+0000","last_change":"2024-04-25T16:50:55.082932+0000","last_active":"2024-04-25T16:51:01.020950+0000","last_peered":"2024-04-25T16:51:01.020950+0000","last_clean":"2024-04-25T16:51:01.020950+0000","last_became_active":"2024-04-25T16:50:55.082589+0000","last_became_peered":"2024-04-25T16:50:55.082589+0000","last_unstale":"2024-04-25T16:51:01.020950+0000","last_undegraded":"2024-04-25T16:51:01.020950+0000","last_fullsized":"2024-04-25T16:51:01.020950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:56.360832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335656,"statfs":{"total":96636764160,"available":96599711744,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:51:20.310 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T16:51:20.310 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T16:51:20.310 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T16:51:20.311 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T16:51:20.311 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T16:51:20.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21af4f0640 1 Processor -- start 2024-04-25T16:51:20.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21af4f0640 1 -- start start 2024-04-25T16:51:20.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21af4f0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21a8070180 0x7f21a815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21af4f0640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a815d620 con 0x7f21a806de20 2024-04-25T16:51:20.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21af4f0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a815d900 con 0x7f21a8160bf0 2024-04-25T16:51:20.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21af4f0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f21a815dec0 con 0x7f21a8070180 2024-04-25T16:51:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21ada8d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f21a806de20 0x7f21a80af2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39012/0 (socket says 172.21.15.94:39012) 2024-04-25T16:51:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21ada8d640 1 -- 172.21.15.94:0/3255902876 learned_addr learned my addr 172.21.15.94:0/3255902876 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21ad28c640 1 --2- 172.21.15.94:0/3255902876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21a8070180 0x7f21a815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 809502424 0 0) 0x7f21a815d620 con 0x7f21a806de20 2024-04-25T16:51:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2194003530 con 0x7f21a806de20 2024-04-25T16:51:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 74003935 0 0) 0x7f2194003530 con 0x7f21a806de20 2024-04-25T16:51:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21a8070180 msgr2=0x7f21a815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 --2- 172.21.15.94:0/3255902876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21a8070180 0x7f21a815cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 >> v1:172.21.15.138:6789/0 conn(0x7f21a8160bf0 legacy=0x7f21a806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a815dbe0 con 0x7f21a806de20 2024-04-25T16:51:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f21a0001680 con 0x7f21a806de20 2024-04-25T16:51:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 >> v1:172.21.15.94:6789/0 conn(0x7f21a806de20 legacy=0x7f21a80af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 --2- 172.21.15.94:0/3255902876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21940039f0 0x7f2194003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 --2- 172.21.15.94:0/3255902876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2194004810 0x7f2194004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 --2- 172.21.15.94:0/3255902876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21a8070180 0x7f2194005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f21a815dec0 con 0x7f21940039f0 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f219400bb00 con 0x7f2194004810 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f219400bd00 con 0x7f21a8070180 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21ad28c640 1 --2- 172.21.15.94:0/3255902876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21940039f0 0x7f2194003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21aca8b640 1 --2- 172.21.15.94:0/3255902876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2194004810 0x7f2194004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21ada8d640 1 --2- 172.21.15.94:0/3255902876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21a8070180 0x7f2194005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21ad28c640 1 -- 172.21.15.94:0/3255902876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21a8070180 msgr2=0x7f2194005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21ad28c640 1 --2- 172.21.15.94:0/3255902876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21a8070180 0x7f2194005210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21ad28c640 1 -- 172.21.15.94:0/3255902876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2194004810 msgr2=0x7f2194004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21ad28c640 1 --2- 172.21.15.94:0/3255902876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2194004810 0x7f2194004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21ad28c640 1 -- 172.21.15.94:0/3255902876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2194003530 con 0x7f21940039f0 2024-04-25T16:51:20.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.496+0000 7f21aca8b640 1 --2- 172.21.15.94:0/3255902876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2194004810 0x7f2194004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.500+0000 7f21ad28c640 1 --2- 172.21.15.94:0/3255902876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21940039f0 0x7f2194003e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f21a8071c30 tx=0x7f21a401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16d6298e315e07a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.500+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a40b2070 con 0x7f21940039f0 2024-04-25T16:51:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.500+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a40a5b00 con 0x7f21940039f0 2024-04-25T16:51:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.500+0000 7f219ffff640 1 -- 172.21.15.94:0/3255902876 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4009c10 con 0x7f21940039f0 2024-04-25T16:51:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.500+0000 7f21af4f0640 1 -- 172.21.15.94:0/3255902876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21940039f0 msgr2=0x7f2194003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.500+0000 7f21af4f0640 1 --2- 172.21.15.94:0/3255902876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21940039f0 0x7f2194003e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f21a8071c30 tx=0x7f21a401cc00 comp rx=0 tx=0).stop 2024-04-25T16:51:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.500+0000 7f21ada8d640 1 -- 172.21.15.94:0/3255902876 reap_dead start 2024-04-25T16:51:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.500+0000 7f21af4f0640 1 -- 172.21.15.94:0/3255902876 shutdown_connections 2024-04-25T16:51:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.500+0000 7f21af4f0640 1 -- 172.21.15.94:0/3255902876 >> 172.21.15.94:0/3255902876 conn(0x7f21a80b47f0 msgr2=0x7f21a80b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.500+0000 7f21af4f0640 1 -- 172.21.15.94:0/3255902876 shutdown_connections 2024-04-25T16:51:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.500+0000 7f21af4f0640 1 -- 172.21.15.94:0/3255902876 wait complete. 2024-04-25T16:51:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21af4f0640 1 Processor -- start 2024-04-25T16:51:20.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21af4f0640 1 -- start start 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21af4f0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21940039f0 0x7f21a809c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21af4f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21a8167bb0 0x7f21a80cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21af4f0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21a8167fc0 0x7f21a80cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21af4f0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f21a8169260 con 0x7f21a8167fc0 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21af4f0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81690e0 con 0x7f21a8167bb0 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21af4f0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f21a81693e0 con 0x7f21940039f0 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21ada8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21a8167fc0 0x7f21a80cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21ada8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21a8167fc0 0x7f21a80cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41994/0 (socket says 172.21.15.94:41994) 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21ada8d640 1 -- 172.21.15.94:0/2982638653 learned_addr learned my addr 172.21.15.94:0/2982638653 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21ad28c640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21940039f0 0x7f21a809c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21aca8b640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21a8167bb0 0x7f21a80cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21ada8d640 1 -- 172.21.15.94:0/2982638653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21940039f0 msgr2=0x7f21a809c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21ada8d640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21940039f0 0x7f21a809c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21ada8d640 1 -- 172.21.15.94:0/2982638653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21a8167bb0 msgr2=0x7f21a80cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21ada8d640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21a8167bb0 0x7f21a80cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21ada8d640 1 -- 172.21.15.94:0/2982638653 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a81638a0 con 0x7f21a8167fc0 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21aca8b640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21a8167bb0 0x7f21a80cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21ada8d640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21a8167fc0 0x7f21a80cf8c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f21a80b0d30 tx=0x7f21a000fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f219dffb640 1 -- 172.21.15.94:0/2982638653 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0010040 con 0x7f21a8167fc0 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f219dffb640 1 -- 172.21.15.94:0/2982638653 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a0019070 con 0x7f21a8167fc0 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21af4f0640 1 -- 172.21.15.94:0/2982638653 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a814f3f0 con 0x7f21a8167fc0 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f219dffb640 1 -- 172.21.15.94:0/2982638653 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a000d910 con 0x7f21a8167fc0 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21af4f0640 1 -- 172.21.15.94:0/2982638653 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a814f690 con 0x7f21a8167fc0 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.504+0000 7f21af4f0640 1 -- 172.21.15.94:0/2982638653 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a8163af0 con 0x7f21a8167fc0 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.508+0000 7f219dffb640 1 -- 172.21.15.94:0/2982638653 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f21a000dab0 con 0x7f21a8167fc0 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.508+0000 7f219dffb640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f218c06f400 0x7f218c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.508+0000 7f219dffb640 1 -- 172.21.15.94:0/2982638653 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f21a00911f0 con 0x7f21a8167fc0 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.508+0000 7f21ad28c640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f218c06f400 0x7f218c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.508+0000 7f21ad28c640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f218c06f400 0x7f218c071910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f21a40af000 tx=0x7f21a40a7040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.512+0000 7f219dffb640 1 -- 172.21.15.94:0/2982638653 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f21a005dda0 con 0x7f21a8167fc0 2024-04-25T16:51:20.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.696+0000 7f21af4f0640 1 -- 172.21.15.94:0/2982638653 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f21a8076830 con 0x7f21a8167fc0 2024-04-25T16:51:20.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.708+0000 7f219dffb640 1 -- 172.21.15.94:0/2982638653 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+179543 (secure 0 0 0) 0x7f21a005d6a0 con 0x7f21a8167fc0 2024-04-25T16:51:20.707 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:20.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.712+0000 7f21af4f0640 1 -- 172.21.15.94:0/2982638653 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f218c06f400 msgr2=0x7f218c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.712+0000 7f21af4f0640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f218c06f400 0x7f218c071910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f21a40af000 tx=0x7f21a40a7040 comp rx=0 tx=0).stop 2024-04-25T16:51:20.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.712+0000 7f21af4f0640 1 -- 172.21.15.94:0/2982638653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21a8167fc0 msgr2=0x7f21a80cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.712+0000 7f21af4f0640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21a8167fc0 0x7f21a80cf8c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f21a80b0d30 tx=0x7f21a000fc20 comp rx=0 tx=0).stop 2024-04-25T16:51:20.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.712+0000 7f21af4f0640 1 -- 172.21.15.94:0/2982638653 shutdown_connections 2024-04-25T16:51:20.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.712+0000 7f21af4f0640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21a8167fc0 0x7f21a80cf8c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.712+0000 7f21af4f0640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21a8167bb0 0x7f21a80cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.712+0000 7f21af4f0640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f218c06f400 0x7f218c071910 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.712+0000 7f21af4f0640 1 --2- 172.21.15.94:0/2982638653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21940039f0 0x7f21a809c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.712+0000 7f21af4f0640 1 -- 172.21.15.94:0/2982638653 >> 172.21.15.94:0/2982638653 conn(0x7f21a80b47f0 msgr2=0x7f21a815e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:20.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.712+0000 7f21af4f0640 1 -- 172.21.15.94:0/2982638653 shutdown_connections 2024-04-25T16:51:20.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.712+0000 7f21af4f0640 1 -- 172.21.15.94:0/2982638653 wait complete. 2024-04-25T16:51:20.727 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1891059149},{"type":"v1","addr":"172.21.15.94:6833","nonce":1891059149}]},"active_addr":"172.21.15.94:6833/1891059149","active_change":"2024-04-25T16:50:33.643322+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4114,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":1430080963}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":1069237191}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":574804751}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":767771341}]}]} 2024-04-25T16:51:20.730 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T16:51:20.730 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T16:51:20.730 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:20.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.880+0000 7f16ac805640 1 Processor -- start 2024-04-25T16:51:20.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.880+0000 7f16ac805640 1 -- start start 2024-04-25T16:51:20.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.884+0000 7f16ac805640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16a815e990 0x7f16a8165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.884+0000 7f16ac805640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f16a8165b90 0x7f16a8167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.884+0000 7f16ac805640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16a806d610 con 0x7f16a8070080 2024-04-25T16:51:20.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.884+0000 7f16ac805640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f16a8158550 con 0x7f16a8165b90 2024-04-25T16:51:20.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.884+0000 7f16ac805640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f16a806f7d0 con 0x7f16a815e990 2024-04-25T16:51:20.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.884+0000 7f16a6d9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f16a8165b90 0x7f16a8167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.884+0000 7f16a6d9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f16a8165b90 0x7f16a8167fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42006/0 (socket says 172.21.15.94:42006) 2024-04-25T16:51:20.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.884+0000 7f16a6d9d640 1 -- 172.21.15.94:0/4078137014 learned_addr learned my addr 172.21.15.94:0/4078137014 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:20.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.884+0000 7f16a5d9b640 1 --2- 172.21.15.94:0/4078137014 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16a815e990 0x7f16a8165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.884+0000 7f16a6d9d640 1 -- 172.21.15.94:0/4078137014 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16a815e990 msgr2=0x7f16a8165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:20.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.884+0000 7f16a6d9d640 1 --2- 172.21.15.94:0/4078137014 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16a815e990 0x7f16a8165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.884+0000 7f16a6d9d640 1 -- 172.21.15.94:0/4078137014 >> v1:172.21.15.138:6789/0 conn(0x7f16a8070080 legacy=0x7f16a815e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:20.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.884+0000 7f16a6d9d640 1 -- 172.21.15.94:0/4078137014 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a806fc60 con 0x7f16a8165b90 2024-04-25T16:51:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16a6d9d640 1 --2- 172.21.15.94:0/4078137014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a8165b90 0x7f16a8167fb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16a81586d0 tx=0x7f16a001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281374d28a6828c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16a559a640 1 -- 172.21.15.94:0/4078137014 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a00a58b0 con 0x7f16a8165b90 2024-04-25T16:51:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16a559a640 1 -- 172.21.15.94:0/4078137014 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a00a5a50 con 0x7f16a8165b90 2024-04-25T16:51:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16a559a640 1 -- 172.21.15.94:0/4078137014 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a00a5e20 con 0x7f16a8165b90 2024-04-25T16:51:20.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16ac805640 1 -- 172.21.15.94:0/4078137014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a8165b90 msgr2=0x7f16a8167fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16ac805640 1 --2- 172.21.15.94:0/4078137014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a8165b90 0x7f16a8167fb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16a81586d0 tx=0x7f16a001c9f0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16ac805640 1 -- 172.21.15.94:0/4078137014 shutdown_connections 2024-04-25T16:51:20.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16ac805640 1 --2- 172.21.15.94:0/4078137014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a8165b90 0x7f16a8167fb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16ac805640 1 --2- 172.21.15.94:0/4078137014 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16a815e990 0x7f16a8165650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16ac805640 1 -- 172.21.15.94:0/4078137014 >> 172.21.15.94:0/4078137014 conn(0x7f16a80b47f0 msgr2=0x7f16a80b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:20.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16ac805640 1 -- 172.21.15.94:0/4078137014 shutdown_connections 2024-04-25T16:51:20.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16ac805640 1 -- 172.21.15.94:0/4078137014 wait complete. 2024-04-25T16:51:20.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16ac805640 1 Processor -- start 2024-04-25T16:51:20.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.888+0000 7f16ac805640 1 -- start start 2024-04-25T16:51:20.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16ac805640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16a8070080 0x7f16a8145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16ac805640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a815e990 0x7f16a80cff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16ac805640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a8165b90 0x7f16a80d0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16ac805640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16a80ae6c0 con 0x7f16a815e990 2024-04-25T16:51:20.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16ac805640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16a806f460 con 0x7f16a8165b90 2024-04-25T16:51:20.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16ac805640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f16a8070490 con 0x7f16a8070080 2024-04-25T16:51:20.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16a5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a815e990 0x7f16a80cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16a5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a815e990 0x7f16a80cff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42008/0 (socket says 172.21.15.94:42008) 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16a5d9b640 1 -- 172.21.15.94:0/2474212047 learned_addr learned my addr 172.21.15.94:0/2474212047 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16a6d9d640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a8165b90 0x7f16a80d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16a659c640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16a8070080 0x7f16a8145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16a5d9b640 1 -- 172.21.15.94:0/2474212047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16a8070080 msgr2=0x7f16a8145480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16a5d9b640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16a8070080 0x7f16a8145480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16a5d9b640 1 -- 172.21.15.94:0/2474212047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a8165b90 msgr2=0x7f16a80d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16a5d9b640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a8165b90 0x7f16a80d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16a5d9b640 1 -- 172.21.15.94:0/2474212047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a809b7c0 con 0x7f16a815e990 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16a6d9d640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a8165b90 0x7f16a80d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16a659c640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16a8070080 0x7f16a8145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16a5d9b640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a815e990 0x7f16a80cff30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f169c00d620 tx=0x7f169c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f1696ffd640 1 -- 172.21.15.94:0/2474212047 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c00b620 con 0x7f16a815e990 2024-04-25T16:51:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f1696ffd640 1 -- 172.21.15.94:0/2474212047 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169c019070 con 0x7f16a815e990 2024-04-25T16:51:20.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16ac805640 1 -- 172.21.15.94:0/2474212047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a809baa0 con 0x7f16a815e990 2024-04-25T16:51:20.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f1696ffd640 1 -- 172.21.15.94:0/2474212047 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c00b9e0 con 0x7f16a815e990 2024-04-25T16:51:20.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16ac805640 1 -- 172.21.15.94:0/2474212047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a809c060 con 0x7f16a815e990 2024-04-25T16:51:20.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f16ac805640 1 -- 172.21.15.94:0/2474212047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a809b530 con 0x7f16a815e990 2024-04-25T16:51:20.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f1696ffd640 1 -- 172.21.15.94:0/2474212047 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f169c00bb80 con 0x7f16a815e990 2024-04-25T16:51:20.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f1696ffd640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f168806f430 0x7f1688071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:20.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.892+0000 7f1696ffd640 1 -- 172.21.15.94:0/2474212047 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f169c08fa00 con 0x7f16a815e990 2024-04-25T16:51:20.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.896+0000 7f16a659c640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f168806f430 0x7f1688071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:20.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.896+0000 7f16a659c640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f168806f430 0x7f1688071940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f16a80b50c0 tx=0x7f1698009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:20.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:20.900+0000 7f1696ffd640 1 -- 172.21.15.94:0/2474212047 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f169c05c5b0 con 0x7f16a815e990 2024-04-25T16:51:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.056+0000 7f16ac805640 1 -- 172.21.15.94:0/2474212047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16a8076830 con 0x7f16a815e990 2024-04-25T16:51:21.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.060+0000 7f1696ffd640 1 -- 172.21.15.94:0/2474212047 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f169c05beb0 con 0x7f16a815e990 2024-04-25T16:51:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:21.057 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:59.528743+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:21.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.060+0000 7f16ac805640 1 -- 172.21.15.94:0/2474212047 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f168806f430 msgr2=0x7f1688071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.060+0000 7f16ac805640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f168806f430 0x7f1688071940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f16a80b50c0 tx=0x7f1698009040 comp rx=0 tx=0).stop 2024-04-25T16:51:21.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.060+0000 7f16ac805640 1 -- 172.21.15.94:0/2474212047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a815e990 msgr2=0x7f16a80cff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.060+0000 7f16ac805640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a815e990 0x7f16a80cff30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f169c00d620 tx=0x7f169c00dae0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.060+0000 7f16ac805640 1 -- 172.21.15.94:0/2474212047 shutdown_connections 2024-04-25T16:51:21.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.060+0000 7f16ac805640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a8165b90 0x7f16a80d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.060+0000 7f16ac805640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a815e990 0x7f16a80cff30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.060+0000 7f16ac805640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f168806f430 0x7f1688071940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.060+0000 7f16ac805640 1 --2- 172.21.15.94:0/2474212047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16a8070080 0x7f16a8145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.060+0000 7f16ac805640 1 -- 172.21.15.94:0/2474212047 >> 172.21.15.94:0/2474212047 conn(0x7f16a80b47f0 msgr2=0x7f16a80b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.060+0000 7f16ac805640 1 -- 172.21.15.94:0/2474212047 shutdown_connections 2024-04-25T16:51:21.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.060+0000 7f16ac805640 1 -- 172.21.15.94:0/2474212047 wait complete. 2024-04-25T16:51:21.078 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T16:51:21.078 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:21.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.228+0000 7f6fa2460640 1 Processor -- start 2024-04-25T16:51:21.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.228+0000 7f6fa2460640 1 -- start start 2024-04-25T16:51:21.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6fa2460640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c15cf40 0x7f6f9c06d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6fa2460640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f9c06e060 con 0x7f6f9c06dc50 2024-04-25T16:51:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6fa2460640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f9c161300 con 0x7f6f9c070180 2024-04-25T16:51:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6fa2460640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f9c15dec0 con 0x7f6f9c15cf40 2024-04-25T16:51:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6fa09fd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f6f9c06dc50 0x7f6f9c0af2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39020/0 (socket says 172.21.15.94:39020) 2024-04-25T16:51:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6fa09fd640 1 -- 172.21.15.94:0/1760747810 learned_addr learned my addr 172.21.15.94:0/1760747810 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9b7fe640 1 --2- 172.21.15.94:0/1760747810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c15cf40 0x7f6f9c06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2303252490 0 0) 0x7f6f940014a0 con 0x7f6f9c06dc50 2024-04-25T16:51:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f84003530 con 0x7f6f9c06dc50 2024-04-25T16:51:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4110848222 0 0) 0x7f6f84003530 con 0x7f6f9c06dc50 2024-04-25T16:51:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c15cf40 msgr2=0x7f6f9c06d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 --2- 172.21.15.94:0/1760747810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c15cf40 0x7f6f9c06d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 >> v1:172.21.15.138:6789/0 conn(0x7f6f9c070180 legacy=0x7f6f9c160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9c15d350 con 0x7f6f9c06dc50 2024-04-25T16:51:21.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f6f94001bd0 con 0x7f6f9c06dc50 2024-04-25T16:51:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 >> v1:172.21.15.94:6789/0 conn(0x7f6f9c06dc50 legacy=0x7f6f9c0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 --2- 172.21.15.94:0/1760747810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f840039f0 0x7f6f84003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 --2- 172.21.15.94:0/1760747810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f84004810 0x7f6f84004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 --2- 172.21.15.94:0/1760747810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c15cf40 0x7f6f84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c15dec0 con 0x7f6f84004810 2024-04-25T16:51:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8400ba80 con 0x7f6f840039f0 2024-04-25T16:51:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8400bc80 con 0x7f6f9c15cf40 2024-04-25T16:51:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9b7fe640 1 --2- 172.21.15.94:0/1760747810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f84004810 0x7f6f84004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9bfff640 1 --2- 172.21.15.94:0/1760747810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f840039f0 0x7f6f84003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6fa09fd640 1 --2- 172.21.15.94:0/1760747810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c15cf40 0x7f6f84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9b7fe640 1 -- 172.21.15.94:0/1760747810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c15cf40 msgr2=0x7f6f84005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9b7fe640 1 --2- 172.21.15.94:0/1760747810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c15cf40 0x7f6f84005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9b7fe640 1 -- 172.21.15.94:0/1760747810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f840039f0 msgr2=0x7f6f84003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9b7fe640 1 --2- 172.21.15.94:0/1760747810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f840039f0 0x7f6f84003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.232+0000 7f6f9b7fe640 1 -- 172.21.15.94:0/1760747810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f84003530 con 0x7f6f84004810 2024-04-25T16:51:21.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.236+0000 7f6f9b7fe640 1 --2- 172.21.15.94:0/1760747810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f84004810 0x7f6f84004c20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009ae0 tx=0x7f6f9001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d20d123f683a900 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.236+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f900b2070 con 0x7f6f84004810 2024-04-25T16:51:21.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.236+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f900a5680 con 0x7f6f84004810 2024-04-25T16:51:21.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.236+0000 7f6f9affd640 1 -- 172.21.15.94:0/1760747810 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f900096b0 con 0x7f6f84004810 2024-04-25T16:51:21.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.236+0000 7f6fa2460640 1 -- 172.21.15.94:0/1760747810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f84004810 msgr2=0x7f6f84004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.236+0000 7f6fa2460640 1 --2- 172.21.15.94:0/1760747810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f84004810 0x7f6f84004c20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009ae0 tx=0x7f6f9001ca90 comp rx=0 tx=0).stop 2024-04-25T16:51:21.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.236+0000 7f6fa09fd640 1 -- 172.21.15.94:0/1760747810 reap_dead start 2024-04-25T16:51:21.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.236+0000 7f6fa2460640 1 -- 172.21.15.94:0/1760747810 shutdown_connections 2024-04-25T16:51:21.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.236+0000 7f6fa2460640 1 -- 172.21.15.94:0/1760747810 >> 172.21.15.94:0/1760747810 conn(0x7f6f9c0b47f0 msgr2=0x7f6f9c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.236+0000 7f6fa2460640 1 -- 172.21.15.94:0/1760747810 shutdown_connections 2024-04-25T16:51:21.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.236+0000 7f6fa2460640 1 -- 172.21.15.94:0/1760747810 wait complete. 2024-04-25T16:51:21.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6fa2460640 1 Processor -- start 2024-04-25T16:51:21.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6fa2460640 1 -- start start 2024-04-25T16:51:21.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6fa2460640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f9c1f6b60 0x7f6f9c1f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6fa2460640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9c1f7990 0x7f6f9c154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6fa2460640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c1f6f70 0x7f6f9c15b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6fa2460640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c15d900 con 0x7f6f9c1f7990 2024-04-25T16:51:21.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6fa2460640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c160760 con 0x7f6f9c1f6b60 2024-04-25T16:51:21.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6fa2460640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f9c15dbe0 con 0x7f6f9c1f6f70 2024-04-25T16:51:21.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9c1f7990 0x7f6f9c154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9c1f7990 0x7f6f9c154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42026/0 (socket says 172.21.15.94:42026) 2024-04-25T16:51:21.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6f9b7fe640 1 -- 172.21.15.94:0/1120381866 learned_addr learned my addr 172.21.15.94:0/1120381866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6f9bfff640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f9c1f6b60 0x7f6f9c1f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6fa09fd640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c1f6f70 0x7f6f9c15b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6f9b7fe640 1 -- 172.21.15.94:0/1120381866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c1f6f70 msgr2=0x7f6f9c15b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6f9b7fe640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c1f6f70 0x7f6f9c15b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6f9b7fe640 1 -- 172.21.15.94:0/1120381866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f9c1f6b60 msgr2=0x7f6f9c1f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6f9b7fe640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f9c1f6b60 0x7f6f9c1f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6f9b7fe640 1 -- 172.21.15.94:0/1120381866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9c15e950 con 0x7f6f9c1f7990 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6fa09fd640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c1f6f70 0x7f6f9c15b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6f9b7fe640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9c1f7990 0x7f6f9c154e60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6f900b0040 tx=0x7f6f90019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6f98ff9640 1 -- 172.21.15.94:0/1120381866 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f900bc030 con 0x7f6f9c1f7990 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6f98ff9640 1 -- 172.21.15.94:0/1120381866 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f900a5e30 con 0x7f6f9c1f7990 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6fa2460640 1 -- 172.21.15.94:0/1120381866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9c15ec30 con 0x7f6f9c1f7990 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6f98ff9640 1 -- 172.21.15.94:0/1120381866 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f900b2040 con 0x7f6f9c1f7990 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6fa2460640 1 -- 172.21.15.94:0/1120381866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f9c15f1f0 con 0x7f6f9c1f7990 2024-04-25T16:51:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.240+0000 7f6fa2460640 1 -- 172.21.15.94:0/1120381866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f9c076830 con 0x7f6f9c1f7990 2024-04-25T16:51:21.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.244+0000 7f6f98ff9640 1 -- 172.21.15.94:0/1120381866 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6f900bf070 con 0x7f6f9c1f7990 2024-04-25T16:51:21.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.244+0000 7f6f98ff9640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6f8806f400 0x7f6f88071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.244+0000 7f6f98ff9640 1 -- 172.21.15.94:0/1120381866 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6f90129520 con 0x7f6f9c1f7990 2024-04-25T16:51:21.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.244+0000 7f6f9bfff640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6f8806f400 0x7f6f88071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.244+0000 7f6f9bfff640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6f8806f400 0x7f6f88071910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c071c30 tx=0x7f6f8c00a9b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.248+0000 7f6f98ff9640 1 -- 172.21.15.94:0/1120381866 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6f900c7020 con 0x7f6f9c1f7990 2024-04-25T16:51:21.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.408+0000 7f6fa2460640 1 -- 172.21.15.94:0/1120381866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f9c152460 con 0x7f6f9c1f7990 2024-04-25T16:51:21.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.408+0000 7f6f98ff9640 1 -- 172.21.15.94:0/1120381866 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f6f900167a0 con 0x7f6f9c1f7990 2024-04-25T16:51:21.408 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:21.408 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:59.528743+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:21.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.412+0000 7f6fa2460640 1 -- 172.21.15.94:0/1120381866 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6f8806f400 msgr2=0x7f6f88071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.412+0000 7f6fa2460640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6f8806f400 0x7f6f88071910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c071c30 tx=0x7f6f8c00a9b0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.412+0000 7f6fa2460640 1 -- 172.21.15.94:0/1120381866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9c1f7990 msgr2=0x7f6f9c154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.412+0000 7f6fa2460640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9c1f7990 0x7f6f9c154e60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6f900b0040 tx=0x7f6f90019830 comp rx=0 tx=0).stop 2024-04-25T16:51:21.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.412+0000 7f6fa2460640 1 -- 172.21.15.94:0/1120381866 shutdown_connections 2024-04-25T16:51:21.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.412+0000 7f6fa2460640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6f8806f400 0x7f6f88071910 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.412+0000 7f6fa2460640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f9c1f6f70 0x7f6f9c15b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.412+0000 7f6fa2460640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9c1f7990 0x7f6f9c154e60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.412+0000 7f6fa2460640 1 --2- 172.21.15.94:0/1120381866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f9c1f6b60 0x7f6f9c1f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.412+0000 7f6fa2460640 1 -- 172.21.15.94:0/1120381866 >> 172.21.15.94:0/1120381866 conn(0x7f6f9c0b47f0 msgr2=0x7f6f9c152d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.412+0000 7f6fa2460640 1 -- 172.21.15.94:0/1120381866 shutdown_connections 2024-04-25T16:51:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.412+0000 7f6fa2460640 1 -- 172.21.15.94:0/1120381866 wait complete. 2024-04-25T16:51:21.429 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T16:51:21.429 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T16:51:21.429 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T16:51:21.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T16:51:21.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T16:51:21.431 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T16:51:21.431 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T16:51:21.431 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T16:51:21.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd388fad640 1 Processor -- start 2024-04-25T16:51:21.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd388fad640 1 -- start start 2024-04-25T16:51:21.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7f719c3c3640 1 Processor -- start 2024-04-25T16:51:21.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7f719c3c3640 1 -- start start 2024-04-25T16:51:21.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7f420751b640 1 Processor -- start 2024-04-25T16:51:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7f5d0ffff640 1 Processor -- start 2024-04-25T16:51:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7f5d0ffff640 1 -- start start 2024-04-25T16:51:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd388fad640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd3840d2b00 0x7fd3840b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd388fad640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd3840d1fb0 0x7fd38416eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd388fad640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3840b7d20 con 0x7fd3840b7910 2024-04-25T16:51:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd388fad640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd3841654c0 con 0x7fd3840d1fb0 2024-04-25T16:51:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd388fad640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd3840cfe80 con 0x7fd3840d2b00 2024-04-25T16:51:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd381d9b640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd3840b7910 0x7fd3840d18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39030/0 (socket says 172.21.15.94:39030) 2024-04-25T16:51:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd381d9b640 1 -- 172.21.15.94:0/3094695383 learned_addr learned my addr 172.21.15.94:0/3094695383 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd382d9d640 1 --2- 172.21.15.94:0/3094695383 >> v2:172.21.15.94:3300/0 conn(0x7fd3840d1fb0 0x7fd38416eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd38259c640 1 --2- 172.21.15.94:0/3094695383 >> v2:172.21.15.138:3300/0 conn(0x7fd3840d2b00 0x7fd3840b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7f420751b640 1 -- start start 2024-04-25T16:51:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fc1636c2640 1 Processor -- start 2024-04-25T16:51:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fc1636c2640 1 -- start start 2024-04-25T16:51:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719c3c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71940d3450 0x7f71940d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719c3c3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f71940d3da0 0x7f71940b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719c3c3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7194164510 con 0x7f71940b7510 2024-04-25T16:51:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719c3c3640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7194164a90 con 0x7f71940d3da0 2024-04-25T16:51:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719c3c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f71940d51b0 con 0x7f71940d3450 2024-04-25T16:51:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc1636c2640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc15c0b54a0 0x7fc15c163fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc1636c2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc15c0b58d0 0x7fc15c06d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc1636c2640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc15c06db50 0x7fc15c072520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc1636c2640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fc15c06df60 con 0x7fc15c0b54a0 2024-04-25T16:51:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc1636c2640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc15c0b45d0 con 0x7fc15c06db50 2024-04-25T16:51:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc1636c2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc15c0b48b0 con 0x7fc15c0b58d0 2024-04-25T16:51:21.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719a960640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f71940b7510 0x7f71940d1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39044/0 (socket says 172.21.15.94:39044) 2024-04-25T16:51:21.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719a960640 1 -- 172.21.15.94:0/849742439 learned_addr learned my addr 172.21.15.94:0/849742439 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:21.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc16145e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc15c0b54a0 0x7fc15c163fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc16145e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc15c0b54a0 0x7fc15c163fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42040/0 (socket says 172.21.15.94:42040) 2024-04-25T16:51:21.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc16145e640 1 -- 172.21.15.94:0/361156133 learned_addr learned my addr 172.21.15.94:0/361156133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719995e640 1 --2- 172.21.15.94:0/849742439 >> v2:172.21.15.94:3300/0 conn(0x7f71940d3da0 0x7f71940b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f5d0ffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5d080e2910 0x7f5d080e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc161c5f640 1 --2- 172.21.15.94:0/361156133 >> v2:172.21.15.138:3300/0 conn(0x7fc15c06db50 0x7fc15c072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc160c5d640 1 --2- 172.21.15.94:0/361156133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc15c0b58d0 0x7fc15c06d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f420751b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f41f80eb8f0 0x7f41f80e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f420751b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f8006810 0x7f41f8006c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f420751b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f8038350 con 0x7f41f8033a80 2024-04-25T16:51:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f420751b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f41f80060a0 con 0x7f41f80eb8f0 2024-04-25T16:51:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f420751b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f41f8006380 con 0x7f41f8006810 2024-04-25T16:51:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f4204ab6640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f41f8033a80 0x7f41f8033e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39054/0 (socket says 172.21.15.94:39054) 2024-04-25T16:51:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f4204ab6640 1 -- 172.21.15.94:0/1827894442 learned_addr learned my addr 172.21.15.94:0/1827894442 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fbd7b56b640 1 Processor -- start 2024-04-25T16:51:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd382d9d640 1 -- 172.21.15.94:0/3094695383 >> v2:172.21.15.138:3300/0 conn(0x7fd3840d2b00 msgr2=0x7fd3840b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd382d9d640 1 --2- 172.21.15.94:0/3094695383 >> v2:172.21.15.138:3300/0 conn(0x7fd3840d2b00 0x7fd3840b73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd382d9d640 1 -- 172.21.15.94:0/3094695383 >> v1:172.21.15.94:6789/0 conn(0x7fd3840b7910 legacy=0x7fd3840d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.652+0000 7fd382d9d640 1 -- 172.21.15.94:0/3094695383 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3840d0160 con 0x7fd3840d1fb0 2024-04-25T16:51:21.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f0f29437640 1 Processor -- start 2024-04-25T16:51:21.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719915d640 1 -- 172.21.15.94:0/849742439 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3682333668 0 0) 0x7f7194164510 con 0x7f71940b7510 2024-04-25T16:51:21.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719915d640 1 -- 172.21.15.94:0/849742439 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7180003530 con 0x7f71940b7510 2024-04-25T16:51:21.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f42052b7640 1 --2- 172.21.15.94:0/1827894442 >> v2:172.21.15.94:3300/0 conn(0x7f41f80eb8f0 0x7f41f80e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719a15f640 1 --2- 172.21.15.94:0/849742439 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71940d3450 0x7f71940d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f0f29437640 1 -- start start 2024-04-25T16:51:21.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d080389d0 0x7f5d08038de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f4205ab8640 1 --2- 172.21.15.94:0/1827894442 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f8006810 0x7f41f8006c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f5d0ffff640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d080df840 con 0x7f5d080514d0 2024-04-25T16:51:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f5d0ffff640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f5d080e2480 con 0x7f5d080e2910 2024-04-25T16:51:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f5d0ffff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d080e6340 con 0x7f5d080389d0 2024-04-25T16:51:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fbd7b56b640 1 -- start start 2024-04-25T16:51:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc16145e640 1 -- 172.21.15.94:0/361156133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc15c0b58d0 msgr2=0x7fc15c06d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc16145e640 1 --2- 172.21.15.94:0/361156133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc15c0b58d0 0x7fc15c06d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc16145e640 1 -- 172.21.15.94:0/361156133 >> v2:172.21.15.138:3300/0 conn(0x7fc15c06db50 msgr2=0x7fc15c072520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc16145e640 1 --2- 172.21.15.94:0/361156133 >> v2:172.21.15.138:3300/0 conn(0x7fc15c06db50 0x7fc15c072520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f5d0f7fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d080389d0 0x7f5d08038de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f5d0f7fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d080389d0 0x7f5d08038de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51734/0 (socket says 172.21.15.94:51734) 2024-04-25T16:51:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f5d0f7fe640 1 -- 172.21.15.94:0/2705166097 learned_addr learned my addr 172.21.15.94:0/2705166097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7fc16145e640 1 -- 172.21.15.94:0/361156133 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c0b4b90 con 0x7fc15c0b54a0 2024-04-25T16:51:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f5d0effd640 1 --2- 172.21.15.94:0/2705166097 >> v2:172.21.15.94:3300/0 conn(0x7f5d080e2910 0x7f5d080e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f0f29437640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f2406dd20 0x7f0f240d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f0f29437640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0f240b4880 0x7f0f240b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f0f29437640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f24070af0 con 0x7f0f241459c0 2024-04-25T16:51:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f0f29437640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f0f24073d60 con 0x7f0f240b4880 2024-04-25T16:51:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f0f29437640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0f24070dd0 con 0x7f0f2406dd20 2024-04-25T16:51:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f4205ab8640 1 -- 172.21.15.94:0/1827894442 >> v2:172.21.15.94:3300/0 conn(0x7f41f80eb8f0 msgr2=0x7f41f80e7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f4205ab8640 1 --2- 172.21.15.94:0/1827894442 >> v2:172.21.15.94:3300/0 conn(0x7f41f80eb8f0 0x7f41f80e7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f4205ab8640 1 -- 172.21.15.94:0/1827894442 >> v1:172.21.15.94:6789/0 conn(0x7f41f8033a80 legacy=0x7f41f8033e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f4205ab8640 1 -- 172.21.15.94:0/1827894442 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f80ec0d0 con 0x7f41f8006810 2024-04-25T16:51:21.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f5d0f7fe640 1 -- 172.21.15.94:0/2705166097 >> v2:172.21.15.94:3300/0 conn(0x7f5d080e2910 msgr2=0x7f5d080e5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f5d0f7fe640 1 --2- 172.21.15.94:0/2705166097 >> v2:172.21.15.94:3300/0 conn(0x7f5d080e2910 0x7f5d080e5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f5d0f7fe640 1 -- 172.21.15.94:0/2705166097 >> v1:172.21.15.94:6789/0 conn(0x7f5d080514d0 legacy=0x7f5d080518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f5d0f7fe640 1 -- 172.21.15.94:0/2705166097 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d08052960 con 0x7f5d080389d0 2024-04-25T16:51:21.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f23fff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0f241459c0 0x7f0f2406d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:39696/0 (socket says 172.21.15.94:39696) 2024-04-25T16:51:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f23fff640 1 -- 172.21.15.94:0/4007234400 learned_addr learned my addr 172.21.15.94:0/4007234400 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719995e640 1 -- 172.21.15.94:0/849742439 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71940d3450 msgr2=0x7f71940d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719995e640 1 --2- 172.21.15.94:0/849742439 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71940d3450 0x7f71940d3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719995e640 1 -- 172.21.15.94:0/849742439 >> v1:172.21.15.94:6789/0 conn(0x7f71940b7510 legacy=0x7f71940d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.656+0000 7f719995e640 1 -- 172.21.15.94:0/849742439 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71941647b0 con 0x7f71940d3da0 2024-04-25T16:51:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fbd7b56b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbd7406c600 0x7fbd7406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fbd7b56b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd7406d330 con 0x7fbd74072f30 2024-04-25T16:51:21.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fbd7b56b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd7406d610 con 0x7fbd7406ed70 2024-04-25T16:51:21.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fbd7b56b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fbd7406d8f0 con 0x7fbd7406c600 2024-04-25T16:51:21.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fd382d9d640 1 --2- 172.21.15.94:0/3094695383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3840d1fb0 0x7fd38416eae0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd3840b7ff0 tx=0x7fd37c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a445b00ca53dfece server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2714596897 0 0) 0x7f0f24070af0 con 0x7f0f241459c0 2024-04-25T16:51:21.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f0c005740 con 0x7f0f241459c0 2024-04-25T16:51:21.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f4205ab8640 1 --2- 172.21.15.94:0/1827894442 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f8006810 0x7f41f8006c20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f420006b1a0 tx=0x7f4200098ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35730c8b52adbcf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fbd79b08640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fbd7406ed70 0x7fbd7406f180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:39708/0 (socket says 172.21.15.94:39708) 2024-04-25T16:51:21.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fbd79b08640 1 -- 172.21.15.94:0/1419417252 learned_addr learned my addr 172.21.15.94:0/1419417252 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:21.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f41f7fff640 1 -- 172.21.15.94:0/1827894442 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200121d30 con 0x7f41f8006810 2024-04-25T16:51:21.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1002125445 0 0) 0x7f0f0c005740 con 0x7f0f241459c0 2024-04-25T16:51:21.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f2406dd20 msgr2=0x7f0f240d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 --2- 172.21.15.94:0/4007234400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f2406dd20 0x7f0f240d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 >> v2:172.21.15.94:3300/0 conn(0x7f0f240b4880 msgr2=0x7f0f240b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 --2- 172.21.15.94:0/4007234400 >> v2:172.21.15.94:3300/0 conn(0x7f0f240b4880 0x7f0f240b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f240710b0 con 0x7f0f241459c0 2024-04-25T16:51:21.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fbd78b06640 1 --2- 172.21.15.94:0/1419417252 >> v2:172.21.15.94:3300/0 conn(0x7fbd7406c600 0x7fbd7406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f41f7fff640 1 -- 172.21.15.94:0/1827894442 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4200129440 con 0x7f41f8006810 2024-04-25T16:51:21.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f41f7fff640 1 -- 172.21.15.94:0/1827894442 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42001298b0 con 0x7f41f8006810 2024-04-25T16:51:21.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fbd78b06640 1 -- 172.21.15.94:0/1419417252 >> v1:172.21.15.138:6789/0 conn(0x7fbd7406ed70 legacy=0x7fbd7406f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fbd6bfff640 1 -- 172.21.15.94:0/1419417252 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787868093 0 0) 0x7fbd7406d610 con 0x7fbd7406ed70 2024-04-25T16:51:21.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fd38159a640 1 -- 172.21.15.94:0/3094695383 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0b2070 con 0x7fd3840d1fb0 2024-04-25T16:51:21.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fbd78b06640 1 -- 172.21.15.94:0/1419417252 >> v1:172.21.15.94:6789/0 conn(0x7fbd74072f30 legacy=0x7fbd74073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fbd78b06640 1 -- 172.21.15.94:0/1419417252 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7406dbd0 con 0x7fbd7406c600 2024-04-25T16:51:21.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fd38159a640 1 -- 172.21.15.94:0/3094695383 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37c0a58e0 con 0x7fd3840d1fb0 2024-04-25T16:51:21.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fd38159a640 1 -- 172.21.15.94:0/3094695383 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0a5cb0 con 0x7fd3840d1fb0 2024-04-25T16:51:21.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f719995e640 1 --2- 172.21.15.94:0/849742439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71940d3da0 0x7f71940b6fd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7194164c10 tx=0x7f718c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e58dbb09e2fa6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f420751b640 1 -- 172.21.15.94:0/1827894442 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f8006810 msgr2=0x7f41f8006c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f420751b640 1 --2- 172.21.15.94:0/1827894442 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f8006810 0x7f41f8006c20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f420006b1a0 tx=0x7f4200098ea0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f0f18001680 con 0x7f0f241459c0 2024-04-25T16:51:21.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 >> v1:172.21.15.138:6789/0 conn(0x7f0f241459c0 legacy=0x7f0f2406d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 --2- 172.21.15.94:0/4007234400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f0c005bd0 0x7f0f0c006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 --2- 172.21.15.94:0/4007234400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f0c006a20 0x7f0f0c006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 --2- 172.21.15.94:0/4007234400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f0c0073c0 0x7f0f0c00ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f24073d60 con 0x7f0f0c006a20 2024-04-25T16:51:21.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f24070dd0 con 0x7f0f0c005bd0 2024-04-25T16:51:21.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0f0c009c40 con 0x7f0f0c0073c0 2024-04-25T16:51:21.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f237fe640 1 --2- 172.21.15.94:0/4007234400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f0c006a20 0x7f0f0c006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f28c36640 1 --2- 172.21.15.94:0/4007234400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f0c0073c0 0x7f0f0c00ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f23fff640 1 --2- 172.21.15.94:0/4007234400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f0c005bd0 0x7f0f0c006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f237fe640 1 -- 172.21.15.94:0/4007234400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f0c0073c0 msgr2=0x7f0f0c00ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f237fe640 1 --2- 172.21.15.94:0/4007234400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f0c0073c0 0x7f0f0c00ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f237fe640 1 -- 172.21.15.94:0/4007234400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f0c005bd0 msgr2=0x7f0f0c006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:21.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f237fe640 1 --2- 172.21.15.94:0/4007234400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f0c005bd0 0x7f0f0c006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f420751b640 1 -- 172.21.15.94:0/1827894442 shutdown_connections 2024-04-25T16:51:21.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f420751b640 1 --2- 172.21.15.94:0/1827894442 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f8006810 0x7f41f8006c20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f420751b640 1 --2- 172.21.15.94:0/1827894442 >> v2:172.21.15.94:3300/0 conn(0x7f41f80eb8f0 0x7f41f80e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f420751b640 1 -- 172.21.15.94:0/1827894442 >> 172.21.15.94:0/1827894442 conn(0x7f41f8054d00 msgr2=0x7f41f8055110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f420751b640 1 -- 172.21.15.94:0/1827894442 shutdown_connections 2024-04-25T16:51:21.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f420751b640 1 -- 172.21.15.94:0/1827894442 wait complete. 2024-04-25T16:51:21.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fd388fad640 1 -- 172.21.15.94:0/3094695383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3840d1fb0 msgr2=0x7fd38416eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fd388fad640 1 --2- 172.21.15.94:0/3094695383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3840d1fb0 0x7fd38416eae0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd3840b7ff0 tx=0x7fd37c01c930 comp rx=0 tx=0).stop 2024-04-25T16:51:21.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f719915d640 1 -- 172.21.15.94:0/849742439 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c0b2070 con 0x7f71940d3da0 2024-04-25T16:51:21.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f719915d640 1 -- 172.21.15.94:0/849742439 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f718c0a58d0 con 0x7f71940d3da0 2024-04-25T16:51:21.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f719915d640 1 -- 172.21.15.94:0/849742439 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c0a5ca0 con 0x7f71940d3da0 2024-04-25T16:51:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fd388fad640 1 -- 172.21.15.94:0/3094695383 shutdown_connections 2024-04-25T16:51:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fd388fad640 1 --2- 172.21.15.94:0/3094695383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3840d1fb0 0x7fd38416eae0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fd388fad640 1 --2- 172.21.15.94:0/3094695383 >> v2:172.21.15.138:3300/0 conn(0x7fd3840d2b00 0x7fd3840b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fd388fad640 1 -- 172.21.15.94:0/3094695383 >> 172.21.15.94:0/3094695383 conn(0x7fd3840d33d0 msgr2=0x7fd3840d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fd388fad640 1 -- 172.21.15.94:0/3094695383 shutdown_connections 2024-04-25T16:51:21.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fd388fad640 1 -- 172.21.15.94:0/3094695383 wait complete. 2024-04-25T16:51:21.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7fc16145e640 1 --2- 172.21.15.94:0/361156133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc15c0b54a0 0x7fc15c163fd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0b67c0 tx=0x7fc15801c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17a37ffd7fef33d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f719c3c3640 1 -- 172.21.15.94:0/849742439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71940d3da0 msgr2=0x7f71940b6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f719c3c3640 1 --2- 172.21.15.94:0/849742439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71940d3da0 0x7f71940b6fd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7194164c10 tx=0x7f718c01c870 comp rx=0 tx=0).stop 2024-04-25T16:51:21.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.660+0000 7f0f237fe640 1 -- 172.21.15.94:0/4007234400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0c005740 con 0x7f0f0c006a20 2024-04-25T16:51:21.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f5d0f7fe640 1 --2- 172.21.15.94:0/2705166097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d080389d0 0x7f5d08038de0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5d1007bea0 tx=0x7f5d1009d070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d084ef854d01f378 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f719c3c3640 1 -- 172.21.15.94:0/849742439 shutdown_connections 2024-04-25T16:51:21.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f719c3c3640 1 --2- 172.21.15.94:0/849742439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71940d3da0 0x7f71940b6fd0 secure :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7194164c10 tx=0x7f718c01c870 comp rx=0 tx=0).stop 2024-04-25T16:51:21.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f719c3c3640 1 --2- 172.21.15.94:0/849742439 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71940d3450 0x7f71940d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f719c3c3640 1 -- 172.21.15.94:0/849742439 >> 172.21.15.94:0/849742439 conn(0x7f719409e8d0 msgr2=0x7f71940b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f5d0dffb640 1 -- 172.21.15.94:0/2705166097 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10125d10 con 0x7f5d080389d0 2024-04-25T16:51:21.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f5d0dffb640 1 -- 172.21.15.94:0/2705166097 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1012d440 con 0x7f5d080389d0 2024-04-25T16:51:21.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f5d0dffb640 1 -- 172.21.15.94:0/2705166097 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1012d8b0 con 0x7f5d080389d0 2024-04-25T16:51:21.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fc153fff640 1 -- 172.21.15.94:0/361156133 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1580b0070 con 0x7fc15c0b54a0 2024-04-25T16:51:21.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fc153fff640 1 -- 172.21.15.94:0/361156133 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1580ab600 con 0x7fc15c0b54a0 2024-04-25T16:51:21.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fc153fff640 1 -- 172.21.15.94:0/361156133 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1580ab9d0 con 0x7fc15c0b54a0 2024-04-25T16:51:21.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f719c3c3640 1 -- 172.21.15.94:0/849742439 shutdown_connections 2024-04-25T16:51:21.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fbd78b06640 1 --2- 172.21.15.94:0/1419417252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd7406c600 0x7fbd7406ca10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009d90 tx=0x7fbd6c01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6128a089e1ec72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f719c3c3640 1 -- 172.21.15.94:0/849742439 wait complete. 2024-04-25T16:51:21.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fd388fad640 1 Processor -- start 2024-04-25T16:51:21.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fd388fad640 1 -- start start 2024-04-25T16:51:21.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fbd6bfff640 1 -- 172.21.15.94:0/1419417252 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0b2070 con 0x7fbd7406c600 2024-04-25T16:51:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fbd6bfff640 1 -- 172.21.15.94:0/1419417252 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6c0a5830 con 0x7fbd7406c600 2024-04-25T16:51:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fbd6bfff640 1 -- 172.21.15.94:0/1419417252 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd740700a0 con 0x7fbd7406c600 2024-04-25T16:51:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f0f237fe640 1 --2- 172.21.15.94:0/4007234400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f0c006a20 0x7f0f0c006e30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0f24071660 tx=0x7f0f1c01cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2421a7d9aa9f79bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f420751b640 1 Processor -- start 2024-04-25T16:51:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fc1636c2640 1 -- 172.21.15.94:0/361156133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc15c0b54a0 msgr2=0x7fc15c163fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fc1636c2640 1 --2- 172.21.15.94:0/361156133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc15c0b54a0 0x7fc15c163fd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0b67c0 tx=0x7fc15801c1c0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f420751b640 1 -- start start 2024-04-25T16:51:21.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fc1636c2640 1 -- 172.21.15.94:0/361156133 shutdown_connections 2024-04-25T16:51:21.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fc1636c2640 1 --2- 172.21.15.94:0/361156133 >> v2:172.21.15.138:3300/0 conn(0x7fc15c06db50 0x7fc15c072520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fc1636c2640 1 --2- 172.21.15.94:0/361156133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc15c0b58d0 0x7fc15c06d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fc1636c2640 1 --2- 172.21.15.94:0/361156133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc15c0b54a0 0x7fc15c163fd0 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0b67c0 tx=0x7fc15801c1c0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fc1636c2640 1 -- 172.21.15.94:0/361156133 >> 172.21.15.94:0/361156133 conn(0x7fc15c06e610 msgr2=0x7fc15c06ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fc1636c2640 1 -- 172.21.15.94:0/361156133 shutdown_connections 2024-04-25T16:51:21.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fc1636c2640 1 -- 172.21.15.94:0/361156133 wait complete. 2024-04-25T16:51:21.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c0b2070 con 0x7f0f0c006a20 2024-04-25T16:51:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f1c0a5ba0 con 0x7f0f0c006a20 2024-04-25T16:51:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f0f22ffd640 1 -- 172.21.15.94:0/4007234400 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c009a30 con 0x7f0f0c006a20 2024-04-25T16:51:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2705166097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d080389d0 msgr2=0x7f5d08038de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f5d0ffff640 1 --2- 172.21.15.94:0/2705166097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d080389d0 0x7f5d08038de0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5d1007bea0 tx=0x7f5d1009d070 comp rx=0 tx=0).stop 2024-04-25T16:51:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fbd7b56b640 1 -- 172.21.15.94:0/1419417252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd7406c600 msgr2=0x7fbd7406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fbd7b56b640 1 --2- 172.21.15.94:0/1419417252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd7406c600 0x7fbd7406ca10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009d90 tx=0x7fbd6c01c990 comp rx=0 tx=0).stop 2024-04-25T16:51:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fbd7b56b640 1 -- 172.21.15.94:0/1419417252 shutdown_connections 2024-04-25T16:51:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fbd7b56b640 1 --2- 172.21.15.94:0/1419417252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd7406c600 0x7fbd7406ca10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fbd7b56b640 1 -- 172.21.15.94:0/1419417252 >> 172.21.15.94:0/1419417252 conn(0x7fbd740b47f0 msgr2=0x7fbd740b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fbd7b56b640 1 -- 172.21.15.94:0/1419417252 shutdown_connections 2024-04-25T16:51:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fb4593e0640 1 Processor -- start 2024-04-25T16:51:21.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2705166097 shutdown_connections 2024-04-25T16:51:21.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f5d0ffff640 1 --2- 172.21.15.94:0/2705166097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d080389d0 0x7f5d08038de0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f5d0ffff640 1 --2- 172.21.15.94:0/2705166097 >> v2:172.21.15.94:3300/0 conn(0x7f5d080e2910 0x7f5d080e5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2705166097 >> 172.21.15.94:0/2705166097 conn(0x7f5d08053990 msgr2=0x7f5d08053da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fb4593e0640 1 -- start start 2024-04-25T16:51:21.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2705166097 shutdown_connections 2024-04-25T16:51:21.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fd388fad640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3840b7910 0x7fd38415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2705166097 wait complete. 2024-04-25T16:51:21.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f0f29437640 1 -- 172.21.15.94:0/4007234400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f0c006a20 msgr2=0x7f0f0c006e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f0f29437640 1 --2- 172.21.15.94:0/4007234400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f0c006a20 0x7f0f0c006e30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0f24071660 tx=0x7f0f1c01cdb0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fd388fad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3840d1fb0 0x7fd38415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f0f28c36640 1 -- 172.21.15.94:0/4007234400 reap_dead start 2024-04-25T16:51:21.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fd388fad640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3840d2b00 0x7fd384154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7fbd7b56b640 1 -- 172.21.15.94:0/1419417252 wait complete. 2024-04-25T16:51:21.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.664+0000 7f0f29437640 1 -- 172.21.15.94:0/4007234400 shutdown_connections 2024-04-25T16:51:21.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7f0f29437640 1 -- 172.21.15.94:0/4007234400 >> 172.21.15.94:0/4007234400 conn(0x7f0f24072ab0 msgr2=0x7f0f24072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd381d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3840d1fb0 0x7fd38415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd381d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3840d1fb0 0x7fd38415e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42108/0 (socket says 172.21.15.94:42108) 2024-04-25T16:51:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd381d9b640 1 -- 172.21.15.94:0/504236126 learned_addr learned my addr 172.21.15.94:0/504236126 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd38259c640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3840b7910 0x7fd38415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd388fad640 1 -- 172.21.15.94:0/504236126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd38406ed20 con 0x7fd3840d1fb0 2024-04-25T16:51:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd388fad640 1 -- 172.21.15.94:0/504236126 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd38406e9b0 con 0x7fd3840d2b00 2024-04-25T16:51:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd388fad640 1 -- 172.21.15.94:0/504236126 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd384161cf0 con 0x7fd3840b7910 2024-04-25T16:51:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7f0f29437640 1 -- 172.21.15.94:0/4007234400 shutdown_connections 2024-04-25T16:51:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd382d9d640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3840d2b00 0x7fd384154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7f0f29437640 1 -- 172.21.15.94:0/4007234400 wait complete. 2024-04-25T16:51:21.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fc1636c2640 1 Processor -- start 2024-04-25T16:51:21.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fc1636c2640 1 -- start start 2024-04-25T16:51:21.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd381d9b640 1 -- 172.21.15.94:0/504236126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3840b7910 msgr2=0x7fd38415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd381d9b640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3840b7910 0x7fd38415f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd381d9b640 1 -- 172.21.15.94:0/504236126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3840d2b00 msgr2=0x7fd384154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd381d9b640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3840d2b00 0x7fd384154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd381d9b640 1 -- 172.21.15.94:0/504236126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd384158420 con 0x7fd3840d1fb0 2024-04-25T16:51:21.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7f719c3c3640 1 Processor -- start 2024-04-25T16:51:21.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7f719c3c3640 1 -- start start 2024-04-25T16:51:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd381d9b640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3840d1fb0 0x7fd38415e1a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd37400c430 tx=0x7fd37400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd37affd640 1 -- 172.21.15.94:0/504236126 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374017070 con 0x7fd3840d1fb0 2024-04-25T16:51:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd37affd640 1 -- 172.21.15.94:0/504236126 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd374005cf0 con 0x7fd3840d1fb0 2024-04-25T16:51:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd37affd640 1 -- 172.21.15.94:0/504236126 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3740072d0 con 0x7fd3840d1fb0 2024-04-25T16:51:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fd388fad640 1 -- 172.21.15.94:0/504236126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd384158700 con 0x7fd3840d1fb0 2024-04-25T16:51:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fb4593e0640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb4541562b0 0x7fb4541566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fb4593e0640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb454072be0 con 0x7fb454160980 2024-04-25T16:51:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fb4593e0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb454072ec0 con 0x7fb4540b4ae0 2024-04-25T16:51:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fb4593e0640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fb4540b0350 con 0x7fb4541562b0 2024-04-25T16:51:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fbd7b56b640 1 Processor -- start 2024-04-25T16:51:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fb453fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fb454160980 0x7fb45406da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39072/0 (socket says 172.21.15.94:39072) 2024-04-25T16:51:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fb453fff640 1 -- 172.21.15.94:0/734176640 learned_addr learned my addr 172.21.15.94:0/734176640 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7f5d0ffff640 1 Processor -- start 2024-04-25T16:51:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fb458bdf640 1 --2- 172.21.15.94:0/734176640 >> v2:172.21.15.94:3300/0 conn(0x7fb4541562b0 0x7fb4541566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7f0f29437640 1 Processor -- start 2024-04-25T16:51:21.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7f0f29437640 1 -- start start 2024-04-25T16:51:21.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc1636c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc15c06db50 0x7fc15c0ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fb452ffd640 1 -- 172.21.15.94:0/734176640 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1910214681 0 0) 0x7fb454072be0 con 0x7fb454160980 2024-04-25T16:51:21.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fb452ffd640 1 -- 172.21.15.94:0/734176640 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb43c003530 con 0x7fb454160980 2024-04-25T16:51:21.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fb458bdf640 1 -- 172.21.15.94:0/734176640 >> v1:172.21.15.138:6789/0 conn(0x7fb4540b4ae0 legacy=0x7fb4540b4ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T16:51:21.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fb458bdf640 1 -- 172.21.15.94:0/734176640 >> v1:172.21.15.94:6789/0 conn(0x7fb454160980 legacy=0x7fb45406da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fb458bdf640 1 -- 172.21.15.94:0/734176640 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4540731a0 con 0x7fb4541562b0 2024-04-25T16:51:21.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f29437640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2415f810 0x7f0f2413d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f29437640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f2413f220 0x7f0f241f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f29437640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f2413dc20 0x7f0f2413e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f29437640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f240cf780 con 0x7f0f2413f220 2024-04-25T16:51:21.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f29437640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f240cf600 con 0x7f0f2415f810 2024-04-25T16:51:21.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f29437640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0f240cf900 con 0x7f0f2413dc20 2024-04-25T16:51:21.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7f5d0ffff640 1 -- start start 2024-04-25T16:51:21.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d080389d0 0x7f5d08048c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.668+0000 7fbd7b56b640 1 -- start start 2024-04-25T16:51:21.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd7b56b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd7406c600 0x7fbd7414ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc16145e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc15c06db50 0x7fc15c0ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc16145e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc15c06db50 0x7fc15c0ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42130/0 (socket says 172.21.15.94:42130) 2024-04-25T16:51:21.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc16145e640 1 -- 172.21.15.94:0/3564184554 learned_addr learned my addr 172.21.15.94:0/3564184554 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:21.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc1636c2640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc15c0b58d0 0x7fc15c1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc1636c2640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc15c156140 0x7fc15c15e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd7b56b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd7406ed70 0x7fbd74152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc1636c2640 1 -- 172.21.15.94:0/3564184554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c0cf760 con 0x7fc15c06db50 2024-04-25T16:51:21.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f23fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2415f810 0x7f0f2413d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f23fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2415f810 0x7f0f2413d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40948/0 (socket says 172.21.15.94:40948) 2024-04-25T16:51:21.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f23fff640 1 -- 172.21.15.94:0/475970042 learned_addr learned my addr 172.21.15.94:0/475970042 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:21.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719c3c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71940b7510 0x7f7194161660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc1636c2640 1 -- 172.21.15.94:0/3564184554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c0cf5e0 con 0x7fc15c0b58d0 2024-04-25T16:51:21.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f28c36640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f2413dc20 0x7f0f2413e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc1636c2640 1 -- 172.21.15.94:0/3564184554 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc15c0cf8e0 con 0x7fc15c156140 2024-04-25T16:51:21.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d080514d0 0x7f5d0817e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd7b56b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd74072f30 0x7fbd74154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719c3c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71940d3450 0x7f71940cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fd37affd640 1 -- 172.21.15.94:0/504236126 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd37401a070 con 0x7fd3840d1fb0 2024-04-25T16:51:21.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd79307640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd7406c600 0x7fbd7414ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd7b56b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74172a40 con 0x7fbd74072f30 2024-04-25T16:51:21.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f237fe640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f2413f220 0x7f0f241f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719c3c3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7194072390 0x7f71940727a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719a15f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71940b7510 0x7f7194161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719a15f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71940b7510 0x7f7194161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51824/0 (socket says 172.21.15.94:51824) 2024-04-25T16:51:21.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d080514d0 0x7f5d0817e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d080514d0 0x7f5d0817e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42154/0 (socket says 172.21.15.94:42154) 2024-04-25T16:51:21.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd79307640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd7406c600 0x7fbd7414ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40950/0 (socket says 172.21.15.94:40950) 2024-04-25T16:51:21.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd79307640 1 -- 172.21.15.94:0/3779211959 learned_addr learned my addr 172.21.15.94:0/3779211959 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:21.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd78b06640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd7406ed70 0x7fbd74152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd79b08640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd74072f30 0x7fbd74154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc161c5f640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc15c156140 0x7fc15c15e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc160c5d640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc15c0b58d0 0x7fc15c1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719a15f640 1 -- 172.21.15.94:0/3394139095 learned_addr learned my addr 172.21.15.94:0/3394139095 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:21.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0effd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d080389d0 0x7f5d08048c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0e7fc640 1 -- 172.21.15.94:0/2775288080 learned_addr learned my addr 172.21.15.94:0/2775288080 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:21.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719c3c3640 1 -- 172.21.15.94:0/3394139095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71940b7d30 con 0x7f7194072390 2024-04-25T16:51:21.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719c3c3640 1 -- 172.21.15.94:0/3394139095 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71940d41b0 con 0x7f71940d3450 2024-04-25T16:51:21.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719c3c3640 1 -- 172.21.15.94:0/3394139095 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f71940b8cd0 con 0x7f71940b7510 2024-04-25T16:51:21.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0ffff640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d080e2910 0x7f5d0818be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f23fff640 1 -- 172.21.15.94:0/475970042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f2413dc20 msgr2=0x7f0f2413e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f23fff640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f2413dc20 0x7f0f2413e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f23fff640 1 -- 172.21.15.94:0/475970042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f2413f220 msgr2=0x7f0f241f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f23fff640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f2413f220 0x7f0f241f5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d080e2d20 con 0x7f5d080514d0 2024-04-25T16:51:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f23fff640 1 -- 172.21.15.94:0/475970042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2413e570 con 0x7f0f2415f810 2024-04-25T16:51:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd7b56b640 1 -- 172.21.15.94:0/3779211959 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741728c0 con 0x7fbd7406c600 2024-04-25T16:51:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd7b56b640 1 -- 172.21.15.94:0/3779211959 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd74172bc0 con 0x7fbd7406ed70 2024-04-25T16:51:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719995e640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71940d3450 0x7f71940cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d080e1e00 con 0x7f5d080e2910 2024-04-25T16:51:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719a960640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7194072390 0x7f71940727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d08052510 con 0x7f5d080389d0 2024-04-25T16:51:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0f7fe640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d080e2910 0x7f5d0818be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f23fff640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2415f810 0x7f0f2413d660 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0f1801f8f0 tx=0x7f0f1801fdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd79b08640 1 -- 172.21.15.94:0/3779211959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd7406ed70 msgr2=0x7fbd74152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd79b08640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd7406ed70 0x7fbd74152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd79b08640 1 -- 172.21.15.94:0/3779211959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd7406c600 msgr2=0x7fbd7414ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd79b08640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd7406c600 0x7fbd7414ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd79b08640 1 -- 172.21.15.94:0/3779211959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd74152dc0 con 0x7fbd74072f30 2024-04-25T16:51:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc16145e640 1 -- 172.21.15.94:0/3564184554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc15c156140 msgr2=0x7fc15c15e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc16145e640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc15c156140 0x7fc15c15e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc16145e640 1 -- 172.21.15.94:0/3564184554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc15c0b58d0 msgr2=0x7fc15c1553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc16145e640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc15c0b58d0 0x7fc15c1553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc16145e640 1 -- 172.21.15.94:0/3564184554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c15d9f0 con 0x7fc15c06db50 2024-04-25T16:51:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719a15f640 1 -- 172.21.15.94:0/3394139095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71940d3450 msgr2=0x7f71940cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719a15f640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71940d3450 0x7f71940cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719a15f640 1 -- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7194072390 msgr2=0x7f71940727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719a15f640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7194072390 0x7f71940727a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719a15f640 1 -- 172.21.15.94:0/3394139095 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71940cc070 con 0x7f71940b7510 2024-04-25T16:51:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719a960640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7194072390 0x7f71940727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd79b08640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd74072f30 0x7fbd74154450 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000c410 tx=0x7fbd7000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fd37affd640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd36406f450 0x7fd364071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fd38259c640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd36406f450 0x7fd364071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fc16145e640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc15c06db50 0x7fc15c0ac870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc1580ae040 tx=0x7fc158018f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fd38259c640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd36406f450 0x7fd364071960 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd384072b90 tx=0x7fd36c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0e7fc640 1 -- 172.21.15.94:0/2775288080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d080389d0 msgr2=0x7f5d08048c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0e7fc640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d080389d0 0x7f5d08048c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0e7fc640 1 -- 172.21.15.94:0/2775288080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d080e2910 msgr2=0x7f5d0818be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0e7fc640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d080e2910 0x7f5d0818be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0e7fc640 1 -- 172.21.15.94:0/2775288080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0818b9a0 con 0x7f5d080514d0 2024-04-25T16:51:21.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5d0e7fc640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d080514d0 0x7f5d0817e0e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400b440 tx=0x7f5d0400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fb458bdf640 1 --2- 172.21.15.94:0/734176640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4541562b0 0x7fb4541566c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4540b04d0 tx=0x7fb44801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498ff8f5f0052b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fd388fad640 1 -- 172.21.15.94:0/504236126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd384158cc0 con 0x7fd3840d1fb0 2024-04-25T16:51:21.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719a15f640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71940b7510 0x7f7194161660 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f719000c850 tx=0x7f719000cd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f20ff9640 1 -- 172.21.15.94:0/475970042 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18004ac0 con 0x7f0f2415f810 2024-04-25T16:51:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f20ff9640 1 -- 172.21.15.94:0/475970042 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f18029070 con 0x7f0f2415f810 2024-04-25T16:51:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f718affd640 1 -- 172.21.15.94:0/3394139095 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190004030 con 0x7f71940b7510 2024-04-25T16:51:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f718affd640 1 -- 172.21.15.94:0/3394139095 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71900041d0 con 0x7f71940b7510 2024-04-25T16:51:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f719c3c3640 1 -- 172.21.15.94:0/3394139095 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7194158430 con 0x7f71940b7510 2024-04-25T16:51:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f718affd640 1 -- 172.21.15.94:0/3394139095 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190007830 con 0x7f71940b7510 2024-04-25T16:51:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f719c3c3640 1 -- 172.21.15.94:0/3394139095 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71940ca410 con 0x7f71940b7510 2024-04-25T16:51:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5cf77fe640 1 -- 172.21.15.94:0/2775288080 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04015020 con 0x7f5d080514d0 2024-04-25T16:51:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5cf77fe640 1 -- 172.21.15.94:0/2775288080 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d04011890 con 0x7f5d080514d0 2024-04-25T16:51:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f5cf77fe640 1 -- 172.21.15.94:0/2775288080 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04011c80 con 0x7f5d080514d0 2024-04-25T16:51:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d08049720 con 0x7f5d080514d0 2024-04-25T16:51:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d08049ce0 con 0x7f5d080514d0 2024-04-25T16:51:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fc151ffb640 1 -- 172.21.15.94:0/3564184554 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1580ba070 con 0x7fc15c06db50 2024-04-25T16:51:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fc1636c2640 1 -- 172.21.15.94:0/3564184554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15c15dcd0 con 0x7fc15c06db50 2024-04-25T16:51:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fc1636c2640 1 -- 172.21.15.94:0/3564184554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15c15e290 con 0x7fc15c06db50 2024-04-25T16:51:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fb452ffd640 1 -- 172.21.15.94:0/734176640 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4480b0070 con 0x7fb4541562b0 2024-04-25T16:51:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f29437640 1 -- 172.21.15.94:0/475970042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2415ce80 con 0x7f0f2415f810 2024-04-25T16:51:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f20ff9640 1 -- 172.21.15.94:0/475970042 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f180060d0 con 0x7f0f2415f810 2024-04-25T16:51:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f0f29437640 1 -- 172.21.15.94:0/475970042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2415d440 con 0x7f0f2415f810 2024-04-25T16:51:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f0f29437640 1 -- 172.21.15.94:0/475970042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f2406a310 con 0x7f0f2415f810 2024-04-25T16:51:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f719c3c3640 1 -- 172.21.15.94:0/3394139095 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7194076830 con 0x7f71940b7510 2024-04-25T16:51:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d080059d0 con 0x7f5d080514d0 2024-04-25T16:51:21.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7fbd69ffb640 1 -- 172.21.15.94:0/3779211959 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd700074e0 con 0x7fbd74072f30 2024-04-25T16:51:21.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fbd69ffb640 1 -- 172.21.15.94:0/3779211959 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd700047f0 con 0x7fbd74072f30 2024-04-25T16:51:21.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fbd69ffb640 1 -- 172.21.15.94:0/3779211959 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70007680 con 0x7fbd74072f30 2024-04-25T16:51:21.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fbd7b56b640 1 -- 172.21.15.94:0/3779211959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd741530a0 con 0x7fbd74072f30 2024-04-25T16:51:21.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fbd7b56b640 1 -- 172.21.15.94:0/3779211959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd74153660 con 0x7fbd74072f30 2024-04-25T16:51:21.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fd378ff9640 1 -- 172.21.15.94:0/504236126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd38406a730 con 0x7fd3840d1fb0 2024-04-25T16:51:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fc1477fe640 1 -- 172.21.15.94:0/3564184554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc13c001040 con 0x7fc15c06db50 2024-04-25T16:51:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fc151ffb640 1 -- 172.21.15.94:0/3564184554 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1580b4420 con 0x7fc15c06db50 2024-04-25T16:51:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fc151ffb640 1 -- 172.21.15.94:0/3564184554 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1580b0040 con 0x7fc15c06db50 2024-04-25T16:51:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f420751b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f8006810 0x7f41f80dc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f420751b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f8033a80 0x7f41f80ea700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f420751b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80eb8f0 0x7f41f80de090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f420751b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41f80ebe80 con 0x7f41f8006810 2024-04-25T16:51:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f420751b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41f80ebd00 con 0x7f41f8033a80 2024-04-25T16:51:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.672+0000 7f420751b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f41f80f4c30 con 0x7f41f80eb8f0 2024-04-25T16:51:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f4205ab8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80eb8f0 0x7f41f80de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f4205ab8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80eb8f0 0x7f41f80de090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51836/0 (socket says 172.21.15.94:51836) 2024-04-25T16:51:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f4205ab8640 1 -- 172.21.15.94:0/2689269174 learned_addr learned my addr 172.21.15.94:0/2689269174 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f4204ab6640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f8033a80 0x7f41f80ea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f42052b7640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f8006810 0x7f41f80dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f4204ab6640 1 -- 172.21.15.94:0/2689269174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80eb8f0 msgr2=0x7f41f80de090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f4204ab6640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80eb8f0 0x7f41f80de090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f4204ab6640 1 -- 172.21.15.94:0/2689269174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f8006810 msgr2=0x7f41f80dc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f4204ab6640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f8006810 0x7f41f80dc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f4204ab6640 1 -- 172.21.15.94:0/2689269174 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f80e0190 con 0x7f41f8033a80 2024-04-25T16:51:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f4205ab8640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80eb8f0 0x7f41f80de090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f4204ab6640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f8033a80 0x7f41f80ea700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00c390 tx=0x7f41fc00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fbd7b56b640 1 -- 172.21.15.94:0/3779211959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbd74076830 con 0x7fbd74072f30 2024-04-25T16:51:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fb452ffd640 1 -- 172.21.15.94:0/734176640 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb448004230 con 0x7fb4541562b0 2024-04-25T16:51:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fb452ffd640 1 -- 172.21.15.94:0/734176640 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448003620 con 0x7fb4541562b0 2024-04-25T16:51:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f718affd640 1 -- 172.21.15.94:0/3394139095 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f71900079d0 con 0x7f71940b7510 2024-04-25T16:51:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fd37affd640 1 -- 172.21.15.94:0/504236126 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd374056f00 con 0x7fd3840d1fb0 2024-04-25T16:51:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fd37affd640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] conn(0x7fd364078b20 0x7fd36407afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fd37affd640 1 -- 172.21.15.94:0/504236126 --> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd374015e50 con 0x7fd364078b20 2024-04-25T16:51:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fd37affd640 1 -- 172.21.15.94:0/504236126 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd3740572c0 con 0x7fd3840d1fb0 2024-04-25T16:51:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7fd382d9d640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] conn(0x7fd364078b20 0x7fd36407afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f718affd640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f717806f430 0x7f7178071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f718affd640 1 -- 172.21.15.94:0/3394139095 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f719008fa40 con 0x7f71940b7510 2024-04-25T16:51:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f719995e640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f717806f430 0x7f7178071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f718affd640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] conn(0x7f7178078b70 0x7f717807b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f718affd640 1 -- 172.21.15.94:0/3394139095 --> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f717807b6d0 con 0x7f7178078b70 2024-04-25T16:51:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.676+0000 7f718affd640 1 -- 172.21.15.94:0/3394139095 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f719008fed0 con 0x7f71940b7510 2024-04-25T16:51:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fb4593e0640 1 -- 172.21.15.94:0/734176640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4541562b0 msgr2=0x7fb4541566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fb4593e0640 1 --2- 172.21.15.94:0/734176640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4541562b0 0x7fb4541566c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4540b04d0 tx=0x7fb44801c480 comp rx=0 tx=0).stop 2024-04-25T16:51:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f719a960640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] conn(0x7f7178078b70 0x7f717807b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fb4593e0640 1 -- 172.21.15.94:0/734176640 shutdown_connections 2024-04-25T16:51:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fb4593e0640 1 --2- 172.21.15.94:0/734176640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4541562b0 0x7fb4541566c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fb4593e0640 1 -- 172.21.15.94:0/734176640 >> 172.21.15.94:0/734176640 conn(0x7fb4540d1020 msgr2=0x7fb4540d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fd382d9d640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] conn(0x7fd364078b20 0x7fd36407afb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fb4593e0640 1 -- 172.21.15.94:0/734176640 shutdown_connections 2024-04-25T16:51:21.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fb4593e0640 1 -- 172.21.15.94:0/734176640 wait complete. 2024-04-25T16:51:21.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f0f20ff9640 1 -- 172.21.15.94:0/475970042 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0f18006270 con 0x7f0f2415f810 2024-04-25T16:51:21.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f719995e640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f717806f430 0x7f7178071940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f718c0af000 tx=0x7f718c0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f719a960640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] conn(0x7f7178078b70 0x7f717807b000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f0f20ff9640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0f1006f400 0x7f0f10071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f0f20ff9640 1 -- 172.21.15.94:0/475970042 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0f180a1150 con 0x7f0f2415f810 2024-04-25T16:51:21.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f0f20ff9640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] conn(0x7f0f10078b50 0x7f0f1007afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f0f20ff9640 1 -- 172.21.15.94:0/475970042 --> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f18021e50 con 0x7f0f10078b50 2024-04-25T16:51:21.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f0f20ff9640 1 -- 172.21.15.94:0/475970042 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0f180a1540 con 0x7f0f2415f810 2024-04-25T16:51:21.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f0f28c36640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] conn(0x7f0f10078b50 0x7f0f1007afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f5cf77fe640 1 -- 172.21.15.94:0/2775288080 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5d04018020 con 0x7f5d080514d0 2024-04-25T16:51:21.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f718affd640 1 -- 172.21.15.94:0/3394139095 <== osd.2 v2:172.21.15.94:6824/1121746691 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f717807b6d0 con 0x7f7178078b70 2024-04-25T16:51:21.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f5cf77fe640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5cfc06f450 0x7f5cfc071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f5cf77fe640 1 -- 172.21.15.94:0/2775288080 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5d0408fc70 con 0x7f5d080514d0 2024-04-25T16:51:21.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fd37affd640 1 -- 172.21.15.94:0/504236126 <== osd.5 v2:172.21.15.138:6823/2199406473 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd374015e50 con 0x7fd364078b20 2024-04-25T16:51:21.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f0f28c36640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] conn(0x7f0f10078b50 0x7f0f1007afe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f0f237fe640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0f1006f400 0x7f0f10071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fbd69ffb640 1 -- 172.21.15.94:0/3779211959 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbd70011070 con 0x7fbd74072f30 2024-04-25T16:51:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f0f20ff9640 1 -- 172.21.15.94:0/475970042 <== osd.1 v2:172.21.15.94:6808/3897403281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f18021e50 con 0x7f0f10078b50 2024-04-25T16:51:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fbd69ffb640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbd5806f400 0x7fbd58071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fb4593e0640 1 Processor -- start 2024-04-25T16:51:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fb4593e0640 1 -- start start 2024-04-25T16:51:21.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fbd69ffb640 1 -- 172.21.15.94:0/3779211959 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbd7008fa20 con 0x7fbd74072f30 2024-04-25T16:51:21.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f0f237fe640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0f1006f400 0x7f0f10071910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0af000 tx=0x7f0f1c004b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fbd69ffb640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] conn(0x7fbd58078ac0 0x7fbd5807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fbd69ffb640 1 -- 172.21.15.94:0/3779211959 --> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd5807b620 con 0x7fbd58078ac0 2024-04-25T16:51:21.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7fbd69ffb640 1 -- 172.21.15.94:0/3779211959 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbd7008feb0 con 0x7fbd74072f30 2024-04-25T16:51:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f41f5ffb640 1 -- 172.21.15.94:0/2689269174 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc017070 con 0x7f41f8033a80 2024-04-25T16:51:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f41f5ffb640 1 -- 172.21.15.94:0/2689269174 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41fc005d10 con 0x7f41f8033a80 2024-04-25T16:51:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f5cf77fe640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] conn(0x7f5cfc078c10 0x7f5cfc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f5cf77fe640 1 -- 172.21.15.94:0/2775288080 --> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d0401ae50 con 0x7f5cfc078c10 2024-04-25T16:51:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f5cf77fe640 1 -- 172.21.15.94:0/2775288080 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5d04090150 con 0x7f5d080514d0 2024-04-25T16:51:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f5d0f7fe640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] conn(0x7f5cfc078c10 0x7f5cfc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.680+0000 7f5d0f7fe640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] conn(0x7f5cfc078c10 0x7f5cfc07b0a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7f5cf77fe640 1 -- 172.21.15.94:0/2775288080 <== osd.3 v2:172.21.15.94:6816/1777163555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d0401ae50 con 0x7f5cfc078c10 2024-04-25T16:51:21.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7f41f5ffb640 1 -- 172.21.15.94:0/2689269174 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc007310 con 0x7f41f8033a80 2024-04-25T16:51:21.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fbd79307640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbd5806f400 0x7fbd58071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fbd78b06640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] conn(0x7fbd58078ac0 0x7fbd5807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fbd79307640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbd5806f400 0x7fbd58071910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbd640017b0 tx=0x7fbd64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fb4593e0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4540b4ae0 0x7fb45420a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fb4593e0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4541562b0 0x7fb454155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fb4593e0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb454160980 0x7fb4542062c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fb4593e0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb454073c80 con 0x7fb4541562b0 2024-04-25T16:51:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fb4593e0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb454160d90 con 0x7fb454160980 2024-04-25T16:51:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fb4593e0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb454073e00 con 0x7fb4540b4ae0 2024-04-25T16:51:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7f420751b640 1 -- 172.21.15.94:0/2689269174 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f80dc7b0 con 0x7f41f8033a80 2024-04-25T16:51:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7f420751b640 1 -- 172.21.15.94:0/2689269174 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f80df0f0 con 0x7f41f8033a80 2024-04-25T16:51:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7f41e37fe640 1 -- 172.21.15.94:0/2689269174 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41dc001040 con 0x7f41f8033a80 2024-04-25T16:51:21.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb458bdf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb454160980 0x7fb4542062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb453fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4540b4ae0 0x7fb45420a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fc151ffb640 1 -- 172.21.15.94:0/3564184554 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc1580bd070 con 0x7fc15c06db50 2024-04-25T16:51:21.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fc151ffb640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc14006f430 0x7fc140071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fc151ffb640 1 -- 172.21.15.94:0/3564184554 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc1580f17b0 con 0x7fc15c06db50 2024-04-25T16:51:21.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fc151ffb640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] conn(0x7fc1400797c0 0x7fc14007bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fc151ffb640 1 -- 172.21.15.94:0/3564184554 --> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc14007c380 con 0x7fc1400797c0 2024-04-25T16:51:21.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fc151ffb640 1 -- 172.21.15.94:0/3564184554 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc1580f19b0 con 0x7fc15c06db50 2024-04-25T16:51:21.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fc161c5f640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] conn(0x7fc1400797c0 0x7fc14007bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fc161c5f640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] conn(0x7fc1400797c0 0x7fc14007bcb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7f5d0effd640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5cfc06f450 0x7f5cfc071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7f5d0effd640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5cfc06f450 0x7f5cfc071960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009ac0 tx=0x7f5d00009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb453fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4540b4ae0 0x7fb45420a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51840/0 (socket says 172.21.15.94:51840) 2024-04-25T16:51:21.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb453fff640 1 -- 172.21.15.94:0/3993592884 learned_addr learned my addr 172.21.15.94:0/3993592884 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:21.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb453fff640 1 -- 172.21.15.94:0/3993592884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb454160980 msgr2=0x7fb4542062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb453fff640 1 --2- 172.21.15.94:0/3993592884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb454160980 0x7fb4542062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb453fff640 1 -- 172.21.15.94:0/3993592884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4541562b0 msgr2=0x7fb454155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb453fff640 1 --2- 172.21.15.94:0/3993592884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4541562b0 0x7fb454155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb453fff640 1 -- 172.21.15.94:0/3993592884 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb454206d80 con 0x7fb4540b4ae0 2024-04-25T16:51:21.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb453fff640 1 --2- 172.21.15.94:0/3993592884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4540b4ae0 0x7fb45420a2e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00cac0 tx=0x7fb44c00cf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb450ff9640 1 -- 172.21.15.94:0/3993592884 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c017070 con 0x7fb4540b4ae0 2024-04-25T16:51:21.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fc151ffb640 1 -- 172.21.15.94:0/3564184554 <== osd.6 v2:172.21.15.138:6802/3744891971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc14007c380 con 0x7fc1400797c0 2024-04-25T16:51:21.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb4593e0640 1 -- 172.21.15.94:0/3993592884 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb454207060 con 0x7fb4540b4ae0 2024-04-25T16:51:21.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb4593e0640 1 -- 172.21.15.94:0/3993592884 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb454209dc0 con 0x7fb4540b4ae0 2024-04-25T16:51:21.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb450ff9640 1 -- 172.21.15.94:0/3993592884 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb44c00ac30 con 0x7fb4540b4ae0 2024-04-25T16:51:21.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.688+0000 7fb43a7fc640 1 -- 172.21.15.94:0/3993592884 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb434001040 con 0x7fb4540b4ae0 2024-04-25T16:51:21.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.684+0000 7fbd78b06640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] conn(0x7fbd58078ac0 0x7fbd5807af50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.692+0000 7fb450ff9640 1 -- 172.21.15.94:0/3993592884 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c012400 con 0x7fb4540b4ae0 2024-04-25T16:51:21.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.692+0000 7fb450ff9640 1 -- 172.21.15.94:0/3993592884 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb44c01a070 con 0x7fb4540b4ae0 2024-04-25T16:51:21.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.692+0000 7fbd69ffb640 1 -- 172.21.15.94:0/3779211959 <== osd.4 v2:172.21.15.138:6816/4237363180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd5807b620 con 0x7fbd58078ac0 2024-04-25T16:51:21.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.692+0000 7fc160c5d640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc14006f430 0x7fc140071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.692+0000 7fc160c5d640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc14006f430 0x7fc140071940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc15c06e420 tx=0x7fc14c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.692+0000 7fb450ff9640 1 --2- 172.21.15.94:0/3993592884 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb44006f430 0x7fb440071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.692+0000 7fb4537fe640 1 --2- 172.21.15.94:0/3993592884 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb44006f430 0x7fb440071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.692+0000 7fb450ff9640 1 -- 172.21.15.94:0/3993592884 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb44c08fb20 con 0x7fb4540b4ae0 2024-04-25T16:51:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.692+0000 7fb450ff9640 1 --2- 172.21.15.94:0/3993592884 >> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] conn(0x7fb440078b70 0x7fb44007b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.692+0000 7fb450ff9640 1 -- 172.21.15.94:0/3993592884 --> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb44c00ac30 con 0x7fb440078b70 2024-04-25T16:51:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.692+0000 7fb450ff9640 1 -- 172.21.15.94:0/3993592884 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb44c08ffb0 con 0x7fb4540b4ae0 2024-04-25T16:51:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7fb4537fe640 1 --2- 172.21.15.94:0/3993592884 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb44006f430 0x7fb440071940 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb4540b0540 tx=0x7fb444006bb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7f41f5ffb640 1 -- 172.21.15.94:0/2689269174 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f41fc01a070 con 0x7f41f8033a80 2024-04-25T16:51:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7fb458bdf640 1 --2- 172.21.15.94:0/3993592884 >> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] conn(0x7fb440078b70 0x7fb44007b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7fb458bdf640 1 --2- 172.21.15.94:0/3993592884 >> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] conn(0x7fb440078b70 0x7fb44007b000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7f41f5ffb640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f41e406f500 0x7f41e4071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7f41f5ffb640 1 -- 172.21.15.94:0/2689269174 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f41fc08ef50 con 0x7f41f8033a80 2024-04-25T16:51:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7f42052b7640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f41e406f500 0x7f41e4071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7f41f5ffb640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] conn(0x7f41e4078cd0 0x7f41e407b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7f41f5ffb640 1 -- 172.21.15.94:0/2689269174 --> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41fc01dec0 con 0x7f41e4078cd0 2024-04-25T16:51:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7f41f5ffb640 1 -- 172.21.15.94:0/2689269174 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f41fc08f340 con 0x7f41f8033a80 2024-04-25T16:51:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7f4205ab8640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] conn(0x7f41e4078cd0 0x7f41e407b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7f42052b7640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f41e406f500 0x7f41e4071a10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f41f80531f0 tx=0x7f41f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7f4205ab8640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] conn(0x7f41e4078cd0 0x7f41e407b160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.696+0000 7fb450ff9640 1 -- 172.21.15.94:0/3993592884 <== osd.0 v2:172.21.15.94:6800/3354844003 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb44c00ac30 con 0x7fb440078b70 2024-04-25T16:51:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f0f29437640 1 -- 172.21.15.94:0/475970042 --> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0f240732d0 con 0x7f0f10078b50 2024-04-25T16:51:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f41f5ffb640 1 -- 172.21.15.94:0/2689269174 <== osd.7 v2:172.21.15.138:6800/3634138036 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41fc01dec0 con 0x7f41e4078cd0 2024-04-25T16:51:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f0f20ff9640 1 -- 172.21.15.94:0/475970042 <== osd.1 v2:172.21.15.94:6808/3897403281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0f240732d0 con 0x7f0f10078b50 2024-04-25T16:51:21.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f0f29437640 1 -- 172.21.15.94:0/475970042 >> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] conn(0x7f0f10078b50 msgr2=0x7f0f1007afe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f0f29437640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] conn(0x7f0f10078b50 0x7f0f1007afe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f0f29437640 1 -- 172.21.15.94:0/475970042 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0f1006f400 msgr2=0x7f0f10071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f0f29437640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0f1006f400 0x7f0f10071910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0af000 tx=0x7f0f1c004b60 comp rx=0 tx=0).stop 2024-04-25T16:51:21.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f0f29437640 1 -- 172.21.15.94:0/475970042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2415f810 msgr2=0x7f0f2413d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f0f29437640 1 --2- 172.21.15.94:0/475970042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2415f810 0x7f0f2413d660 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0f1801f8f0 tx=0x7f0f1801fdb0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f0f28c36640 1 -- 172.21.15.94:0/475970042 reap_dead start 2024-04-25T16:51:21.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f0f29437640 1 -- 172.21.15.94:0/475970042 shutdown_connections 2024-04-25T16:51:21.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f0f29437640 1 -- 172.21.15.94:0/475970042 >> 172.21.15.94:0/475970042 conn(0x7f0f24072ab0 msgr2=0x7f0f24160260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f0f29437640 1 -- 172.21.15.94:0/475970042 shutdown_connections 2024-04-25T16:51:21.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.700+0000 7f0f29437640 1 -- 172.21.15.94:0/475970042 wait complete. 2024-04-25T16:51:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.708+0000 7f719c3c3640 1 -- 172.21.15.94:0/3394139095 --> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71940d2c40 con 0x7f7178078b70 2024-04-25T16:51:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.708+0000 7f718affd640 1 -- 172.21.15.94:0/3394139095 <== osd.2 v2:172.21.15.94:6824/1121746691 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f71940d2c40 con 0x7f7178078b70 2024-04-25T16:51:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.708+0000 7f7188ff9640 1 -- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] conn(0x7f7178078b70 msgr2=0x7f717807b000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.708+0000 7f7188ff9640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] conn(0x7f7178078b70 0x7f717807b000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.708+0000 7fd378ff9640 1 -- 172.21.15.94:0/504236126 --> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd38415ec00 con 0x7fd364078b20 2024-04-25T16:51:21.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.708+0000 7f7188ff9640 1 -- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f717806f430 msgr2=0x7f7178071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.708+0000 7f7188ff9640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f717806f430 0x7f7178071940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f718c0af000 tx=0x7f718c0a7070 comp rx=0 tx=0).stop 2024-04-25T16:51:21.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.708+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 --> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5d080e0360 con 0x7f5cfc078c10 2024-04-25T16:51:21.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.708+0000 7f7188ff9640 1 -- 172.21.15.94:0/3394139095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71940b7510 msgr2=0x7f7194161660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.708+0000 7f7188ff9640 1 --2- 172.21.15.94:0/3394139095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71940b7510 0x7f7194161660 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f719000c850 tx=0x7f719000cd10 comp rx=0 tx=0).stop 2024-04-25T16:51:21.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.708+0000 7f719a960640 1 -- 172.21.15.94:0/3394139095 reap_dead start 2024-04-25T16:51:21.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.708+0000 7f7188ff9640 1 -- 172.21.15.94:0/3394139095 shutdown_connections 2024-04-25T16:51:21.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f7188ff9640 1 -- 172.21.15.94:0/3394139095 >> 172.21.15.94:0/3394139095 conn(0x7f719409e8d0 msgr2=0x7f71940b8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f7188ff9640 1 -- 172.21.15.94:0/3394139095 shutdown_connections 2024-04-25T16:51:21.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f7188ff9640 1 -- 172.21.15.94:0/3394139095 wait complete. 2024-04-25T16:51:21.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f5cf77fe640 1 -- 172.21.15.94:0/2775288080 <== osd.3 v2:172.21.15.94:6816/1777163555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5d080e0360 con 0x7f5cfc078c10 2024-04-25T16:51:21.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 >> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] conn(0x7f5cfc078c10 msgr2=0x7f5cfc07b0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f5d0ffff640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] conn(0x7f5cfc078c10 0x7f5cfc07b0a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7fd37affd640 1 -- 172.21.15.94:0/504236126 <== osd.5 v2:172.21.15.138:6823/2199406473 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd38415ec00 con 0x7fd364078b20 2024-04-25T16:51:21.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5cfc06f450 msgr2=0x7f5cfc071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f5d0ffff640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5cfc06f450 0x7f5cfc071960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009ac0 tx=0x7f5d00009190 comp rx=0 tx=0).stop 2024-04-25T16:51:21.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d080514d0 msgr2=0x7f5d0817e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f5d0ffff640 1 --2- 172.21.15.94:0/2775288080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d080514d0 0x7f5d0817e0e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400b440 tx=0x7f5d0400b900 comp rx=0 tx=0).stop 2024-04-25T16:51:21.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f5d0f7fe640 1 -- 172.21.15.94:0/2775288080 reap_dead start 2024-04-25T16:51:21.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7fd378ff9640 1 -- 172.21.15.94:0/504236126 >> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] conn(0x7fd364078b20 msgr2=0x7fd36407afb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7fd378ff9640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] conn(0x7fd364078b20 0x7fd36407afb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 shutdown_connections 2024-04-25T16:51:21.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 >> 172.21.15.94:0/2775288080 conn(0x7f5d08053990 msgr2=0x7f5d080518e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7fd378ff9640 1 -- 172.21.15.94:0/504236126 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd36406f450 msgr2=0x7fd364071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7fd378ff9640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd36406f450 0x7fd364071960 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd384072b90 tx=0x7fd36c009190 comp rx=0 tx=0).stop 2024-04-25T16:51:21.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 shutdown_connections 2024-04-25T16:51:21.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7f5d0ffff640 1 -- 172.21.15.94:0/2775288080 wait complete. 2024-04-25T16:51:21.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7fd378ff9640 1 -- 172.21.15.94:0/504236126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3840d1fb0 msgr2=0x7fd38415e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7fd378ff9640 1 --2- 172.21.15.94:0/504236126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3840d1fb0 0x7fd38415e1a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd37400c430 tx=0x7fd37400c8f0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7fd382d9d640 1 -- 172.21.15.94:0/504236126 reap_dead start 2024-04-25T16:51:21.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7fd378ff9640 1 -- 172.21.15.94:0/504236126 shutdown_connections 2024-04-25T16:51:21.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7fd378ff9640 1 -- 172.21.15.94:0/504236126 >> 172.21.15.94:0/504236126 conn(0x7fd3840d33d0 msgr2=0x7fd3841639f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7fd378ff9640 1 -- 172.21.15.94:0/504236126 shutdown_connections 2024-04-25T16:51:21.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7fd378ff9640 1 -- 172.21.15.94:0/504236126 wait complete. 2024-04-25T16:51:21.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.712+0000 7fbd7b56b640 1 -- 172.21.15.94:0/3779211959 --> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbd7406e8e0 con 0x7fbd58078ac0 2024-04-25T16:51:21.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.716+0000 7fbd69ffb640 1 -- 172.21.15.94:0/3779211959 <== osd.4 v2:172.21.15.138:6816/4237363180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbd7406e8e0 con 0x7fbd58078ac0 2024-04-25T16:51:21.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.716+0000 7fbd577fe640 1 -- 172.21.15.94:0/3779211959 >> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] conn(0x7fbd58078ac0 msgr2=0x7fbd5807af50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.716+0000 7fbd577fe640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] conn(0x7fbd58078ac0 0x7fbd5807af50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.716+0000 7fbd577fe640 1 -- 172.21.15.94:0/3779211959 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbd5806f400 msgr2=0x7fbd58071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.716+0000 7fbd577fe640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbd5806f400 0x7fbd58071910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbd640017b0 tx=0x7fbd64009040 comp rx=0 tx=0).stop 2024-04-25T16:51:21.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.716+0000 7fbd577fe640 1 -- 172.21.15.94:0/3779211959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd74072f30 msgr2=0x7fbd74154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.716+0000 7fbd577fe640 1 --2- 172.21.15.94:0/3779211959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd74072f30 0x7fbd74154450 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000c410 tx=0x7fbd7000c8d0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.716+0000 7fbd79b08640 1 -- 172.21.15.94:0/3779211959 reap_dead start 2024-04-25T16:51:21.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.716+0000 7fbd577fe640 1 -- 172.21.15.94:0/3779211959 shutdown_connections 2024-04-25T16:51:21.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.716+0000 7fbd577fe640 1 -- 172.21.15.94:0/3779211959 >> 172.21.15.94:0/3779211959 conn(0x7fbd740b47f0 msgr2=0x7fbd74068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.716+0000 7fbd577fe640 1 -- 172.21.15.94:0/3779211959 shutdown_connections 2024-04-25T16:51:21.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.716+0000 7fbd577fe640 1 -- 172.21.15.94:0/3779211959 wait complete. 2024-04-25T16:51:21.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.724+0000 7fb43a7fc640 1 -- 172.21.15.94:0/3993592884 --> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb434002de0 con 0x7fb440078b70 2024-04-25T16:51:21.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.724+0000 7fb450ff9640 1 -- 172.21.15.94:0/3993592884 <== osd.0 v2:172.21.15.94:6800/3354844003 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb434002de0 con 0x7fb440078b70 2024-04-25T16:51:21.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.724+0000 7fc1477fe640 1 -- 172.21.15.94:0/3564184554 --> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc13c002de0 con 0x7fc1400797c0 2024-04-25T16:51:21.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.724+0000 7fb4593e0640 1 -- 172.21.15.94:0/3993592884 >> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] conn(0x7fb440078b70 msgr2=0x7fb44007b000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.724+0000 7fb4593e0640 1 --2- 172.21.15.94:0/3993592884 >> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] conn(0x7fb440078b70 0x7fb44007b000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fb4593e0640 1 -- 172.21.15.94:0/3993592884 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb44006f430 msgr2=0x7fb440071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fb4593e0640 1 --2- 172.21.15.94:0/3993592884 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb44006f430 0x7fb440071940 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb4540b0540 tx=0x7fb444006bb0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fb4593e0640 1 -- 172.21.15.94:0/3993592884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4540b4ae0 msgr2=0x7fb45420a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fb4593e0640 1 --2- 172.21.15.94:0/3993592884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4540b4ae0 0x7fb45420a2e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00cac0 tx=0x7fb44c00cf80 comp rx=0 tx=0).stop 2024-04-25T16:51:21.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fb458bdf640 1 -- 172.21.15.94:0/3993592884 reap_dead start 2024-04-25T16:51:21.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fb4593e0640 1 -- 172.21.15.94:0/3993592884 shutdown_connections 2024-04-25T16:51:21.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fb4593e0640 1 -- 172.21.15.94:0/3993592884 >> 172.21.15.94:0/3993592884 conn(0x7fb4540d1020 msgr2=0x7fb45406cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fb4593e0640 1 -- 172.21.15.94:0/3993592884 shutdown_connections 2024-04-25T16:51:21.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fb4593e0640 1 -- 172.21.15.94:0/3993592884 wait complete. 2024-04-25T16:51:21.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fc151ffb640 1 -- 172.21.15.94:0/3564184554 <== osd.6 v2:172.21.15.138:6802/3744891971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc13c002de0 con 0x7fc1400797c0 2024-04-25T16:51:21.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7f41e37fe640 1 -- 172.21.15.94:0/2689269174 --> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f41dc002de0 con 0x7f41e4078cd0 2024-04-25T16:51:21.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fc1477fe640 1 -- 172.21.15.94:0/3564184554 >> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] conn(0x7fc1400797c0 msgr2=0x7fc14007bcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fc1477fe640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] conn(0x7fc1400797c0 0x7fc14007bcb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fc1477fe640 1 -- 172.21.15.94:0/3564184554 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc14006f430 msgr2=0x7fc140071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fc1477fe640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc14006f430 0x7fc140071940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc15c06e420 tx=0x7fc14c009190 comp rx=0 tx=0).stop 2024-04-25T16:51:21.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fc1477fe640 1 -- 172.21.15.94:0/3564184554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc15c06db50 msgr2=0x7fc15c0ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fc1477fe640 1 --2- 172.21.15.94:0/3564184554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc15c06db50 0x7fc15c0ac870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc1580ae040 tx=0x7fc158018f60 comp rx=0 tx=0).stop 2024-04-25T16:51:21.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fc161c5f640 1 -- 172.21.15.94:0/3564184554 reap_dead start 2024-04-25T16:51:21.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fc1477fe640 1 -- 172.21.15.94:0/3564184554 shutdown_connections 2024-04-25T16:51:21.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7fc1477fe640 1 -- 172.21.15.94:0/3564184554 >> 172.21.15.94:0/3564184554 conn(0x7fc15c06e610 msgr2=0x7fc15c0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7f41f5ffb640 1 -- 172.21.15.94:0/2689269174 <== osd.7 v2:172.21.15.138:6800/3634138036 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f41dc002de0 con 0x7f41e4078cd0 2024-04-25T16:51:21.772 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:51:21.773 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T16:51:21.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7f41e37fe640 1 -- 172.21.15.94:0/2689269174 >> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] conn(0x7f41e4078cd0 msgr2=0x7f41e407b160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.728+0000 7f41e37fe640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] conn(0x7f41e4078cd0 0x7f41e407b160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:21.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.732+0000 7f41e37fe640 1 -- 172.21.15.94:0/2689269174 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f41e406f500 msgr2=0x7f41e4071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.732+0000 7f41e37fe640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f41e406f500 0x7f41e4071a10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f41f80531f0 tx=0x7f41f0008040 comp rx=0 tx=0).stop 2024-04-25T16:51:21.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.732+0000 7f41e37fe640 1 -- 172.21.15.94:0/2689269174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f8033a80 msgr2=0x7f41f80ea700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:21.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.732+0000 7f41e37fe640 1 --2- 172.21.15.94:0/2689269174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f8033a80 0x7f41f80ea700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00c390 tx=0x7f41fc00c850 comp rx=0 tx=0).stop 2024-04-25T16:51:21.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.732+0000 7f4205ab8640 1 -- 172.21.15.94:0/2689269174 reap_dead start 2024-04-25T16:51:21.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.732+0000 7f41e37fe640 1 -- 172.21.15.94:0/2689269174 shutdown_connections 2024-04-25T16:51:21.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.732+0000 7f41e37fe640 1 -- 172.21.15.94:0/2689269174 >> 172.21.15.94:0/2689269174 conn(0x7f41f8054d00 msgr2=0x7f41f80e10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:21.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.732+0000 7f41e37fe640 1 -- 172.21.15.94:0/2689269174 shutdown_connections 2024-04-25T16:51:21.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.732+0000 7f41e37fe640 1 -- 172.21.15.94:0/2689269174 wait complete. 2024-04-25T16:51:21.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.740+0000 7fc1477fe640 1 -- 172.21.15.94:0/3564184554 shutdown_connections 2024-04-25T16:51:21.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:21.740+0000 7fc1477fe640 1 -- 172.21.15.94:0/3564184554 wait complete. 2024-04-25T16:51:21.776 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:51:21.776 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T16:51:21.777 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:51:21.777 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T16:51:21.778 INFO:teuthology.orchestra.run.smithi094.stdout:55834574860 2024-04-25T16:51:21.778 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T16:51:21.779 INFO:teuthology.orchestra.run.smithi094.stdout:55834574860 2024-04-25T16:51:21.779 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T16:51:21.780 INFO:teuthology.orchestra.run.smithi094.stdout:55834574860 2024-04-25T16:51:21.780 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T16:51:21.781 INFO:teuthology.orchestra.run.smithi094.stdout:55834574860 2024-04-25T16:51:21.781 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T16:51:21.782 INFO:teuthology.orchestra.run.smithi094.stdout:55834574858 2024-04-25T16:51:21.783 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T16:51:22.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7f32667ad640 1 Processor -- start 2024-04-25T16:51:22.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7f32667ad640 1 -- start start 2024-04-25T16:51:22.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7f72fdf58640 1 Processor -- start 2024-04-25T16:51:22.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7f72fdf58640 1 -- start start 2024-04-25T16:51:22.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7ff0cbfff640 1 Processor -- start 2024-04-25T16:51:22.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7ff0cbfff640 1 -- start start 2024-04-25T16:51:22.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7f765bfff640 1 Processor -- start 2024-04-25T16:51:22.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7f765bfff640 1 -- start start 2024-04-25T16:51:22.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7f0bfa911640 1 Processor -- start 2024-04-25T16:51:22.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7f32667ad640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32600b49a0 0x7f326015b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7f32667ad640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f326006c8e0 0x7f326006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7f32667ad640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f326015c240 con 0x7f32600b5270 2024-04-25T16:51:22.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7f32667ad640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f326015c520 con 0x7f326006c8e0 2024-04-25T16:51:22.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7f32667ad640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f326015c800 con 0x7f32600b49a0 2024-04-25T16:51:22.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.036+0000 7f0bfa911640 1 -- start start 2024-04-25T16:51:22.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f32657ab640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32600b49a0 0x7f326015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f32657ab640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32600b49a0 0x7f326015b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51848/0 (socket says 172.21.15.94:51848) 2024-04-25T16:51:22.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f32657ab640 1 -- 172.21.15.94:0/172419393 learned_addr learned my addr 172.21.15.94:0/172419393 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f3265fac640 1 --2- 172.21.15.94:0/172419393 >> v2:172.21.15.138:3300/0 conn(0x7f326006c8e0 0x7f326006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f4de0a9c640 1 Processor -- start 2024-04-25T16:51:22.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f4de0a9c640 1 -- start start 2024-04-25T16:51:22.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f64b553f640 1 Processor -- start 2024-04-25T16:51:22.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f64b553f640 1 -- start start 2024-04-25T16:51:22.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0cbfff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0cc0b48e0 0x7ff0cc0cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0cbfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff0cc0b4cf0 0x7ff0cc0cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0cbfff640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0cc15fc80 con 0x7ff0cc06ce90 2024-04-25T16:51:22.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0cbfff640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ff0cc15ff60 con 0x7ff0cc0b4cf0 2024-04-25T16:51:22.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0cbfff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff0cc160240 con 0x7ff0cc0b48e0 2024-04-25T16:51:22.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72fdf58640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f72f80cde00 0x7f72f80b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72fdf58640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f80ce210 0x7f72f80b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72fdf58640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f72f80b2bb0 0x7f72f80b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72fdf58640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f72f815de40 con 0x7f72f80cde00 2024-04-25T16:51:22.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72fdf58640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f72f80d2180 con 0x7f72f80b2bb0 2024-04-25T16:51:22.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72fdf58640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f72f80d2460 con 0x7f72f80ce210 2024-04-25T16:51:22.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f75a14f1640 1 Processor -- start 2024-04-25T16:51:22.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f75a14f1640 1 -- start start 2024-04-25T16:51:22.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0caffd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0cc0b48e0 0x7ff0cc0cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0caffd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0cc0b48e0 0x7ff0cc0cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51856/0 (socket says 172.21.15.94:51856) 2024-04-25T16:51:22.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0caffd640 1 -- 172.21.15.94:0/2125649857 learned_addr learned my addr 172.21.15.94:0/2125649857 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72effff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f80ce210 0x7f72f80b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72effff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f80ce210 0x7f72f80b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51850/0 (socket says 172.21.15.94:51850) 2024-04-25T16:51:22.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72effff640 1 -- 172.21.15.94:0/3297694842 learned_addr learned my addr 172.21.15.94:0/3297694842 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f765bfff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76540e0a40 0x7f7654050bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f765bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7654034f70 0x7f7654008190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f765bfff640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7654051510 con 0x7f7654051100 2024-04-25T16:51:22.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f765bfff640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f76540e0e50 con 0x7f7654034f70 2024-04-25T16:51:22.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f765bfff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f76540e1a80 con 0x7f76540e0a40 2024-04-25T16:51:22.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72f7fff640 1 --2- 172.21.15.94:0/3297694842 >> v2:172.21.15.138:3300/0 conn(0x7f72f80b2bb0 0x7f72f80b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f0bfa911640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0bec033a50 0x7f0bec033e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f0bfa911640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0bec0067e0 0x7f0bec006bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f0bfa911640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bec038320 con 0x7f0bec0eb8c0 2024-04-25T16:51:22.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f0bfa911640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f0bec006070 con 0x7f0bec033a50 2024-04-25T16:51:22.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f0bfa911640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0bec006350 con 0x7f0bec0067e0 2024-04-25T16:51:22.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f765b7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7654034f70 0x7f7654008190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f765b7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7654034f70 0x7f7654008190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42226/0 (socket says 172.21.15.94:42226) 2024-04-25T16:51:22.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f765b7fe640 1 -- 172.21.15.94:0/3505529715 learned_addr learned my addr 172.21.15.94:0/3505529715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f765affd640 1 --2- 172.21.15.94:0/3505529715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76540e0a40 0x7f7654050bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f3257fff640 1 -- 172.21.15.94:0/172419393 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827273063 0 0) 0x7f326015c240 con 0x7f32600b5270 2024-04-25T16:51:22.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f3257fff640 1 -- 172.21.15.94:0/172419393 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f324c003530 con 0x7f32600b5270 2024-04-25T16:51:22.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f32657ab640 1 -- 172.21.15.94:0/172419393 >> v2:172.21.15.138:3300/0 conn(0x7f326006c8e0 msgr2=0x7f326006ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:22.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f32657ab640 1 --2- 172.21.15.94:0/172419393 >> v2:172.21.15.138:3300/0 conn(0x7f326006c8e0 0x7f326006ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0ca7fc640 1 --2- 172.21.15.94:0/2125649857 >> v2:172.21.15.94:3300/0 conn(0x7ff0cc0b4cf0 0x7ff0cc0cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72f77fe640 1 --2- 172.21.15.94:0/3297694842 >> v2:172.21.15.94:3300/0 conn(0x7f72f80cde00 0x7f72f80b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72f7fff640 1 -- 172.21.15.94:0/3297694842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f80ce210 msgr2=0x7f72f80b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:22.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72f7fff640 1 --2- 172.21.15.94:0/3297694842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f80ce210 0x7f72f80b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72f7fff640 1 -- 172.21.15.94:0/3297694842 >> v2:172.21.15.94:3300/0 conn(0x7f72f80cde00 msgr2=0x7f72f80b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:22.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72f7fff640 1 --2- 172.21.15.94:0/3297694842 >> v2:172.21.15.94:3300/0 conn(0x7f72f80cde00 0x7f72f80b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f72f7fff640 1 -- 172.21.15.94:0/3297694842 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f80d2740 con 0x7f72f80b2bb0 2024-04-25T16:51:22.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0c9ffb640 1 -- 172.21.15.94:0/2125649857 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2410406407 0 0) 0x7ff0cc15fc80 con 0x7ff0cc06ce90 2024-04-25T16:51:22.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0c9ffb640 1 -- 172.21.15.94:0/2125649857 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0b4003530 con 0x7ff0cc06ce90 2024-04-25T16:51:22.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0caffd640 1 -- 172.21.15.94:0/2125649857 >> v2:172.21.15.94:3300/0 conn(0x7ff0cc0b4cf0 msgr2=0x7ff0cc0cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:22.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0caffd640 1 --2- 172.21.15.94:0/2125649857 >> v2:172.21.15.94:3300/0 conn(0x7ff0cc0b4cf0 0x7ff0cc0cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0caffd640 1 -- 172.21.15.94:0/2125649857 >> v1:172.21.15.94:6789/0 conn(0x7ff0cc06ce90 legacy=0x7ff0cc06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7ff0caffd640 1 -- 172.21.15.94:0/2125649857 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc160520 con 0x7ff0cc0b48e0 2024-04-25T16:51:22.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f0bf990f640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0bec0eb8c0 0x7f0bec0e7be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:39728/0 (socket says 172.21.15.94:39728) 2024-04-25T16:51:22.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f0bf990f640 1 -- 172.21.15.94:0/202060371 learned_addr learned my addr 172.21.15.94:0/202060371 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:22.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f0bf910e640 1 --2- 172.21.15.94:0/202060371 >> v2:172.21.15.94:3300/0 conn(0x7f0bec033a50 0x7f0bec033e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f32657ab640 1 -- 172.21.15.94:0/172419393 >> v1:172.21.15.138:6789/0 conn(0x7f32600b5270 legacy=0x7f32600b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f32657ab640 1 -- 172.21.15.94:0/172419393 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326015cae0 con 0x7f32600b49a0 2024-04-25T16:51:22.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f75a14f1640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f759c0cdab0 0x7f759c15fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f75a14f1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f759c072e80 0x7f759c06d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f75a14f1640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f759c06db50 0x7f759c0cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f75a14f1640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f759c0b6670 con 0x7f759c0cdab0 2024-04-25T16:51:22.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f75a14f1640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f759c06df60 con 0x7f759c06db50 2024-04-25T16:51:22.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f75a14f1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f759c0b60f0 con 0x7f759c072e80 2024-04-25T16:51:22.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f64b553f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f64a8008750 0x7f64a8056910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f0bf890d640 1 -- 172.21.15.94:0/202060371 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1685260533 0 0) 0x7f0bf4050cc0 con 0x7f0bec0eb8c0 2024-04-25T16:51:22.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f0bf890d640 1 -- 172.21.15.94:0/202060371 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0be0003530 con 0x7f0bec0eb8c0 2024-04-25T16:51:22.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f64b553f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f64a80393b0 0x7f64a800d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f0bf910e640 1 -- 172.21.15.94:0/202060371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0bec0067e0 msgr2=0x7f0bec006bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:22.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f759affd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f759c0cdab0 0x7f759c15fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f759affd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f759c0cdab0 0x7f759c15fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42250/0 (socket says 172.21.15.94:42250) 2024-04-25T16:51:22.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f759affd640 1 -- 172.21.15.94:0/724972467 learned_addr learned my addr 172.21.15.94:0/724972467 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f759affd640 1 -- 172.21.15.94:0/724972467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f759c072e80 msgr2=0x7f759c06d610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T16:51:22.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f759affd640 1 --2- 172.21.15.94:0/724972467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f759c072e80 0x7f759c06d610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f759affd640 1 -- 172.21.15.94:0/724972467 >> v2:172.21.15.138:3300/0 conn(0x7f759c06db50 msgr2=0x7f759c0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:22.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f759affd640 1 --2- 172.21.15.94:0/724972467 >> v2:172.21.15.138:3300/0 conn(0x7f759c06db50 0x7f759c0cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f759affd640 1 -- 172.21.15.94:0/724972467 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c0726c0 con 0x7f759c0cdab0 2024-04-25T16:51:22.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f0bf890d640 1 -- 172.21.15.94:0/202060371 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1304763 0 0) 0x7f0be0003530 con 0x7f0bec0eb8c0 2024-04-25T16:51:22.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f765affd640 1 -- 172.21.15.94:0/3505529715 >> v2:172.21.15.94:3300/0 conn(0x7f7654034f70 msgr2=0x7f7654008190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:22.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f765affd640 1 --2- 172.21.15.94:0/3505529715 >> v2:172.21.15.94:3300/0 conn(0x7f7654034f70 0x7f7654008190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f765affd640 1 -- 172.21.15.94:0/3505529715 >> v1:172.21.15.138:6789/0 conn(0x7f7654051100 legacy=0x7f7654034860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:22.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f765affd640 1 -- 172.21.15.94:0/3505529715 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7654054d90 con 0x7f76540e0a40 2024-04-25T16:51:22.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f0bfa110640 1 --2- 172.21.15.94:0/202060371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0bec0067e0 0x7f0bec006bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f0bf910e640 1 --2- 172.21.15.94:0/202060371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0bec0067e0 0x7f0bec006bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f0bf910e640 1 -- 172.21.15.94:0/202060371 >> v1:172.21.15.138:6789/0 conn(0x7f0bec0eb8c0 legacy=0x7f0bec0e7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f0bf910e640 1 -- 172.21.15.94:0/202060371 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bec0343a0 con 0x7f0bec033a50 2024-04-25T16:51:22.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f0bfa110640 1 --2- 172.21.15.94:0/202060371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0bec0067e0 0x7f0bec006bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:22.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f4de0a9c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4ddc0cdab0 0x7f4ddc15b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f4de0a9c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ddc06d610 0x7f4ddc06da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f4de0a9c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ddc06df60 con 0x7f4ddc15fea0 2024-04-25T16:51:22.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f4de0a9c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4ddc0b6670 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f4de0a9c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4ddc15bd80 con 0x7f4ddc06d610 2024-04-25T16:51:22.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f4ddad9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ddc06d610 0x7f4ddc06da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f4ddad9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ddc06d610 0x7f4ddc06da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51896/0 (socket says 172.21.15.94:51896) 2024-04-25T16:51:22.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f4ddad9d640 1 -- 172.21.15.94:0/2416768061 learned_addr learned my addr 172.21.15.94:0/2416768061 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f4dd959a640 1 -- 172.21.15.94:0/2416768061 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1490013833 0 0) 0x7f4dd40014a0 con 0x7f4ddc15fea0 2024-04-25T16:51:22.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f4dd959a640 1 -- 172.21.15.94:0/2416768061 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dc4003530 con 0x7f4ddc15fea0 2024-04-25T16:51:22.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.040+0000 7f4ddad9d640 1 -- 172.21.15.94:0/2416768061 >> v2:172.21.15.138:3300/0 conn(0x7f4ddc0cdab0 msgr2=0x7f4ddc15b840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T16:51:22.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f4ddad9d640 1 --2- 172.21.15.94:0/2416768061 >> v2:172.21.15.138:3300/0 conn(0x7f4ddc0cdab0 0x7f4ddc15b840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f4ddad9d640 1 -- 172.21.15.94:0/2416768061 >> v1:172.21.15.94:6789/0 conn(0x7f4ddc15fea0 legacy=0x7f4ddc072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f4ddad9d640 1 -- 172.21.15.94:0/2416768061 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc0726c0 con 0x7f4ddc06d610 2024-04-25T16:51:22.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f64b553f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64a800c6b0 con 0x7f64a803c460 2024-04-25T16:51:22.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f64b553f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f64a803c870 con 0x7f64a80393b0 2024-04-25T16:51:22.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f64b553f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f64a800c990 con 0x7f64a8008750 2024-04-25T16:51:22.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f64af7fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f64a8008750 0x7f64a8056910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f64affff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f64a803c460 0x7f64a8008040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:39734/0 (socket says 172.21.15.94:39734) 2024-04-25T16:51:22.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f64b4d3e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f64a80393b0 0x7f64a800d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f64affff640 1 -- 172.21.15.94:0/597082663 learned_addr learned my addr 172.21.15.94:0/597082663 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:22.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f72f7fff640 1 --2- 172.21.15.94:0/3297694842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f80b2bb0 0x7f72f80b2fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72f80d2390 tx=0x7f72f001c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4b2cf329bba78d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f32657ab640 1 --2- 172.21.15.94:0/172419393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32600b49a0 0x7f326015b920 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f325c008870 tx=0x7f325c01ea20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc7ad76357bcfbc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7ff0caffd640 1 --2- 172.21.15.94:0/2125649857 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0cc0b48e0 0x7ff0cc0cea40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00aa10 tx=0x7ff0bc01d8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10358ba595f0609e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f64b4d3e640 1 -- 172.21.15.94:0/597082663 >> v2:172.21.15.138:3300/0 conn(0x7f64a8008750 msgr2=0x7f64a8056910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:22.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f64b4d3e640 1 --2- 172.21.15.94:0/597082663 >> v2:172.21.15.138:3300/0 conn(0x7f64a8008750 0x7f64a8056910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f64b4d3e640 1 -- 172.21.15.94:0/597082663 >> v1:172.21.15.138:6789/0 conn(0x7f64a803c460 legacy=0x7f64a8008040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f64b4d3e640 1 -- 172.21.15.94:0/597082663 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a800cc70 con 0x7f64a80393b0 2024-04-25T16:51:22.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f64af7fe640 1 --2- 172.21.15.94:0/597082663 >> v2:172.21.15.138:3300/0 conn(0x7f64a8008750 0x7f64a8056910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:22.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f72f6ffd640 1 -- 172.21.15.94:0/3297694842 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f00b2070 con 0x7f72f80b2bb0 2024-04-25T16:51:22.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7ff0c9ffb640 1 -- 172.21.15.94:0/2125649857 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0b3070 con 0x7ff0cc0b48e0 2024-04-25T16:51:22.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f72f6ffd640 1 -- 172.21.15.94:0/3297694842 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f00a58f0 con 0x7f72f80b2bb0 2024-04-25T16:51:22.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f72f6ffd640 1 -- 172.21.15.94:0/3297694842 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f00a5cc0 con 0x7f72f80b2bb0 2024-04-25T16:51:22.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f3257fff640 1 -- 172.21.15.94:0/172419393 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c0ad070 con 0x7f32600b49a0 2024-04-25T16:51:22.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f3257fff640 1 -- 172.21.15.94:0/172419393 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f325c0a7ab0 con 0x7f32600b49a0 2024-04-25T16:51:22.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f3257fff640 1 -- 172.21.15.94:0/172419393 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c0b1400 con 0x7f32600b49a0 2024-04-25T16:51:22.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7ff0c9ffb640 1 -- 172.21.15.94:0/2125649857 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0bc0a6810 con 0x7ff0cc0b48e0 2024-04-25T16:51:22.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7ff0c9ffb640 1 -- 172.21.15.94:0/2125649857 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc00b040 con 0x7ff0cc0b48e0 2024-04-25T16:51:22.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f765affd640 1 --2- 172.21.15.94:0/3505529715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76540e0a40 0x7f7654050bc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f764c0097c0 tx=0x7f764c01c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=212c5367c57eb7be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f4ddad9d640 1 --2- 172.21.15.94:0/2416768061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ddc06d610 0x7f4ddc06da20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009870 tx=0x7f4dcc01c580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be6c84b3134906e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f32667ad640 1 -- 172.21.15.94:0/172419393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32600b49a0 msgr2=0x7f326015b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f32667ad640 1 --2- 172.21.15.94:0/172419393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32600b49a0 0x7f326015b920 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f325c008870 tx=0x7f325c01ea20 comp rx=0 tx=0).stop 2024-04-25T16:51:22.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f32667ad640 1 -- 172.21.15.94:0/172419393 shutdown_connections 2024-04-25T16:51:22.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f32667ad640 1 --2- 172.21.15.94:0/172419393 >> v2:172.21.15.138:3300/0 conn(0x7f326006c8e0 0x7f326006ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f32667ad640 1 --2- 172.21.15.94:0/172419393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32600b49a0 0x7f326015b920 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f32667ad640 1 -- 172.21.15.94:0/172419393 >> 172.21.15.94:0/172419393 conn(0x7f32600d1020 msgr2=0x7f32600d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f7659ffb640 1 -- 172.21.15.94:0/3505529715 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c0b0070 con 0x7f76540e0a40 2024-04-25T16:51:22.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f7659ffb640 1 -- 172.21.15.94:0/3505529715 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f764c0ab600 con 0x7f76540e0a40 2024-04-25T16:51:22.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f7659ffb640 1 -- 172.21.15.94:0/3505529715 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c0ab9d0 con 0x7f76540e0a40 2024-04-25T16:51:22.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f0bf910e640 1 --2- 172.21.15.94:0/202060371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bec033a50 0x7f0bec033e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0be40097c0 tx=0x7f0be401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b0b1ced813c29f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7ff0cbfff640 1 -- 172.21.15.94:0/2125649857 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0cc0b48e0 msgr2=0x7ff0cc0cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7ff0cbfff640 1 --2- 172.21.15.94:0/2125649857 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0cc0b48e0 0x7ff0cc0cea40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00aa10 tx=0x7ff0bc01d8e0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f72fdf58640 1 -- 172.21.15.94:0/3297694842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f80b2bb0 msgr2=0x7f72f80b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f72fdf58640 1 --2- 172.21.15.94:0/3297694842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f80b2bb0 0x7f72f80b2fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72f80d2390 tx=0x7f72f001c8b0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f32667ad640 1 -- 172.21.15.94:0/172419393 shutdown_connections 2024-04-25T16:51:22.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f759affd640 1 --2- 172.21.15.94:0/724972467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759c0cdab0 0x7f759c15fea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f759c0b5880 tx=0x7f759001ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7ce378b882137f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7ff0cbfff640 1 -- 172.21.15.94:0/2125649857 shutdown_connections 2024-04-25T16:51:22.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7ff0cbfff640 1 --2- 172.21.15.94:0/2125649857 >> v2:172.21.15.94:3300/0 conn(0x7ff0cc0b4cf0 0x7ff0cc0cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7ff0cbfff640 1 --2- 172.21.15.94:0/2125649857 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0cc0b48e0 0x7ff0cc0cea40 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00aa10 tx=0x7ff0bc01d8e0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7ff0cbfff640 1 -- 172.21.15.94:0/2125649857 >> 172.21.15.94:0/2125649857 conn(0x7ff0cc0b3e00 msgr2=0x7ff0cc0ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f4dd959a640 1 -- 172.21.15.94:0/2416768061 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0a7070 con 0x7f4ddc06d610 2024-04-25T16:51:22.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7ff0cbfff640 1 -- 172.21.15.94:0/2125649857 shutdown_connections 2024-04-25T16:51:22.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7ff0cbfff640 1 -- 172.21.15.94:0/2125649857 wait complete. 2024-04-25T16:51:22.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f72fdf58640 1 -- 172.21.15.94:0/3297694842 shutdown_connections 2024-04-25T16:51:22.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f72fdf58640 1 --2- 172.21.15.94:0/3297694842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f80b2bb0 0x7f72f80b2fc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f72fdf58640 1 --2- 172.21.15.94:0/3297694842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f80ce210 0x7f72f80b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f72fdf58640 1 --2- 172.21.15.94:0/3297694842 >> v2:172.21.15.94:3300/0 conn(0x7f72f80cde00 0x7f72f80b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.044+0000 7f72fdf58640 1 -- 172.21.15.94:0/3297694842 >> 172.21.15.94:0/3297694842 conn(0x7f72f80d0100 msgr2=0x7f72f80d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f72fdf58640 1 -- 172.21.15.94:0/3297694842 shutdown_connections 2024-04-25T16:51:22.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f0bf890d640 1 -- 172.21.15.94:0/202060371 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be40aa5f0 con 0x7f0bec033a50 2024-04-25T16:51:22.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f0bf890d640 1 -- 172.21.15.94:0/202060371 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be40af070 con 0x7f0bec033a50 2024-04-25T16:51:22.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f0bf890d640 1 -- 172.21.15.94:0/202060371 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be40aa9b0 con 0x7f0bec033a50 2024-04-25T16:51:22.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f7599ffb640 1 -- 172.21.15.94:0/724972467 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75900ad070 con 0x7f759c0cdab0 2024-04-25T16:51:22.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f765bfff640 1 -- 172.21.15.94:0/3505529715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76540e0a40 msgr2=0x7f7654050bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f765bfff640 1 --2- 172.21.15.94:0/3505529715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76540e0a40 0x7f7654050bc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f764c0097c0 tx=0x7f764c01c270 comp rx=0 tx=0).stop 2024-04-25T16:51:22.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f72fdf58640 1 -- 172.21.15.94:0/3297694842 wait complete. 2024-04-25T16:51:22.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f7599ffb640 1 -- 172.21.15.94:0/724972467 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75900a78e0 con 0x7f759c0cdab0 2024-04-25T16:51:22.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f765bfff640 1 -- 172.21.15.94:0/3505529715 shutdown_connections 2024-04-25T16:51:22.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f765bfff640 1 --2- 172.21.15.94:0/3505529715 >> v2:172.21.15.94:3300/0 conn(0x7f7654034f70 0x7f7654008190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f765bfff640 1 --2- 172.21.15.94:0/3505529715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76540e0a40 0x7f7654050bc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f765bfff640 1 -- 172.21.15.94:0/3505529715 >> 172.21.15.94:0/3505529715 conn(0x7f7654055e10 msgr2=0x7f7654056220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f4dd959a640 1 -- 172.21.15.94:0/2416768061 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dcc0b0070 con 0x7f4ddc06d610 2024-04-25T16:51:22.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f4dd959a640 1 -- 172.21.15.94:0/2416768061 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0ab870 con 0x7f4ddc06d610 2024-04-25T16:51:22.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f0bfa911640 1 -- 172.21.15.94:0/202060371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bec033a50 msgr2=0x7f0bec033e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f0bfa911640 1 --2- 172.21.15.94:0/202060371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bec033a50 0x7f0bec033e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0be40097c0 tx=0x7f0be401c490 comp rx=0 tx=0).stop 2024-04-25T16:51:22.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f75a14f1640 1 -- 172.21.15.94:0/724972467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759c0cdab0 msgr2=0x7f759c15fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f75a14f1640 1 --2- 172.21.15.94:0/724972467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759c0cdab0 0x7f759c15fea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f759c0b5880 tx=0x7f759001ea70 comp rx=0 tx=0).stop 2024-04-25T16:51:22.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f0bfa911640 1 -- 172.21.15.94:0/202060371 shutdown_connections 2024-04-25T16:51:22.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f0bfa911640 1 --2- 172.21.15.94:0/202060371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0bec0067e0 0x7f0bec006bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f0bfa911640 1 --2- 172.21.15.94:0/202060371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bec033a50 0x7f0bec033e60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f0bfa911640 1 -- 172.21.15.94:0/202060371 >> 172.21.15.94:0/202060371 conn(0x7f0bec054cd0 msgr2=0x7f0bec0550e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f765bfff640 1 -- 172.21.15.94:0/3505529715 shutdown_connections 2024-04-25T16:51:22.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f4de0a9c640 1 -- 172.21.15.94:0/2416768061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ddc06d610 msgr2=0x7f4ddc06da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f4de0a9c640 1 --2- 172.21.15.94:0/2416768061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ddc06d610 0x7f4ddc06da20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009870 tx=0x7f4dcc01c580 comp rx=0 tx=0).stop 2024-04-25T16:51:22.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f4de0a9c640 1 -- 172.21.15.94:0/2416768061 shutdown_connections 2024-04-25T16:51:22.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f4de0a9c640 1 --2- 172.21.15.94:0/2416768061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ddc06d610 0x7f4ddc06da20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f4de0a9c640 1 --2- 172.21.15.94:0/2416768061 >> v2:172.21.15.138:3300/0 conn(0x7f4ddc0cdab0 0x7f4ddc15b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f4de0a9c640 1 -- 172.21.15.94:0/2416768061 >> 172.21.15.94:0/2416768061 conn(0x7f4ddc06e610 msgr2=0x7f4ddc06ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f0bfa911640 1 -- 172.21.15.94:0/202060371 shutdown_connections 2024-04-25T16:51:22.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f0bfa911640 1 -- 172.21.15.94:0/202060371 wait complete. 2024-04-25T16:51:22.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f765bfff640 1 -- 172.21.15.94:0/3505529715 wait complete. 2024-04-25T16:51:22.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f32667ad640 1 -- 172.21.15.94:0/172419393 wait complete. 2024-04-25T16:51:22.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f32667ad640 1 Processor -- start 2024-04-25T16:51:22.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7ff0cbfff640 1 Processor -- start 2024-04-25T16:51:22.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7ff0cbfff640 1 -- start start 2024-04-25T16:51:22.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f75a14f1640 1 -- 172.21.15.94:0/724972467 shutdown_connections 2024-04-25T16:51:22.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f75a14f1640 1 --2- 172.21.15.94:0/724972467 >> v2:172.21.15.138:3300/0 conn(0x7f759c06db50 0x7f759c0cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f75a14f1640 1 --2- 172.21.15.94:0/724972467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f759c072e80 0x7f759c06d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f75a14f1640 1 --2- 172.21.15.94:0/724972467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759c0cdab0 0x7f759c15fea0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f75a14f1640 1 -- 172.21.15.94:0/724972467 >> 172.21.15.94:0/724972467 conn(0x7f759c06e610 msgr2=0x7f759c06ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f75a14f1640 1 -- 172.21.15.94:0/724972467 shutdown_connections 2024-04-25T16:51:22.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f75a14f1640 1 -- 172.21.15.94:0/724972467 wait complete. 2024-04-25T16:51:22.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f75a14f1640 1 Processor -- start 2024-04-25T16:51:22.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f64b4d3e640 1 --2- 172.21.15.94:0/597082663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a80393b0 0x7f64a800d750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f64b00660c0 tx=0x7f64b009b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f107ae429ceb57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f64aeffd640 1 -- 172.21.15.94:0/597082663 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0130070 con 0x7f64a80393b0 2024-04-25T16:51:22.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f64aeffd640 1 -- 172.21.15.94:0/597082663 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b012b600 con 0x7f64a80393b0 2024-04-25T16:51:22.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f64aeffd640 1 -- 172.21.15.94:0/597082663 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b012ba70 con 0x7f64a80393b0 2024-04-25T16:51:22.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f64b553f640 1 -- 172.21.15.94:0/597082663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a80393b0 msgr2=0x7f64a800d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f64b553f640 1 --2- 172.21.15.94:0/597082663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a80393b0 0x7f64a800d750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f64b00660c0 tx=0x7f64b009b700 comp rx=0 tx=0).stop 2024-04-25T16:51:22.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f64b553f640 1 -- 172.21.15.94:0/597082663 shutdown_connections 2024-04-25T16:51:22.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f64b553f640 1 --2- 172.21.15.94:0/597082663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a80393b0 0x7f64a800d750 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f64b553f640 1 --2- 172.21.15.94:0/597082663 >> v2:172.21.15.138:3300/0 conn(0x7f64a8008750 0x7f64a8056910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f64b553f640 1 -- 172.21.15.94:0/597082663 >> 172.21.15.94:0/597082663 conn(0x7f64a80202b0 msgr2=0x7f64a803bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f64b553f640 1 -- 172.21.15.94:0/597082663 shutdown_connections 2024-04-25T16:51:22.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f72fdf58640 1 Processor -- start 2024-04-25T16:51:22.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f72fdf58640 1 -- start start 2024-04-25T16:51:22.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f0bfa911640 1 Processor -- start 2024-04-25T16:51:22.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f4de0a9c640 1 -- 172.21.15.94:0/2416768061 shutdown_connections 2024-04-25T16:51:22.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f72fdf58640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72f80b2bb0 0x7f72f81faec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0cc06ce90 0x7ff0cc1fd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.048+0000 7f32667ad640 1 -- start start 2024-04-25T16:51:22.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f32667ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f326006c8e0 0x7f3260151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cbfff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0cc0b4cf0 0x7ff0cc1fda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0cc2017f0 0x7ff0cc201c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cbfff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc169070 con 0x7ff0cc2017f0 2024-04-25T16:51:22.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f32667ad640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32600b49a0 0x7f32600cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cbfff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc168ef0 con 0x7ff0cc06ce90 2024-04-25T16:51:22.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cbfff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff0cc1691f0 con 0x7ff0cc0b4cf0 2024-04-25T16:51:22.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f32667ad640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32600b5270 0x7f32600cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f64b553f640 1 -- 172.21.15.94:0/597082663 wait complete. 2024-04-25T16:51:22.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f765bfff640 1 Processor -- start 2024-04-25T16:51:22.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f765bfff640 1 -- start start 2024-04-25T16:51:22.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f32667ad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32601687d0 con 0x7f326006c8e0 2024-04-25T16:51:22.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f32667ad640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3260168650 con 0x7f32600b5270 2024-04-25T16:51:22.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f32657ab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f326006c8e0 0x7f3260151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f32667ad640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3260168950 con 0x7f32600b49a0 2024-04-25T16:51:22.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f32657ab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f326006c8e0 0x7f3260151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42282/0 (socket says 172.21.15.94:42282) 2024-04-25T16:51:22.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f32657ab640 1 -- 172.21.15.94:0/477622796 learned_addr learned my addr 172.21.15.94:0/477622796 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0cc2017f0 0x7ff0cc201c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0cc2017f0 0x7ff0cc201c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42298/0 (socket says 172.21.15.94:42298) 2024-04-25T16:51:22.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cb7fe640 1 -- 172.21.15.94:0/3304288794 learned_addr learned my addr 172.21.15.94:0/3304288794 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0ca7fc640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0cc0b4cf0 0x7ff0cc1fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f3264faa640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32600b49a0 0x7f32600cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f3265fac640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32600b5270 0x7f32600cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f75a14f1640 1 -- start start 2024-04-25T16:51:22.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f75a14f1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f759c06db50 0x7f759c1fdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f75a14f1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c072e80 0x7f759c200340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f75a14f1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759c0cdab0 0x7f759c151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f75a14f1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f759c073410 con 0x7f759c0cdab0 2024-04-25T16:51:22.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f75a14f1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f759c073290 con 0x7f759c072e80 2024-04-25T16:51:22.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f75a14f1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f759c0b46a0 con 0x7f759c06db50 2024-04-25T16:51:22.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f72fdf58640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f80cde00 0x7f72f8150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72f80b2bb0 0x7f72f81faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f72fdf58640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f80ce210 0x7f72f81579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f72fdf58640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72f80d1e70 con 0x7f72f80b2bb0 2024-04-25T16:51:22.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f72fdf58640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72f80d1cf0 con 0x7f72f80cde00 2024-04-25T16:51:22.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72f80b2bb0 0x7f72f81faec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42276/0 (socket says 172.21.15.94:42276) 2024-04-25T16:51:22.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f72fdf58640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f72f80c7670 con 0x7f72f80ce210 2024-04-25T16:51:22.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f72f77fe640 1 -- 172.21.15.94:0/2794030709 learned_addr learned my addr 172.21.15.94:0/2794030709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f765bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7654034f70 0x7f765417e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f765bfff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7654051100 0x7f7654181690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f72f7fff640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f80ce210 0x7f72f81579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f72effff640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f80cde00 0x7f72f8150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f64b553f640 1 Processor -- start 2024-04-25T16:51:22.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f0bfa911640 1 -- start start 2024-04-25T16:51:22.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f765bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76540e0a40 0x7f76540eac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f64b553f640 1 -- start start 2024-04-25T16:51:22.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f4de0a9c640 1 -- 172.21.15.94:0/2416768061 wait complete. 2024-04-25T16:51:22.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f765bfff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76540daf90 con 0x7f7654034f70 2024-04-25T16:51:22.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f765bfff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76540dae10 con 0x7f76540e0a40 2024-04-25T16:51:22.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f765bfff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f76540518a0 con 0x7f7654051100 2024-04-25T16:51:22.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f3264faa640 1 -- 172.21.15.94:0/477622796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32600b5270 msgr2=0x7f32600cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f3264faa640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32600b5270 0x7f32600cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f3264faa640 1 -- 172.21.15.94:0/477622796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f326006c8e0 msgr2=0x7f3260151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f3264faa640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f326006c8e0 0x7f3260151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f3264faa640 1 -- 172.21.15.94:0/477622796 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3260159e90 con 0x7f32600b49a0 2024-04-25T16:51:22.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f32657ab640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f326006c8e0 0x7f3260151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:22.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0caffd640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0cc06ce90 0x7ff0cc1fd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cb7fe640 1 -- 172.21.15.94:0/3304288794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0cc0b4cf0 msgr2=0x7ff0cc1fda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cb7fe640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0cc0b4cf0 0x7ff0cc1fda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cb7fe640 1 -- 172.21.15.94:0/3304288794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0cc06ce90 msgr2=0x7ff0cc1fd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cb7fe640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0cc06ce90 0x7ff0cc1fd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0cb7fe640 1 -- 172.21.15.94:0/3304288794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc14f890 con 0x7ff0cc2017f0 2024-04-25T16:51:22.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7ff0caffd640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0cc06ce90 0x7ff0cc1fd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:22.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f765a7fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7654051100 0x7f7654181690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f765a7fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7654051100 0x7f7654181690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51932/0 (socket says 172.21.15.94:51932) 2024-04-25T16:51:22.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f765a7fc640 1 -- 172.21.15.94:0/1073789158 learned_addr learned my addr 172.21.15.94:0/1073789158 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f765b7fe640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76540e0a40 0x7f76540eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f765affd640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7654034f70 0x7f765417e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f759affd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f759c06db50 0x7f759c1fdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f759b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759c0cdab0 0x7f759c151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f759b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759c0cdab0 0x7f759c151370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42312/0 (socket says 172.21.15.94:42312) 2024-04-25T16:51:22.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f759b7fe640 1 -- 172.21.15.94:0/2573554179 learned_addr learned my addr 172.21.15.94:0/2573554179 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.052+0000 7f759a7fc640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c072e80 0x7f759c200340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f759affd640 1 -- 172.21.15.94:0/2573554179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c072e80 msgr2=0x7f759c200340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f759affd640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c072e80 0x7f759c200340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f759affd640 1 -- 172.21.15.94:0/2573554179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759c0cdab0 msgr2=0x7f759c151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f759affd640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759c0cdab0 0x7f759c151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f759affd640 1 -- 172.21.15.94:0/2573554179 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c1fe4d0 con 0x7f759c06db50 2024-04-25T16:51:22.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f759b7fe640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759c0cdab0 0x7f759c151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:22.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7ff0cb7fe640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0cc2017f0 0x7ff0cc201c00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000c3c0 tx=0x7ff0c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f72f77fe640 1 -- 172.21.15.94:0/2794030709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f80ce210 msgr2=0x7f72f81579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f72f77fe640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f80ce210 0x7f72f81579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f72f77fe640 1 -- 172.21.15.94:0/2794030709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f80cde00 msgr2=0x7f72f8150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f72f77fe640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f80cde00 0x7f72f8150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f72f77fe640 1 -- 172.21.15.94:0/2794030709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f80cc090 con 0x7f72f80b2bb0 2024-04-25T16:51:22.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f765affd640 1 -- 172.21.15.94:0/1073789158 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7654051100 msgr2=0x7f7654181690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f765affd640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7654051100 0x7f7654181690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f765affd640 1 -- 172.21.15.94:0/1073789158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76540e0a40 msgr2=0x7f76540eac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f765affd640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76540e0a40 0x7f76540eac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f765affd640 1 -- 172.21.15.94:0/1073789158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76540394c0 con 0x7f7654034f70 2024-04-25T16:51:22.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f3264faa640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32600b49a0 0x7f32600cf690 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f325800c760 tx=0x7f325800cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f759affd640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f759c06db50 0x7f759c1fdf90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f759c06f5e0 tx=0x7f759001d2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bfa911640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0bec0067e0 0x7f0bec04bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bfa911640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bec033a50 0x7f0bec0ea360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bfa911640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bec0eb8c0 0x7f0bec0ea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bfa911640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec0ebe50 con 0x7f0bec033a50 2024-04-25T16:51:22.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bfa911640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec0ebcd0 con 0x7f0bec0eb8c0 2024-04-25T16:51:22.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bfa911640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0bec0f4c00 con 0x7f0bec0067e0 2024-04-25T16:51:22.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7ff0b37fe640 1 -- 172.21.15.94:0/3304288794 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0017070 con 0x7ff0cc2017f0 2024-04-25T16:51:22.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7ff0b37fe640 1 -- 172.21.15.94:0/3304288794 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c0003600 con 0x7ff0cc2017f0 2024-04-25T16:51:22.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7ff0b37fe640 1 -- 172.21.15.94:0/3304288794 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0003a70 con 0x7ff0cc2017f0 2024-04-25T16:51:22.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f72f77fe640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72f80b2bb0 0x7f72f81faec0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f72e800d5f0 tx=0x7f72e800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f4de0a9c640 1 Processor -- start 2024-04-25T16:51:22.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bf990f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0bec0067e0 0x7f0bec04bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bf990f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0bec0067e0 0x7f0bec04bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51940/0 (socket says 172.21.15.94:51940) 2024-04-25T16:51:22.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bf990f640 1 -- 172.21.15.94:0/4244925835 learned_addr learned my addr 172.21.15.94:0/4244925835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f64b553f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a80393b0 0x7f64a80dd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f765affd640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7654034f70 0x7f765417e810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7654055950 tx=0x7f764c01aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f64b553f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a803c460 0x7f64a804df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f64affff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a80393b0 0x7f64a80dd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f64affff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a80393b0 0x7f64a80dd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41078/0 (socket says 172.21.15.94:41078) 2024-04-25T16:51:22.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f64af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a803c460 0x7f64a804df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f64af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a803c460 0x7f64a804df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42342/0 (socket says 172.21.15.94:42342) 2024-04-25T16:51:22.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f72f4ff9640 1 -- 172.21.15.94:0/2794030709 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8004240 con 0x7f72f80b2bb0 2024-04-25T16:51:22.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f72f4ff9640 1 -- 172.21.15.94:0/2794030709 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72e80043e0 con 0x7f72f80b2bb0 2024-04-25T16:51:22.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f72f4ff9640 1 -- 172.21.15.94:0/2794030709 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8003620 con 0x7f72f80b2bb0 2024-04-25T16:51:22.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f72fdf58640 1 -- 172.21.15.94:0/2794030709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f80cc5c0 con 0x7f72f80b2bb0 2024-04-25T16:51:22.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f72fdf58640 1 -- 172.21.15.94:0/2794030709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f80ccb80 con 0x7f72f80b2bb0 2024-04-25T16:51:22.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f64affff640 1 -- 172.21.15.94:0/852830661 learned_addr learned my addr 172.21.15.94:0/852830661 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f64b553f640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a80e0d10 0x7f64a817d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f64b553f640 1 -- 172.21.15.94:0/852830661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64a800dc90 con 0x7f64a803c460 2024-04-25T16:51:22.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f76437fe640 1 -- 172.21.15.94:0/1073789158 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c0b0050 con 0x7f7654034f70 2024-04-25T16:51:22.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bf990f640 1 -- 172.21.15.94:0/4244925835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bec0eb8c0 msgr2=0x7f0bec0ea8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bf990f640 1 --2- 172.21.15.94:0/4244925835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bec0eb8c0 0x7f0bec0ea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bf990f640 1 -- 172.21.15.94:0/4244925835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bec033a50 msgr2=0x7f0bec0ea360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bf990f640 1 --2- 172.21.15.94:0/4244925835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bec033a50 0x7f0bec0ea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bf990f640 1 -- 172.21.15.94:0/4244925835 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bec04c900 con 0x7f0bec0067e0 2024-04-25T16:51:22.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bf990f640 1 --2- 172.21.15.94:0/4244925835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0bec0067e0 0x7f0bec04bc10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0bf407b970 tx=0x7f0bf4069c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bea7fc640 1 -- 172.21.15.94:0/4244925835 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4075ec0 con 0x7f0bec0067e0 2024-04-25T16:51:22.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bea7fc640 1 -- 172.21.15.94:0/4244925835 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf4076060 con 0x7f0bec0067e0 2024-04-25T16:51:22.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bea7fc640 1 -- 172.21.15.94:0/4244925835 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf406c600 con 0x7f0bec0067e0 2024-04-25T16:51:22.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bfa911640 1 -- 172.21.15.94:0/4244925835 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bec04cbe0 con 0x7f0bec0067e0 2024-04-25T16:51:22.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f0bfa911640 1 -- 172.21.15.94:0/4244925835 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bec04cec0 con 0x7f0bec0067e0 2024-04-25T16:51:22.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f3255ffb640 1 -- 172.21.15.94:0/477622796 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258003db0 con 0x7f32600b49a0 2024-04-25T16:51:22.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f32667ad640 1 -- 172.21.15.94:0/477622796 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326015a170 con 0x7f32600b49a0 2024-04-25T16:51:22.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f32667ad640 1 -- 172.21.15.94:0/477622796 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f326015a730 con 0x7f32600b49a0 2024-04-25T16:51:22.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f3255ffb640 1 -- 172.21.15.94:0/477622796 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3258015070 con 0x7f32600b49a0 2024-04-25T16:51:22.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f32667ad640 1 -- 172.21.15.94:0/477622796 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f326006a310 con 0x7f32600b49a0 2024-04-25T16:51:22.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f3255ffb640 1 -- 172.21.15.94:0/477622796 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258005290 con 0x7f32600b49a0 2024-04-25T16:51:22.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7ff0cbfff640 1 -- 172.21.15.94:0/3304288794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0cc14fb70 con 0x7ff0cc2017f0 2024-04-25T16:51:22.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7ff0cbfff640 1 -- 172.21.15.94:0/3304288794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0cc157700 con 0x7ff0cc2017f0 2024-04-25T16:51:22.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f75837fe640 1 -- 172.21.15.94:0/2573554179 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75900c0070 con 0x7f759c06db50 2024-04-25T16:51:22.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f75a14f1640 1 -- 172.21.15.94:0/2573554179 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759c200880 con 0x7f759c06db50 2024-04-25T16:51:22.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f75a14f1640 1 -- 172.21.15.94:0/2573554179 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f759c155de0 con 0x7f759c06db50 2024-04-25T16:51:22.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f75837fe640 1 -- 172.21.15.94:0/2573554179 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75900a76e0 con 0x7f759c06db50 2024-04-25T16:51:22.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f75837fe640 1 -- 172.21.15.94:0/2573554179 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759000b360 con 0x7f759c06db50 2024-04-25T16:51:22.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f75a14f1640 1 -- 172.21.15.94:0/2573554179 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f757c0052a0 con 0x7f759c06db50 2024-04-25T16:51:22.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7ff0cbfff640 1 -- 172.21.15.94:0/3304288794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ac0052a0 con 0x7ff0cc2017f0 2024-04-25T16:51:22.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f64b4d3e640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a80e0d10 0x7f64a817d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f4de0a9c640 1 -- start start 2024-04-25T16:51:22.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f76437fe640 1 -- 172.21.15.94:0/1073789158 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f764c0bc070 con 0x7f7654034f70 2024-04-25T16:51:22.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f76437fe640 1 -- 172.21.15.94:0/1073789158 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c0b5830 con 0x7f7654034f70 2024-04-25T16:51:22.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f765bfff640 1 -- 172.21.15.94:0/1073789158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76540397a0 con 0x7f7654034f70 2024-04-25T16:51:22.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.056+0000 7f765bfff640 1 -- 172.21.15.94:0/1073789158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f765403b530 con 0x7f7654034f70 2024-04-25T16:51:22.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f64b553f640 1 -- 172.21.15.94:0/852830661 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64a80397c0 con 0x7f64a80393b0 2024-04-25T16:51:22.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f0bfa911640 1 -- 172.21.15.94:0/4244925835 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bec005550 con 0x7f0bec0067e0 2024-04-25T16:51:22.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f64b553f640 1 -- 172.21.15.94:0/852830661 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f64a800de10 con 0x7f64a80e0d10 2024-04-25T16:51:22.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f72f4ff9640 1 -- 172.21.15.94:0/2794030709 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f72e801b020 con 0x7f72f80b2bb0 2024-04-25T16:51:22.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f72fdf58640 1 -- 172.21.15.94:0/2794030709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72dc0052a0 con 0x7f72f80b2bb0 2024-04-25T16:51:22.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f765bfff640 1 -- 172.21.15.94:0/1073789158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f763c0052a0 con 0x7f7654034f70 2024-04-25T16:51:22.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f64af7fe640 1 -- 172.21.15.94:0/852830661 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a80e0d10 msgr2=0x7f64a817d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f64af7fe640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a80e0d10 0x7f64a817d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f64af7fe640 1 -- 172.21.15.94:0/852830661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a80393b0 msgr2=0x7f64a80dd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f64af7fe640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a80393b0 0x7f64a80dd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f64af7fe640 1 -- 172.21.15.94:0/852830661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a80e6710 con 0x7f64a803c460 2024-04-25T16:51:22.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f72f4ff9640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f72d806f400 0x7f72d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4de0a9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ddc06d610 0x7f4ddc150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4de0a9c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ddc0cdab0 0x7f4ddc150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4de0a9c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ddc15fea0 0x7f4ddc1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4de0a9c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc0cfcd0 con 0x7f4ddc06d610 2024-04-25T16:51:22.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4de0a9c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc0cfb50 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4de0a9c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4ddc0cfe50 con 0x7f4ddc15fea0 2024-04-25T16:51:22.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4dd9d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ddc0cdab0 0x7f4ddc150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4dd9d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ddc0cdab0 0x7f4ddc150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41080/0 (socket says 172.21.15.94:41080) 2024-04-25T16:51:22.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4dd9d9b640 1 -- 172.21.15.94:0/720065992 learned_addr learned my addr 172.21.15.94:0/720065992 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:22.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f72f4ff9640 1 -- 172.21.15.94:0/2794030709 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f72e808f250 con 0x7f72f80b2bb0 2024-04-25T16:51:22.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4dd9d9b640 1 -- 172.21.15.94:0/720065992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ddc15fea0 msgr2=0x7f4ddc1558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4dd9d9b640 1 --2- 172.21.15.94:0/720065992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ddc15fea0 0x7f4ddc1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4dd9d9b640 1 -- 172.21.15.94:0/720065992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ddc06d610 msgr2=0x7f4ddc150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4dd9d9b640 1 --2- 172.21.15.94:0/720065992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ddc06d610 0x7f4ddc150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4dd9d9b640 1 -- 172.21.15.94:0/720065992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc154b90 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f75837fe640 1 -- 172.21.15.94:0/2573554179 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f75900059a0 con 0x7f759c06db50 2024-04-25T16:51:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f75837fe640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f758806f430 0x7f7588071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f75837fe640 1 -- 172.21.15.94:0/2573554179 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7590129540 con 0x7f759c06db50 2024-04-25T16:51:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f0bea7fc640 1 -- 172.21.15.94:0/4244925835 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0bf406c7a0 con 0x7f0bec0067e0 2024-04-25T16:51:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f759a7fc640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f758806f430 0x7f7588071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f0bea7fc640 1 --2- 172.21.15.94:0/4244925835 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0bd806f400 0x7f0bd8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f0bea7fc640 1 -- 172.21.15.94:0/4244925835 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0bf410b900 con 0x7f0bec0067e0 2024-04-25T16:51:22.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4dd9d9b640 1 --2- 172.21.15.94:0/720065992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ddc0cdab0 0x7f4ddc150bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400c770 tx=0x7f4dd400cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4dcaffd640 1 -- 172.21.15.94:0/720065992 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4016020 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f759a7fc640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f758806f430 0x7f7588071940 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f759c161240 tx=0x7f758c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4de0a9c640 1 -- 172.21.15.94:0/720065992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ddc0c8630 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4de0a9c640 1 -- 172.21.15.94:0/720065992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ddc1550e0 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f64af7fe640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a803c460 0x7f64a804df60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f64a400d5d0 tx=0x7f64a400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4dcaffd640 1 -- 172.21.15.94:0/720065992 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd400aa10 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4dcaffd640 1 -- 172.21.15.94:0/720065992 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd400ae00 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f4de0a9c640 1 -- 172.21.15.94:0/720065992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ddc076830 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f76437fe640 1 -- 172.21.15.94:0/1073789158 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f764c0b59d0 con 0x7f7654034f70 2024-04-25T16:51:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7ff0b37fe640 1 -- 172.21.15.94:0/3304288794 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff0c001a070 con 0x7ff0cc2017f0 2024-04-25T16:51:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f3255ffb640 1 -- 172.21.15.94:0/477622796 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3258005430 con 0x7f32600b49a0 2024-04-25T16:51:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f72effff640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f72d806f400 0x7f72d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.060+0000 7f76437fe640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f764806f450 0x7f7648071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7ff0b37fe640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff0b806f450 0x7ff0b8071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7ff0b37fe640 1 -- 172.21.15.94:0/3304288794 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff0c008ed60 con 0x7ff0cc2017f0 2024-04-25T16:51:22.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7ff0caffd640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff0b806f450 0x7ff0b8071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f3255ffb640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f324406f430 0x7f3244071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f0bf910e640 1 --2- 172.21.15.94:0/4244925835 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0bd806f400 0x7f0bd8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f765a7fc640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f764806f450 0x7f7648071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f72effff640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f72d806f400 0x7f72d8071910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f72f80cbf50 tx=0x7f72e0009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f0bf910e640 1 --2- 172.21.15.94:0/4244925835 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0bd806f400 0x7f0bd8071910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0be40ae000 tx=0x7f0be4004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f765a7fc640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f764806f450 0x7f7648071960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76540e3090 tx=0x7f76500090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7ff0caffd640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff0b806f450 0x7ff0b8071960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0b0000 tx=0x7ff0bc0a8070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f4dcaffd640 1 -- 172.21.15.94:0/720065992 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4dd4019020 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f3255ffb640 1 -- 172.21.15.94:0/477622796 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3258093910 con 0x7f32600b49a0 2024-04-25T16:51:22.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f32657ab640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f324406f430 0x7f3244071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f32657ab640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f324406f430 0x7f3244071940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f325c008870 tx=0x7f325c00a040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f4dcaffd640 1 --2- 172.21.15.94:0/720065992 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4dbc06f430 0x7f4dbc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f4dda59c640 1 --2- 172.21.15.94:0/720065992 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4dbc06f430 0x7f4dbc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f4dcaffd640 1 -- 172.21.15.94:0/720065992 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4dd408fe50 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.064+0000 7f4dda59c640 1 --2- 172.21.15.94:0/720065992 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4dbc06f430 0x7f4dbc071940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009650 tx=0x7f4dd0006bb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.068+0000 7f76437fe640 1 -- 172.21.15.94:0/1073789158 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f764c0f2d90 con 0x7f7654034f70 2024-04-25T16:51:22.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.068+0000 7f64acff9640 1 -- 172.21.15.94:0/852830661 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4018020 con 0x7f64a803c460 2024-04-25T16:51:22.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.068+0000 7f64b553f640 1 -- 172.21.15.94:0/852830661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a80e6150 con 0x7f64a803c460 2024-04-25T16:51:22.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.068+0000 7f64b553f640 1 -- 172.21.15.94:0/852830661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a817d0b0 con 0x7f64a803c460 2024-04-25T16:51:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.068+0000 7ff0b37fe640 1 -- 172.21.15.94:0/3304288794 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff0c0021020 con 0x7ff0cc2017f0 2024-04-25T16:51:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.068+0000 7f64b553f640 1 -- 172.21.15.94:0/852830661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a8005580 con 0x7f64a803c460 2024-04-25T16:51:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.072+0000 7f72f4ff9640 1 -- 172.21.15.94:0/2794030709 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f72e805b770 con 0x7f72f80b2bb0 2024-04-25T16:51:22.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.076+0000 7f75837fe640 1 -- 172.21.15.94:0/2573554179 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f759c1fe4d0 con 0x7f759c06db50 2024-04-25T16:51:22.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.076+0000 7f76437fe640 1 -- 172.21.15.94:0/1073789158 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7654051510 con 0x7f7654034f70 2024-04-25T16:51:22.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.076+0000 7f64acff9640 1 -- 172.21.15.94:0/852830661 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a4014620 con 0x7f64a803c460 2024-04-25T16:51:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.076+0000 7f64acff9640 1 -- 172.21.15.94:0/852830661 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a40149d0 con 0x7f64a803c460 2024-04-25T16:51:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.080+0000 7f64acff9640 1 -- 172.21.15.94:0/852830661 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f64a401b020 con 0x7f64a803c460 2024-04-25T16:51:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.080+0000 7f64acff9640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f649406f400 0x7f6494071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.080+0000 7f64acff9640 1 -- 172.21.15.94:0/852830661 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f64a408fed0 con 0x7f64a803c460 2024-04-25T16:51:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.080+0000 7f64affff640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f649406f400 0x7f6494071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.080+0000 7f64affff640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f649406f400 0x7f6494071910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f649c004990 tx=0x7f649c007ea0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.084+0000 7f4dcaffd640 1 -- 172.21.15.94:0/720065992 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4dd4020020 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.088+0000 7f64acff9640 1 -- 172.21.15.94:0/852830661 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f64a4022020 con 0x7f64a803c460 2024-04-25T16:51:22.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.088+0000 7f3255ffb640 1 -- 172.21.15.94:0/477622796 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3258028020 con 0x7f32600b49a0 2024-04-25T16:51:22.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.104+0000 7f0bea7fc640 1 -- 172.21.15.94:0/4244925835 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0bec038320 con 0x7f0bec0067e0 2024-04-25T16:51:22.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.376+0000 7ff0cbfff640 1 -- 172.21.15.94:0/3304288794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff0ac005860 con 0x7ff0cc2017f0 2024-04-25T16:51:22.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.376+0000 7ff0b37fe640 1 -- 172.21.15.94:0/3304288794 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff0c0007e20 con 0x7ff0cc2017f0 2024-04-25T16:51:22.375 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:51:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.380+0000 7ff0b17fa640 1 -- 172.21.15.94:0/3304288794 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff0b806f450 msgr2=0x7ff0b8071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.380+0000 7ff0b17fa640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff0b806f450 0x7ff0b8071960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0b0000 tx=0x7ff0bc0a8070 comp rx=0 tx=0).stop 2024-04-25T16:51:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.380+0000 7ff0b17fa640 1 -- 172.21.15.94:0/3304288794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0cc2017f0 msgr2=0x7ff0cc201c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.380+0000 7ff0b17fa640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0cc2017f0 0x7ff0cc201c00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000c3c0 tx=0x7ff0c000c880 comp rx=0 tx=0).stop 2024-04-25T16:51:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.380+0000 7ff0b17fa640 1 -- 172.21.15.94:0/3304288794 shutdown_connections 2024-04-25T16:51:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.380+0000 7ff0b17fa640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0cc2017f0 0x7ff0cc201c00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.380+0000 7ff0b17fa640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff0b806f450 0x7ff0b8071960 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.380+0000 7ff0b17fa640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0cc0b4cf0 0x7ff0cc1fda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.380+0000 7ff0b17fa640 1 --2- 172.21.15.94:0/3304288794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0cc06ce90 0x7ff0cc1fd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.380+0000 7ff0b17fa640 1 -- 172.21.15.94:0/3304288794 >> 172.21.15.94:0/3304288794 conn(0x7ff0cc0b3e00 msgr2=0x7ff0cc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.380+0000 7ff0b17fa640 1 -- 172.21.15.94:0/3304288794 shutdown_connections 2024-04-25T16:51:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.380+0000 7ff0b17fa640 1 -- 172.21.15.94:0/3304288794 wait complete. 2024-04-25T16:51:22.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.396+0000 7f72fdf58640 1 -- 172.21.15.94:0/2794030709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f72dc005590 con 0x7f72f80b2bb0 2024-04-25T16:51:22.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.396+0000 7f72f4ff9640 1 -- 172.21.15.94:0/2794030709 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f72e805fa70 con 0x7f72f80b2bb0 2024-04-25T16:51:22.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.396+0000 7f75a14f1640 1 -- 172.21.15.94:0/2573554179 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f757c005860 con 0x7f759c06db50 2024-04-25T16:51:22.396 INFO:teuthology.orchestra.run.smithi094.stdout:55834574857 2024-04-25T16:51:22.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f72edffb640 1 -- 172.21.15.94:0/2794030709 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f72d806f400 msgr2=0x7f72d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f72edffb640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f72d806f400 0x7f72d8071910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f72f80cbf50 tx=0x7f72e0009240 comp rx=0 tx=0).stop 2024-04-25T16:51:22.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f72edffb640 1 -- 172.21.15.94:0/2794030709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72f80b2bb0 msgr2=0x7f72f81faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f72edffb640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72f80b2bb0 0x7f72f81faec0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f72e800d5f0 tx=0x7f72e800dab0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f72edffb640 1 -- 172.21.15.94:0/2794030709 shutdown_connections 2024-04-25T16:51:22.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f72edffb640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f72d806f400 0x7f72d8071910 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f72edffb640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f80ce210 0x7f72f81579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f72edffb640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f80cde00 0x7f72f8150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f72edffb640 1 --2- 172.21.15.94:0/2794030709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72f80b2bb0 0x7f72f81faec0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f72edffb640 1 -- 172.21.15.94:0/2794030709 >> 172.21.15.94:0/2794030709 conn(0x7f72f80d0100 msgr2=0x7f72f80d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f75837fe640 1 -- 172.21.15.94:0/2573554179 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7590018660 con 0x7f759c06db50 2024-04-25T16:51:22.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f72edffb640 1 -- 172.21.15.94:0/2794030709 shutdown_connections 2024-04-25T16:51:22.398 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:51:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f72edffb640 1 -- 172.21.15.94:0/2794030709 wait complete. 2024-04-25T16:51:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f765bfff640 1 -- 172.21.15.94:0/1073789158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f763c005860 con 0x7f7654034f70 2024-04-25T16:51:22.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.400+0000 7f76437fe640 1 -- 172.21.15.94:0/1073789158 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f764c0f6b00 con 0x7f7654034f70 2024-04-25T16:51:22.399 INFO:teuthology.orchestra.run.smithi094.stdout:55834574858 2024-04-25T16:51:22.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f75817fa640 1 -- 172.21.15.94:0/2573554179 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f758806f430 msgr2=0x7f7588071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f75817fa640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f758806f430 0x7f7588071940 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f759c161240 tx=0x7f758c008040 comp rx=0 tx=0).stop 2024-04-25T16:51:22.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f75817fa640 1 -- 172.21.15.94:0/2573554179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f759c06db50 msgr2=0x7f759c1fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f75817fa640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f759c06db50 0x7f759c1fdf90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f759c06f5e0 tx=0x7f759001d2b0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f4de0a9c640 1 -- 172.21.15.94:0/720065992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4ddc1594e0 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f75817fa640 1 -- 172.21.15.94:0/2573554179 shutdown_connections 2024-04-25T16:51:22.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f75817fa640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759c0cdab0 0x7f759c151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f75817fa640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c072e80 0x7f759c200340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f75817fa640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f758806f430 0x7f7588071940 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f75817fa640 1 --2- 172.21.15.94:0/2573554179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f759c06db50 0x7f759c1fdf90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f75817fa640 1 -- 172.21.15.94:0/2573554179 >> 172.21.15.94:0/2573554179 conn(0x7f759c06e610 msgr2=0x7f759c0b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f75817fa640 1 -- 172.21.15.94:0/2573554179 shutdown_connections 2024-04-25T16:51:22.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f75817fa640 1 -- 172.21.15.94:0/2573554179 wait complete. 2024-04-25T16:51:22.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f765bfff640 1 -- 172.21.15.94:0/1073789158 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f764806f450 msgr2=0x7f7648071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f765bfff640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f764806f450 0x7f7648071960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76540e3090 tx=0x7f76500090e0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f765bfff640 1 -- 172.21.15.94:0/1073789158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7654034f70 msgr2=0x7f765417e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f765bfff640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7654034f70 0x7f765417e810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7654055950 tx=0x7f764c01aab0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f4dcaffd640 1 -- 172.21.15.94:0/720065992 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4ddc06df60 con 0x7f4ddc0cdab0 2024-04-25T16:51:22.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f765bfff640 1 -- 172.21.15.94:0/1073789158 shutdown_connections 2024-04-25T16:51:22.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f765bfff640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76540e0a40 0x7f76540eac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f765bfff640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f764806f450 0x7f7648071960 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f765bfff640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7654051100 0x7f7654181690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f765bfff640 1 --2- 172.21.15.94:0/1073789158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7654034f70 0x7f765417e810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.404+0000 7f765bfff640 1 -- 172.21.15.94:0/1073789158 >> 172.21.15.94:0/1073789158 conn(0x7f7654055e10 msgr2=0x7f7654035b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.405 INFO:teuthology.orchestra.run.smithi094.stdout:55834574858 2024-04-25T16:51:22.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.2 2024-04-25T16:51:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f765bfff640 1 -- 172.21.15.94:0/1073789158 shutdown_connections 2024-04-25T16:51:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f765bfff640 1 -- 172.21.15.94:0/1073789158 wait complete. 2024-04-25T16:51:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f4de0a9c640 1 -- 172.21.15.94:0/720065992 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4dbc06f430 msgr2=0x7f4dbc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f4de0a9c640 1 --2- 172.21.15.94:0/720065992 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4dbc06f430 0x7f4dbc071940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009650 tx=0x7f4dd0006bb0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f4de0a9c640 1 -- 172.21.15.94:0/720065992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ddc0cdab0 msgr2=0x7f4ddc150bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f4de0a9c640 1 --2- 172.21.15.94:0/720065992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ddc0cdab0 0x7f4ddc150bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400c770 tx=0x7f4dd400cc30 comp rx=0 tx=0).stop 2024-04-25T16:51:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f4de0a9c640 1 -- 172.21.15.94:0/720065992 shutdown_connections 2024-04-25T16:51:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f4de0a9c640 1 --2- 172.21.15.94:0/720065992 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4dbc06f430 0x7f4dbc071940 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f4de0a9c640 1 --2- 172.21.15.94:0/720065992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ddc15fea0 0x7f4ddc1558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f4de0a9c640 1 --2- 172.21.15.94:0/720065992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ddc0cdab0 0x7f4ddc150bb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f4de0a9c640 1 --2- 172.21.15.94:0/720065992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ddc06d610 0x7f4ddc150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f4de0a9c640 1 -- 172.21.15.94:0/720065992 >> 172.21.15.94:0/720065992 conn(0x7f4ddc06e610 msgr2=0x7f4ddc15ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f4de0a9c640 1 -- 172.21.15.94:0/720065992 shutdown_connections 2024-04-25T16:51:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.408+0000 7f4de0a9c640 1 -- 172.21.15.94:0/720065992 wait complete. 2024-04-25T16:51:22.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.412+0000 7f64b553f640 1 -- 172.21.15.94:0/852830661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f64a80e5ee0 con 0x7f64a803c460 2024-04-25T16:51:22.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.412+0000 7f64acff9640 1 -- 172.21.15.94:0/852830661 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f64a405c430 con 0x7f64a803c460 2024-04-25T16:51:22.411 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:51:22.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.412+0000 7f32667ad640 1 -- 172.21.15.94:0/477622796 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f32600cd500 con 0x7f32600b49a0 2024-04-25T16:51:22.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f3255ffb640 1 -- 172.21.15.94:0/477622796 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f325805fe50 con 0x7f32600b49a0 2024-04-25T16:51:22.412 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:51:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f64b553f640 1 -- 172.21.15.94:0/852830661 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f649406f400 msgr2=0x7f6494071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f64b553f640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f649406f400 0x7f6494071910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f649c004990 tx=0x7f649c007ea0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f64b553f640 1 -- 172.21.15.94:0/852830661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a803c460 msgr2=0x7f64a804df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f64b553f640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a803c460 0x7f64a804df60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f64a400d5d0 tx=0x7f64a400da90 comp rx=0 tx=0).stop 2024-04-25T16:51:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f64b553f640 1 -- 172.21.15.94:0/852830661 shutdown_connections 2024-04-25T16:51:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f64b553f640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f649406f400 0x7f6494071910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f64b553f640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a80e0d10 0x7f64a817d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f64b553f640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a803c460 0x7f64a804df60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f64b553f640 1 --2- 172.21.15.94:0/852830661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a80393b0 0x7f64a80dd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f64b553f640 1 -- 172.21.15.94:0/852830661 >> 172.21.15.94:0/852830661 conn(0x7f64a80202b0 msgr2=0x7f64a80577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f64b553f640 1 -- 172.21.15.94:0/852830661 shutdown_connections 2024-04-25T16:51:22.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f64b553f640 1 -- 172.21.15.94:0/852830661 wait complete. 2024-04-25T16:51:22.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f32667ad640 1 -- 172.21.15.94:0/477622796 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f324406f430 msgr2=0x7f3244071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f32667ad640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f324406f430 0x7f3244071940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f325c008870 tx=0x7f325c00a040 comp rx=0 tx=0).stop 2024-04-25T16:51:22.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f32667ad640 1 -- 172.21.15.94:0/477622796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32600b49a0 msgr2=0x7f32600cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f32667ad640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32600b49a0 0x7f32600cf690 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f325800c760 tx=0x7f325800cc20 comp rx=0 tx=0).stop 2024-04-25T16:51:22.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f32667ad640 1 -- 172.21.15.94:0/477622796 shutdown_connections 2024-04-25T16:51:22.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f32667ad640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32600b5270 0x7f32600cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f32667ad640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f324406f430 0x7f3244071940 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f32667ad640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32600b49a0 0x7f32600cf690 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f32667ad640 1 --2- 172.21.15.94:0/477622796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f326006c8e0 0x7f3260151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.416+0000 7f32667ad640 1 -- 172.21.15.94:0/477622796 >> 172.21.15.94:0/477622796 conn(0x7f32600d1020 msgr2=0x7f3260154400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.420+0000 7f32667ad640 1 -- 172.21.15.94:0/477622796 shutdown_connections 2024-04-25T16:51:22.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.420+0000 7f32667ad640 1 -- 172.21.15.94:0/477622796 wait complete. 2024-04-25T16:51:22.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.428+0000 7f0bfa911640 1 -- 172.21.15.94:0/4244925835 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0bec0e62f0 con 0x7f0bec0067e0 2024-04-25T16:51:22.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.428+0000 7f0bea7fc640 1 -- 172.21.15.94:0/4244925835 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0bf40d8410 con 0x7f0bec0067e0 2024-04-25T16:51:22.426 INFO:teuthology.orchestra.run.smithi094.stdout:55834574858 2024-04-25T16:51:22.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.432+0000 7f0bfa911640 1 -- 172.21.15.94:0/4244925835 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0bd806f400 msgr2=0x7f0bd8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.432+0000 7f0bfa911640 1 --2- 172.21.15.94:0/4244925835 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0bd806f400 0x7f0bd8071910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0be40ae000 tx=0x7f0be4004740 comp rx=0 tx=0).stop 2024-04-25T16:51:22.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.432+0000 7f0bfa911640 1 -- 172.21.15.94:0/4244925835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0bec0067e0 msgr2=0x7f0bec04bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:22.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.432+0000 7f0bfa911640 1 --2- 172.21.15.94:0/4244925835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0bec0067e0 0x7f0bec04bc10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0bf407b970 tx=0x7f0bf4069c20 comp rx=0 tx=0).stop 2024-04-25T16:51:22.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.432+0000 7f0bfa911640 1 -- 172.21.15.94:0/4244925835 shutdown_connections 2024-04-25T16:51:22.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.432+0000 7f0bfa911640 1 --2- 172.21.15.94:0/4244925835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bec0eb8c0 0x7f0bec0ea8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.432+0000 7f0bfa911640 1 --2- 172.21.15.94:0/4244925835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bec033a50 0x7f0bec0ea360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.432+0000 7f0bfa911640 1 --2- 172.21.15.94:0/4244925835 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0bd806f400 0x7f0bd8071910 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.432+0000 7f0bfa911640 1 --2- 172.21.15.94:0/4244925835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0bec0067e0 0x7f0bec04bc10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:22.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.432+0000 7f0bfa911640 1 -- 172.21.15.94:0/4244925835 >> 172.21.15.94:0/4244925835 conn(0x7f0bec054cd0 msgr2=0x7f0bec039bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:22.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.432+0000 7f0bfa911640 1 -- 172.21.15.94:0/4244925835 shutdown_connections 2024-04-25T16:51:22.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:22.432+0000 7f0bfa911640 1 -- 172.21.15.94:0/4244925835 wait complete. 2024-04-25T16:51:22.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2024-04-25T16:51:22.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2024-04-25T16:51:22.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.0 2024-04-25T16:51:22.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-04-25T16:51:22.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.5 2024-04-25T16:51:22.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.7 2024-04-25T16:51:22.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.4 2024-04-25T16:51:23.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T16:51:23.435 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T16:51:23.436 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T16:51:23.439 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T16:51:23.447 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T16:51:23.449 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T16:51:23.450 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T16:51:23.458 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T16:51:23.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.636+0000 7fcb8fa00640 1 Processor -- start 2024-04-25T16:51:23.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.636+0000 7fcb8fa00640 1 -- start start 2024-04-25T16:51:23.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.636+0000 7fcb8fa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb88164340 0x7fcb8815a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.636+0000 7fcb8fa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcb88164750 0x7fcb8815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.636+0000 7fcb8fa00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb88164d20 con 0x7fcb880704b0 2024-04-25T16:51:23.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.636+0000 7fcb8fa00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcb880cda30 con 0x7fcb88164750 2024-04-25T16:51:23.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.636+0000 7fcb8fa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcb88165000 con 0x7fcb88164340 2024-04-25T16:51:23.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.636+0000 7fcb8cf9b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcb88164750 0x7fcb8815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.636+0000 7fcb8cf9b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcb88164750 0x7fcb8815ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41096/0 (socket says 172.21.15.94:41096) 2024-04-25T16:51:23.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.636+0000 7fcb8cf9b640 1 -- 172.21.15.94:0/669555695 learned_addr learned my addr 172.21.15.94:0/669555695 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:23.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.640+0000 7fcb8cf9b640 1 -- 172.21.15.94:0/669555695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb88164340 msgr2=0x7fcb8815a940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:51:23.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.640+0000 7fcb8cf9b640 1 --2- 172.21.15.94:0/669555695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb88164340 0x7fcb8815a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.640+0000 7fcb8cf9b640 1 -- 172.21.15.94:0/669555695 >> v1:172.21.15.138:6789/0 conn(0x7fcb880704b0 legacy=0x7fcb880708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.640+0000 7fcb8cf9b640 1 -- 172.21.15.94:0/669555695 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb880b4270 con 0x7fcb88164750 2024-04-25T16:51:23.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.644+0000 7fcb8cf9b640 1 --2- 172.21.15.94:0/669555695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88164750 0x7fcb8815ae80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009e50 tx=0x7fcb8001c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=270fc1ebdaca3bf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.644+0000 7fcb7ffff640 1 -- 172.21.15.94:0/669555695 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800a57c0 con 0x7fcb88164750 2024-04-25T16:51:23.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.644+0000 7fcb7ffff640 1 -- 172.21.15.94:0/669555695 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb800a5960 con 0x7fcb88164750 2024-04-25T16:51:23.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.644+0000 7fcb7ffff640 1 -- 172.21.15.94:0/669555695 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800a5d70 con 0x7fcb88164750 2024-04-25T16:51:23.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.644+0000 7fcb8fa00640 1 -- 172.21.15.94:0/669555695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88164750 msgr2=0x7fcb8815ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.644+0000 7fcb8fa00640 1 --2- 172.21.15.94:0/669555695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88164750 0x7fcb8815ae80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009e50 tx=0x7fcb8001c9e0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.644+0000 7fcb8fa00640 1 -- 172.21.15.94:0/669555695 shutdown_connections 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.644+0000 7fcb8fa00640 1 --2- 172.21.15.94:0/669555695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88164750 0x7fcb8815ae80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.644+0000 7fcb8fa00640 1 --2- 172.21.15.94:0/669555695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb88164340 0x7fcb8815a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.644+0000 7fcb8fa00640 1 -- 172.21.15.94:0/669555695 >> 172.21.15.94:0/669555695 conn(0x7fcb8806f990 msgr2=0x7fcb8806fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.644+0000 7fcb8fa00640 1 -- 172.21.15.94:0/669555695 shutdown_connections 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.644+0000 7fcb8fa00640 1 -- 172.21.15.94:0/669555695 wait complete. 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.648+0000 7fcb8fa00640 1 Processor -- start 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.648+0000 7fcb8fa00640 1 -- start start 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.648+0000 7fcb8fa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb880704b0 0x7fcb880709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.648+0000 7fcb8fa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88164340 0x7fcb880aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.648+0000 7fcb8fa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb88164750 0x7fcb880aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.648+0000 7fcb8fa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb880b5f90 con 0x7fcb88164750 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.648+0000 7fcb8fa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb880b5e10 con 0x7fcb88164340 2024-04-25T16:51:23.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.648+0000 7fcb8fa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcb88167a10 con 0x7fcb880704b0 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.648+0000 7fcb8df9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb88164750 0x7fcb880aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.648+0000 7fcb8df9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb88164750 0x7fcb880aaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42354/0 (socket says 172.21.15.94:42354) 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.648+0000 7fcb8df9d640 1 -- 172.21.15.94:0/245591537 learned_addr learned my addr 172.21.15.94:0/245591537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb8d79c640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb880704b0 0x7fcb880709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb8cf9b640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88164340 0x7fcb880aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb8df9d640 1 -- 172.21.15.94:0/245591537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb880704b0 msgr2=0x7fcb880709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb8df9d640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb880704b0 0x7fcb880709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb8df9d640 1 -- 172.21.15.94:0/245591537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88164340 msgr2=0x7fcb880aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb8df9d640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88164340 0x7fcb880aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb8df9d640 1 -- 172.21.15.94:0/245591537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb88162750 con 0x7fcb88164750 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb8df9d640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb88164750 0x7fcb880aaa10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400c3c0 tx=0x7fcb8400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb7dffb640 1 -- 172.21.15.94:0/245591537 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb840052d0 con 0x7fcb88164750 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb8fa00640 1 -- 172.21.15.94:0/245591537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb88162a30 con 0x7fcb88164750 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb8fa00640 1 -- 172.21.15.94:0/245591537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb88162ff0 con 0x7fcb88164750 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb7dffb640 1 -- 172.21.15.94:0/245591537 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb84005470 con 0x7fcb88164750 2024-04-25T16:51:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb7dffb640 1 -- 172.21.15.94:0/245591537 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84007240 con 0x7fcb88164750 2024-04-25T16:51:23.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.652+0000 7fcb8fa00640 1 -- 172.21.15.94:0/245591537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb88076830 con 0x7fcb88164750 2024-04-25T16:51:23.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.656+0000 7fcb7dffb640 1 -- 172.21.15.94:0/245591537 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcb840073e0 con 0x7fcb88164750 2024-04-25T16:51:23.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.660+0000 7fcb7dffb640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcb6c06f500 0x7fcb6c071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.660+0000 7fcb7dffb640 1 -- 172.21.15.94:0/245591537 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcb8408e960 con 0x7fcb88164750 2024-04-25T16:51:23.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.664+0000 7fcb8d79c640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcb6c06f500 0x7fcb6c071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.664+0000 7fcb7dffb640 1 -- 172.21.15.94:0/245591537 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcb8405aea0 con 0x7fcb88164750 2024-04-25T16:51:23.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.664+0000 7fcb8d79c640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcb6c06f500 0x7fcb6c071a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcb78006fd0 tx=0x7fcb78009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.668+0000 7fb5b4c09640 1 Processor -- start 2024-04-25T16:51:23.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.668+0000 7fb5b4c09640 1 -- start start 2024-04-25T16:51:23.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.668+0000 7fb5b4c09640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5b00b09c0 0x7fb5b0164280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.668+0000 7fb5b4c09640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb5b01647c0 0x7fb5b006d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.668+0000 7fb5b4c09640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb5b006db50 0x7fb5b006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.668+0000 7fb5b4c09640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fb5b006df60 con 0x7fb5b01647c0 2024-04-25T16:51:23.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.668+0000 7fb5b4c09640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb5b0165230 con 0x7fb5b006db50 2024-04-25T16:51:23.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.668+0000 7fb5b4c09640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b00b45d0 con 0x7fb5b00b09c0 2024-04-25T16:51:23.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.668+0000 7fb5add9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb5b01647c0 0x7fb5b006d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.668+0000 7fb5add9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb5b01647c0 0x7fb5b006d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42360/0 (socket says 172.21.15.94:42360) 2024-04-25T16:51:23.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.668+0000 7fb5add9b640 1 -- 172.21.15.94:0/2578547291 learned_addr learned my addr 172.21.15.94:0/2578547291 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:23.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.668+0000 7fb5aed9d640 1 --2- 172.21.15.94:0/2578547291 >> v2:172.21.15.138:3300/0 conn(0x7fb5b006db50 0x7fb5b006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.672+0000 7fb5aed9d640 1 -- 172.21.15.94:0/2578547291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5b00b09c0 msgr2=0x7fb5b0164280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T16:51:23.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.672+0000 7fb5aed9d640 1 --2- 172.21.15.94:0/2578547291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5b00b09c0 0x7fb5b0164280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.672+0000 7fb5aed9d640 1 -- 172.21.15.94:0/2578547291 >> v2:172.21.15.94:3300/0 conn(0x7fb5b01647c0 msgr2=0x7fb5b006d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:23.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.672+0000 7fb5aed9d640 1 --2- 172.21.15.94:0/2578547291 >> v2:172.21.15.94:3300/0 conn(0x7fb5b01647c0 0x7fb5b006d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.672+0000 7fb5aed9d640 1 -- 172.21.15.94:0/2578547291 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b0164c70 con 0x7fb5b006db50 2024-04-25T16:51:23.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.672+0000 7fb5aed9d640 1 --2- 172.21.15.94:0/2578547291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b006db50 0x7fb5b006eab0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0070840 tx=0x7fb5a801c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eec956c20d9c2126 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5ad59a640 1 -- 172.21.15.94:0/2578547291 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80b2070 con 0x7fb5b006db50 2024-04-25T16:51:23.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5ad59a640 1 -- 172.21.15.94:0/2578547291 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a80a5880 con 0x7fb5b006db50 2024-04-25T16:51:23.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5ad59a640 1 -- 172.21.15.94:0/2578547291 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80a5c50 con 0x7fb5b006db50 2024-04-25T16:51:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2578547291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b006db50 msgr2=0x7fb5b006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5b4c09640 1 --2- 172.21.15.94:0/2578547291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b006db50 0x7fb5b006eab0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0070840 tx=0x7fb5a801c820 comp rx=0 tx=0).stop 2024-04-25T16:51:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2578547291 shutdown_connections 2024-04-25T16:51:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5b4c09640 1 --2- 172.21.15.94:0/2578547291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b006db50 0x7fb5b006eab0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5b4c09640 1 --2- 172.21.15.94:0/2578547291 >> v2:172.21.15.94:3300/0 conn(0x7fb5b01647c0 0x7fb5b006d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5b4c09640 1 --2- 172.21.15.94:0/2578547291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5b00b09c0 0x7fb5b0164280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2578547291 >> 172.21.15.94:0/2578547291 conn(0x7fb5b00d1a30 msgr2=0x7fb5b0072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2578547291 shutdown_connections 2024-04-25T16:51:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2578547291 wait complete. 2024-04-25T16:51:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5b4c09640 1 Processor -- start 2024-04-25T16:51:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.676+0000 7fb5b4c09640 1 -- start start 2024-04-25T16:51:23.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5b4c09640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5b006db50 0x7fb5b015d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5b4c09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b00b09c0 0x7fb5b0153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5b4c09640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b01647c0 0x7fb5b00b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5add9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b00b09c0 0x7fb5b0153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5b4c09640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b00b4f10 con 0x7fb5b00b09c0 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5b4c09640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b00b4d90 con 0x7fb5b01647c0 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5b4c09640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b00b5090 con 0x7fb5b006db50 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5add9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b00b09c0 0x7fb5b0153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42364/0 (socket says 172.21.15.94:42364) 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5add9b640 1 -- 172.21.15.94:0/2887010740 learned_addr learned my addr 172.21.15.94:0/2887010740 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5ae59c640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5b006db50 0x7fb5b015d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5aed9d640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b01647c0 0x7fb5b00b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5add9b640 1 -- 172.21.15.94:0/2887010740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5b006db50 msgr2=0x7fb5b015d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5add9b640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5b006db50 0x7fb5b015d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5add9b640 1 -- 172.21.15.94:0/2887010740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b01647c0 msgr2=0x7fb5b00b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5add9b640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b01647c0 0x7fb5b00b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5add9b640 1 -- 172.21.15.94:0/2887010740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b015b050 con 0x7fb5b00b09c0 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5ae59c640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5b006db50 0x7fb5b015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5aed9d640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b01647c0 0x7fb5b00b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5add9b640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b00b09c0 0x7fb5b0153cf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000e8a0 tx=0x7fb5a000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb59effd640 1 -- 172.21.15.94:0/2887010740 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0017070 con 0x7fb5b00b09c0 2024-04-25T16:51:23.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2887010740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b015d950 con 0x7fb5b00b09c0 2024-04-25T16:51:23.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb59effd640 1 -- 172.21.15.94:0/2887010740 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a000c940 con 0x7fb5b00b09c0 2024-04-25T16:51:23.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2887010740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b0148b30 con 0x7fb5b00b09c0 2024-04-25T16:51:23.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fb59effd640 1 -- 172.21.15.94:0/2887010740 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a000cd10 con 0x7fb5b00b09c0 2024-04-25T16:51:23.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fa2dce1b640 1 Processor -- start 2024-04-25T16:51:23.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7fa2dce1b640 1 -- start start 2024-04-25T16:51:23.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2887010740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b015db20 con 0x7fb5b00b09c0 2024-04-25T16:51:23.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.680+0000 7f29099de640 1 Processor -- start 2024-04-25T16:51:23.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f29099de640 1 -- start start 2024-04-25T16:51:23.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f86c0875640 1 Processor -- start 2024-04-25T16:51:23.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa2dce1b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2d80d46f0 0x7fa2d815fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa2dce1b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d806c9a0 0x7fa2d806cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa2dce1b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2d80d0f90 con 0x7fa2d80d0b80 2024-04-25T16:51:23.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa2dce1b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa2d8163d60 con 0x7fa2d80d46f0 2024-04-25T16:51:23.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa2dce1b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d8164040 con 0x7fa2d806c9a0 2024-04-25T16:51:23.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f86c0875640 1 -- start start 2024-04-25T16:51:23.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fb59effd640 1 -- 172.21.15.94:0/2887010740 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb5a001a070 con 0x7fb5b00b09c0 2024-04-25T16:51:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa2d6ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2d80d46f0 0x7fa2d815fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d806c9a0 0x7fa2d806cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d806c9a0 0x7fa2d806cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51978/0 (socket says 172.21.15.94:51978) 2024-04-25T16:51:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa2d7fff640 1 -- 172.21.15.94:0/658705937 learned_addr learned my addr 172.21.15.94:0/658705937 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:23.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fb59effd640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb59006f430 0x7fb590071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fb59effd640 1 -- 172.21.15.94:0/2887010740 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb5a008f910 con 0x7fb5b00b09c0 2024-04-25T16:51:23.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fb5ae59c640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb59006f430 0x7fb590071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f29099de640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2904072b10 0x7f2904164aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f29099de640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904164fe0 0x7f29040d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f29099de640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2904073770 con 0x7f2904072f20 2024-04-25T16:51:23.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f29099de640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f29040b5cd0 con 0x7f2904072b10 2024-04-25T16:51:23.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f29099de640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2904073490 con 0x7f2904164fe0 2024-04-25T16:51:23.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f29089dc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2904072b10 0x7f2904164aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f86c0875640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f86bc0b3950 0x7f86bc166c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f86c0875640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86bc0d1210 con 0x7f86bc15bc90 2024-04-25T16:51:23.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f86c0875640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86bc0b1820 con 0x7f86bc0b1370 2024-04-25T16:51:23.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f86c0875640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f86bc06e0c0 con 0x7f86bc0b3950 2024-04-25T16:51:23.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f2903fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904164fe0 0x7f29040d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f2903fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904164fe0 0x7f29040d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51986/0 (socket says 172.21.15.94:51986) 2024-04-25T16:51:23.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f2903fff640 1 -- 172.21.15.94:0/2832485284 learned_addr learned my addr 172.21.15.94:0/2832485284 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:23.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f86ba59c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f86bc15bc90 0x7f86bc0b0c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39124/0 (socket says 172.21.15.94:39124) 2024-04-25T16:51:23.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f86ba59c640 1 -- 172.21.15.94:0/1207753704 learned_addr learned my addr 172.21.15.94:0/1207753704 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:23.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fb5ae59c640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb59006f430 0x7fb590071940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0071430 tx=0x7fb5a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa2d7fff640 1 -- 172.21.15.94:0/658705937 >> v2:172.21.15.138:3300/0 conn(0x7fa2d80d46f0 msgr2=0x7fa2d815fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:23.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa2d7fff640 1 --2- 172.21.15.94:0/658705937 >> v2:172.21.15.138:3300/0 conn(0x7fa2d80d46f0 0x7fa2d815fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa2d7fff640 1 -- 172.21.15.94:0/658705937 >> v1:172.21.15.138:6789/0 conn(0x7fa2d80d0b80 legacy=0x7fa2d80d3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa2d7fff640 1 -- 172.21.15.94:0/658705937 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d8164320 con 0x7fa2d806c9a0 2024-04-25T16:51:23.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f86bad9d640 1 --2- 172.21.15.94:0/1207753704 >> v2:172.21.15.94:3300/0 conn(0x7f86bc0b3950 0x7f86bc166c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1132198836 0 0) 0x7f86bc0b1820 con 0x7f86bc0b1370 2024-04-25T16:51:23.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86a4003450 con 0x7f86bc0b1370 2024-04-25T16:51:23.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f74034ce640 1 Processor -- start 2024-04-25T16:51:23.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f74034ce640 1 -- start start 2024-04-25T16:51:23.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f2903fff640 1 -- 172.21.15.94:0/2832485284 >> v2:172.21.15.94:3300/0 conn(0x7f2904072b10 msgr2=0x7f2904164aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:23.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f2903fff640 1 --2- 172.21.15.94:0/2832485284 >> v2:172.21.15.94:3300/0 conn(0x7f2904072b10 0x7f2904164aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f2903fff640 1 -- 172.21.15.94:0/2832485284 >> v1:172.21.15.94:6789/0 conn(0x7f2904072f20 legacy=0x7f290406c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7f2903fff640 1 -- 172.21.15.94:0/2832485284 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29040b4dd0 con 0x7f2904164fe0 2024-04-25T16:51:23.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1701140810 0 0) 0x7f86a4003450 con 0x7f86bc0b1370 2024-04-25T16:51:23.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 >> v2:172.21.15.94:3300/0 conn(0x7f86bc0b3950 msgr2=0x7f86bc166c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:23.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa80b3e5640 1 Processor -- start 2024-04-25T16:51:23.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.684+0000 7fa80b3e5640 1 -- start start 2024-04-25T16:51:23.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b959a640 1 --2- 172.21.15.94:0/1207753704 >> v2:172.21.15.94:3300/0 conn(0x7f86bc0b3950 0x7f86bc166c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 >> v1:172.21.15.94:6789/0 conn(0x7f86bc15bc90 legacy=0x7f86bc0b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc0b1b00 con 0x7f86bc0b1370 2024-04-25T16:51:23.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f86ac001680 con 0x7f86bc0b1370 2024-04-25T16:51:23.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 >> v1:172.21.15.138:6789/0 conn(0x7f86bc0b1370 legacy=0x7f86bc0b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b959a640 1 --2- 172.21.15.94:0/1207753704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a4003ab0 0x7f86a4003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b959a640 1 --2- 172.21.15.94:0/1207753704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86a400ac60 0x7f86a400d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b959a640 1 --2- 172.21.15.94:0/1207753704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86a400d7a0 0x7f86a400fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc06e0c0 con 0x7f86a400d7a0 2024-04-25T16:51:23.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86a40068b0 con 0x7f86a4003ab0 2024-04-25T16:51:23.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f86a4006ab0 con 0x7f86a400ac60 2024-04-25T16:51:23.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa2d7fff640 1 --2- 172.21.15.94:0/658705937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d806c9a0 0x7fa2d806cdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009e50 tx=0x7fa2d001ca70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f29db69dd60d6eac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86ba59c640 1 --2- 172.21.15.94:0/1207753704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a4003ab0 0x7f86a4003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b9d9b640 1 --2- 172.21.15.94:0/1207753704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86a400ac60 0x7f86a400d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86bad9d640 1 --2- 172.21.15.94:0/1207753704 >> v2:172.21.15.94:3300/0 conn(0x7f86bc0b3950 0x7f86bc166c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa2d67fc640 1 -- 172.21.15.94:0/658705937 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00a5850 con 0x7fa2d806c9a0 2024-04-25T16:51:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa2d67fc640 1 -- 172.21.15.94:0/658705937 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d00a59f0 con 0x7fa2d806c9a0 2024-04-25T16:51:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa2d67fc640 1 -- 172.21.15.94:0/658705937 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00a5e00 con 0x7fa2d806c9a0 2024-04-25T16:51:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa80b3e5640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa7fc0e37d0 0x7fa7fc0540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa80b3e5640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7fc054a30 con 0x7fa7fc054620 2024-04-25T16:51:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa80b3e5640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7fc038660 con 0x7fa7fc057460 2024-04-25T16:51:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa80b3e5640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fa7fc057870 con 0x7fa7fc0e37d0 2024-04-25T16:51:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f29089dc640 1 --2- 172.21.15.94:0/2832485284 >> v2:172.21.15.94:3300/0 conn(0x7f2904072b10 0x7f2904164aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f74034ce640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f73fc0731d0 0x7f73fc0735e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa809181640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa7fc057460 0x7fa7fc03c7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:39774/0 (socket says 172.21.15.94:39774) 2024-04-25T16:51:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa809181640 1 -- 172.21.15.94:0/1062235421 learned_addr learned my addr 172.21.15.94:0/1062235421 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f74024cc640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f73fc15b920 0x7f73fc15bd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39132/0 (socket says 172.21.15.94:39132) 2024-04-25T16:51:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f74024cc640 1 -- 172.21.15.94:0/3482626225 learned_addr learned my addr 172.21.15.94:0/3482626225 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa808980640 1 --2- 172.21.15.94:0/1062235421 >> v2:172.21.15.94:3300/0 conn(0x7fa7fc0e37d0 0x7fa7fc0540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f74034ce640 1 --2- 172.21.15.94:0/3482626225 >> v2:172.21.15.94:3300/0 conn(0x7f73fc06cae0 0x7f73fc06cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f7401ccb640 1 --2- 172.21.15.94:0/3482626225 >> v2:172.21.15.138:3300/0 conn(0x7f73fc0731d0 0x7f73fc0735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f7402ccd640 1 --2- 172.21.15.94:0/3482626225 >> v2:172.21.15.94:3300/0 conn(0x7f73fc06cae0 0x7f73fc06cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f74034ce640 1 -- 172.21.15.94:0/3482626225 --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73fc0b57a0 con 0x7f73fc15b920 2024-04-25T16:51:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f74034ce640 1 -- 172.21.15.94:0/3482626225 --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f73fc072d80 con 0x7f73fc06cae0 2024-04-25T16:51:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f74034ce640 1 -- 172.21.15.94:0/3482626225 --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f73fc073b20 con 0x7f73fc0731d0 2024-04-25T16:51:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f7402ccd640 1 -- 172.21.15.94:0/3482626225 >> v2:172.21.15.138:3300/0 conn(0x7f73fc0731d0 msgr2=0x7f73fc0735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f7402ccd640 1 --2- 172.21.15.94:0/3482626225 >> v2:172.21.15.138:3300/0 conn(0x7f73fc0731d0 0x7f73fc0735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f7402ccd640 1 -- 172.21.15.94:0/3482626225 >> v1:172.21.15.94:6789/0 conn(0x7f73fc15b920 legacy=0x7f73fc15bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f7402ccd640 1 -- 172.21.15.94:0/3482626225 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc0b5a10 con 0x7f73fc06cae0 2024-04-25T16:51:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b9d9b640 1 -- 172.21.15.94:0/1207753704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a4003ab0 msgr2=0x7f86a4003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b9d9b640 1 --2- 172.21.15.94:0/1207753704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a4003ab0 0x7f86a4003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b9d9b640 1 -- 172.21.15.94:0/1207753704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86a400d7a0 msgr2=0x7f86a400fc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:51:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b9d9b640 1 --2- 172.21.15.94:0/1207753704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86a400d7a0 0x7f86a400fc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7f86b9d9b640 1 -- 172.21.15.94:0/1207753704 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc0b1b00 con 0x7f86a400ac60 2024-04-25T16:51:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f86bad9d640 1 -- 172.21.15.94:0/1207753704 reap_dead start 2024-04-25T16:51:23.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa2dce1b640 1 -- 172.21.15.94:0/658705937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d806c9a0 msgr2=0x7fa2d806cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa2dce1b640 1 --2- 172.21.15.94:0/658705937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d806c9a0 0x7fa2d806cdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009e50 tx=0x7fa2d001ca70 comp rx=0 tx=0).stop 2024-04-25T16:51:23.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa7fbfff640 1 -- 172.21.15.94:0/1062235421 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 303882478 0 0) 0x7fa7fc038660 con 0x7fa7fc057460 2024-04-25T16:51:23.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa7fbfff640 1 -- 172.21.15.94:0/1062235421 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7f0003530 con 0x7fa7fc057460 2024-04-25T16:51:23.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa808980640 1 -- 172.21.15.94:0/1062235421 >> v1:172.21.15.138:6789/0 conn(0x7fa7fc057460 legacy=0x7fa7fc03c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa808980640 1 -- 172.21.15.94:0/1062235421 >> v1:172.21.15.94:6789/0 conn(0x7fa7fc054620 legacy=0x7fa7fc007330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.688+0000 7fa808980640 1 -- 172.21.15.94:0/1062235421 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc006570 con 0x7fa7fc0e37d0 2024-04-25T16:51:23.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f7401ccb640 1 --2- 172.21.15.94:0/3482626225 >> v2:172.21.15.138:3300/0 conn(0x7f73fc0731d0 0x7f73fc0735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:23.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7fa2dce1b640 1 -- 172.21.15.94:0/658705937 shutdown_connections 2024-04-25T16:51:23.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7fa2dce1b640 1 --2- 172.21.15.94:0/658705937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d806c9a0 0x7fa2d806cdb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7fa2dce1b640 1 --2- 172.21.15.94:0/658705937 >> v2:172.21.15.138:3300/0 conn(0x7fa2d80d46f0 0x7fa2d815fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7fa2dce1b640 1 -- 172.21.15.94:0/658705937 >> 172.21.15.94:0/658705937 conn(0x7fa2d80b6ef0 msgr2=0x7fa2d80b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:23.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7fa2dce1b640 1 -- 172.21.15.94:0/658705937 shutdown_connections 2024-04-25T16:51:23.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7fa808980640 1 --2- 172.21.15.94:0/1062235421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7fc0e37d0 0x7fa7fc0540e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa8000097c0 tx=0x7fa80001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239fc4a50dcfee89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7fa2dce1b640 1 -- 172.21.15.94:0/658705937 wait complete. 2024-04-25T16:51:23.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f86b9d9b640 1 --2- 172.21.15.94:0/1207753704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86a400ac60 0x7f86a400d080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c1e0 tx=0x7f86ac01cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=915dc8a582cb10fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f2903fff640 1 --2- 172.21.15.94:0/2832485284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904164fe0 0x7f29040d04f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009e50 tx=0x7f28fc01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=500eba142d7b5e7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f29037fe640 1 -- 172.21.15.94:0/2832485284 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0a5740 con 0x7f2904164fe0 2024-04-25T16:51:23.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0b2070 con 0x7f86a400ac60 2024-04-25T16:51:23.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f29037fe640 1 -- 172.21.15.94:0/2832485284 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28fc0a58e0 con 0x7f2904164fe0 2024-04-25T16:51:23.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f29037fe640 1 -- 172.21.15.94:0/2832485284 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0a5cb0 con 0x7f2904164fe0 2024-04-25T16:51:23.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86ac0a5ab0 con 0x7f86a400ac60 2024-04-25T16:51:23.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f86b959a640 1 -- 172.21.15.94:0/1207753704 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac007c50 con 0x7f86a400ac60 2024-04-25T16:51:23.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f86c0875640 1 -- 172.21.15.94:0/1207753704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86a400ac60 msgr2=0x7f86a400d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f86c0875640 1 --2- 172.21.15.94:0/1207753704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86a400ac60 0x7f86a400d080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c1e0 tx=0x7f86ac01cca0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f86c0875640 1 -- 172.21.15.94:0/1207753704 shutdown_connections 2024-04-25T16:51:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f86c0875640 1 --2- 172.21.15.94:0/1207753704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86a400ac60 0x7f86a400d080 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f86c0875640 1 -- 172.21.15.94:0/1207753704 >> 172.21.15.94:0/1207753704 conn(0x7f86bc072970 msgr2=0x7f86bc0b6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f29099de640 1 -- 172.21.15.94:0/2832485284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904164fe0 msgr2=0x7f29040d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f29099de640 1 --2- 172.21.15.94:0/2832485284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904164fe0 0x7f29040d04f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009e50 tx=0x7f28fc01c990 comp rx=0 tx=0).stop 2024-04-25T16:51:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f29099de640 1 -- 172.21.15.94:0/2832485284 shutdown_connections 2024-04-25T16:51:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f29099de640 1 --2- 172.21.15.94:0/2832485284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904164fe0 0x7f29040d04f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f29099de640 1 --2- 172.21.15.94:0/2832485284 >> v2:172.21.15.94:3300/0 conn(0x7f2904072b10 0x7f2904164aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f29099de640 1 -- 172.21.15.94:0/2832485284 >> 172.21.15.94:0/2832485284 conn(0x7f29040d1020 msgr2=0x7f29040d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f86c0875640 1 -- 172.21.15.94:0/1207753704 shutdown_connections 2024-04-25T16:51:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f86c0875640 1 -- 172.21.15.94:0/1207753704 wait complete. 2024-04-25T16:51:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f29099de640 1 -- 172.21.15.94:0/2832485284 shutdown_connections 2024-04-25T16:51:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f29099de640 1 -- 172.21.15.94:0/2832485284 wait complete. 2024-04-25T16:51:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7f7402ccd640 1 --2- 172.21.15.94:0/3482626225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73fc06cae0 0x7f73fc06cef0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f73fc069230 tx=0x7f73f401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89a9acaf9ea70d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.692+0000 7fb59effd640 1 -- 172.21.15.94:0/2887010740 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb5a005c3a0 con 0x7fb5b00b09c0 2024-04-25T16:51:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7fa2dce1b640 1 Processor -- start 2024-04-25T16:51:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7fa7fbfff640 1 -- 172.21.15.94:0/1062235421 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8000aa620 con 0x7fa7fc0e37d0 2024-04-25T16:51:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f74014ca640 1 -- 172.21.15.94:0/3482626225 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f40b2070 con 0x7f73fc06cae0 2024-04-25T16:51:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7fa7fbfff640 1 -- 172.21.15.94:0/1062235421 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8000af070 con 0x7fa7fc0e37d0 2024-04-25T16:51:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7fa7fbfff640 1 -- 172.21.15.94:0/1062235421 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8000aa9c0 con 0x7fa7fc0e37d0 2024-04-25T16:51:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86c0875640 1 Processor -- start 2024-04-25T16:51:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7fa80b3e5640 1 -- 172.21.15.94:0/1062235421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7fc0e37d0 msgr2=0x7fa7fc0540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7fa80b3e5640 1 --2- 172.21.15.94:0/1062235421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7fc0e37d0 0x7fa7fc0540e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa8000097c0 tx=0x7fa80001c4a0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f74014ca640 1 -- 172.21.15.94:0/3482626225 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f40a58e0 con 0x7f73fc06cae0 2024-04-25T16:51:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f74034ce640 1 -- 172.21.15.94:0/3482626225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73fc06cae0 msgr2=0x7f73fc06cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f74034ce640 1 --2- 172.21.15.94:0/3482626225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73fc06cae0 0x7f73fc06cef0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f73fc069230 tx=0x7f73f401c950 comp rx=0 tx=0).stop 2024-04-25T16:51:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f29099de640 1 Processor -- start 2024-04-25T16:51:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f29099de640 1 -- start start 2024-04-25T16:51:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86c0875640 1 -- start start 2024-04-25T16:51:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f74034ce640 1 -- 172.21.15.94:0/3482626225 shutdown_connections 2024-04-25T16:51:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f74034ce640 1 --2- 172.21.15.94:0/3482626225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73fc06cae0 0x7f73fc06cef0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f74034ce640 1 --2- 172.21.15.94:0/3482626225 >> v2:172.21.15.138:3300/0 conn(0x7f73fc0731d0 0x7f73fc0735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f74034ce640 1 -- 172.21.15.94:0/3482626225 >> 172.21.15.94:0/3482626225 conn(0x7f73fc0d1020 msgr2=0x7f73fc0d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86c0875640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a400ac60 0x7f86bc094640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f74034ce640 1 -- 172.21.15.94:0/3482626225 shutdown_connections 2024-04-25T16:51:23.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86c0875640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc094b80 0x7f86bc15f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86c0875640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86bc15f8e0 0x7f86bc159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86b9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc094b80 0x7f86bc15f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86c0875640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc0b1de0 con 0x7f86bc094b80 2024-04-25T16:51:23.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86b9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc094b80 0x7f86bc15f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42412/0 (socket says 172.21.15.94:42412) 2024-04-25T16:51:23.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86b9d9b640 1 -- 172.21.15.94:0/2430853029 learned_addr learned my addr 172.21.15.94:0/2430853029 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:23.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86c0875640 1 -- 172.21.15.94:0/2430853029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc06c910 con 0x7f86a400ac60 2024-04-25T16:51:23.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86c0875640 1 -- 172.21.15.94:0/2430853029 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f86bc15c0a0 con 0x7f86bc15f8e0 2024-04-25T16:51:23.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86ba59c640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a400ac60 0x7f86bc094640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86bad9d640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86bc15f8e0 0x7f86bc159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86b9d9b640 1 -- 172.21.15.94:0/2430853029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86bc15f8e0 msgr2=0x7f86bc159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86b9d9b640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86bc15f8e0 0x7f86bc159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86b9d9b640 1 -- 172.21.15.94:0/2430853029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a400ac60 msgr2=0x7f86bc094640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86b9d9b640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a400ac60 0x7f86bc094640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86b9d9b640 1 -- 172.21.15.94:0/2430853029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc15fcf0 con 0x7f86bc094b80 2024-04-25T16:51:23.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86bad9d640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86bc15f8e0 0x7f86bc159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:23.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f86b9d9b640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc094b80 0x7f86bc15f3a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f86ac000c10 tx=0x7f86a4003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa80b3e5640 1 -- 172.21.15.94:0/1062235421 shutdown_connections 2024-04-25T16:51:23.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa80b3e5640 1 --2- 172.21.15.94:0/1062235421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7fc0e37d0 0x7fa7fc0540e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa80b3e5640 1 -- 172.21.15.94:0/1062235421 >> 172.21.15.94:0/1062235421 conn(0x7fa7fc0591a0 msgr2=0x7fa7fc0595b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:23.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f29099de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2904072b10 0x7f290415f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f29099de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2904072f20 0x7f2904159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f29089dc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2904072b10 0x7f290415f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f29089dc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2904072b10 0x7f290415f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41194/0 (socket says 172.21.15.94:41194) 2024-04-25T16:51:23.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa80b3e5640 1 -- 172.21.15.94:0/1062235421 shutdown_connections 2024-04-25T16:51:23.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f2903fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2904072f20 0x7f2904159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f2903fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2904072f20 0x7f2904159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42438/0 (socket says 172.21.15.94:42438) 2024-04-25T16:51:23.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f29089dc640 1 -- 172.21.15.94:0/362928717 learned_addr learned my addr 172.21.15.94:0/362928717 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:23.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f29099de640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904164fe0 0x7f2904206170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7f74034ce640 1 -- 172.21.15.94:0/3482626225 wait complete. 2024-04-25T16:51:23.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f74034ce640 1 Processor -- start 2024-04-25T16:51:23.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f29099de640 1 -- 172.21.15.94:0/362928717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f290415e440 con 0x7f2904072f20 2024-04-25T16:51:23.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f74034ce640 1 -- start start 2024-04-25T16:51:23.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f29099de640 1 -- 172.21.15.94:0/362928717 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f290415e2c0 con 0x7f2904072b10 2024-04-25T16:51:23.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f29099de640 1 -- 172.21.15.94:0/362928717 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f290415e5c0 con 0x7f2904164fe0 2024-04-25T16:51:23.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f29091dd640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904164fe0 0x7f2904206170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f2903fff640 1 -- 172.21.15.94:0/362928717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904164fe0 msgr2=0x7f2904206170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f2903fff640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904164fe0 0x7f2904206170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f2903fff640 1 -- 172.21.15.94:0/362928717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2904072b10 msgr2=0x7f290415f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f2903fff640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2904072b10 0x7f290415f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f2903fff640 1 -- 172.21.15.94:0/362928717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29040cee90 con 0x7f2904072f20 2024-04-25T16:51:23.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f2903fff640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2904072f20 0x7f2904159bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0ab000 tx=0x7f28fc019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa80b3e5640 1 -- 172.21.15.94:0/1062235421 wait complete. 2024-04-25T16:51:23.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa80b3e5640 1 Processor -- start 2024-04-25T16:51:23.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa80b3e5640 1 -- start start 2024-04-25T16:51:23.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7f74034ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73fc06cae0 0x7f73fc15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f74024cc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73fc06cae0 0x7f73fc15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f74024cc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73fc06cae0 0x7f73fc15bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42442/0 (socket says 172.21.15.94:42442) 2024-04-25T16:51:23.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f74024cc640 1 -- 172.21.15.94:0/1104534029 learned_addr learned my addr 172.21.15.94:0/1104534029 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:23.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f74034ce640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73fc0731d0 0x7f73fc1644f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f86aaffd640 1 -- 172.21.15.94:0/2430853029 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0bc070 con 0x7f86bc094b80 2024-04-25T16:51:23.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f74034ce640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73fc15b920 0x7f73fc156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f86c0875640 1 -- 172.21.15.94:0/2430853029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc1572e0 con 0x7f86bc094b80 2024-04-25T16:51:23.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f86c0875640 1 -- 172.21.15.94:0/2430853029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc1578a0 con 0x7f86bc094b80 2024-04-25T16:51:23.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f74034ce640 1 -- 172.21.15.94:0/1104534029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc168a10 con 0x7f73fc06cae0 2024-04-25T16:51:23.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f29017fa640 1 -- 172.21.15.94:0/362928717 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc009510 con 0x7f2904072f20 2024-04-25T16:51:23.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f74034ce640 1 -- 172.21.15.94:0/1104534029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc168890 con 0x7f73fc15b920 2024-04-25T16:51:23.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f74034ce640 1 -- 172.21.15.94:0/1104534029 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f73fc168b90 con 0x7f73fc0731d0 2024-04-25T16:51:23.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f7401ccb640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73fc0731d0 0x7f73fc1644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f29099de640 1 -- 172.21.15.94:0/362928717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29040cf170 con 0x7f2904072f20 2024-04-25T16:51:23.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f29099de640 1 -- 172.21.15.94:0/362928717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29040cf730 con 0x7f2904072f20 2024-04-25T16:51:23.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa80b3e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7fc054620 0x7fa7fc0e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f86aaffd640 1 -- 172.21.15.94:0/2430853029 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86ac0a5ab0 con 0x7f86bc094b80 2024-04-25T16:51:23.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f86aaffd640 1 -- 172.21.15.94:0/2430853029 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0b2040 con 0x7f86bc094b80 2024-04-25T16:51:23.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f29017fa640 1 -- 172.21.15.94:0/362928717 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28fc0be070 con 0x7f2904072f20 2024-04-25T16:51:23.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f29017fa640 1 -- 172.21.15.94:0/362928717 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0b8870 con 0x7f2904072f20 2024-04-25T16:51:23.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7fa2dce1b640 1 -- start start 2024-04-25T16:51:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.696+0000 7fa2dce1b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d806c9a0 0x7fa2d8150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2d77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d806c9a0 0x7fa2d8150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2d77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d806c9a0 0x7fa2d8150360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42428/0 (socket says 172.21.15.94:42428) 2024-04-25T16:51:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2d77fe640 1 -- 172.21.15.94:0/4157092563 learned_addr learned my addr 172.21.15.94:0/4157092563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2dce1b640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d80d0b80 0x7fa2d815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2dce1b640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d81607f0 0x7fa2d8160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2dce1b640 1 -- 172.21.15.94:0/4157092563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8163a50 con 0x7fa2d806c9a0 2024-04-25T16:51:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2dce1b640 1 -- 172.21.15.94:0/4157092563 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81638d0 con 0x7fa2d80d0b80 2024-04-25T16:51:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2dce1b640 1 -- 172.21.15.94:0/4157092563 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d816d270 con 0x7fa2d81607f0 2024-04-25T16:51:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2d6ffd640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d80d0b80 0x7fa2d815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2d77fe640 1 -- 172.21.15.94:0/4157092563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d81607f0 msgr2=0x7fa2d8160c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T16:51:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2d77fe640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d81607f0 0x7fa2d8160c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2d77fe640 1 -- 172.21.15.94:0/4157092563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d80d0b80 msgr2=0x7fa2d815ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2d77fe640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d80d0b80 0x7fa2d815ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2d77fe640 1 -- 172.21.15.94:0/4157092563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d815f250 con 0x7fa2d806c9a0 2024-04-25T16:51:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.700+0000 7fa2d77fe640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d806c9a0 0x7fa2d8150360 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00c3f0 tx=0x7fa2cc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa2bffff640 1 -- 172.21.15.94:0/4157092563 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc017070 con 0x7fa2d806c9a0 2024-04-25T16:51:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa2dce1b640 1 -- 172.21.15.94:0/4157092563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d815dc70 con 0x7fa2d806c9a0 2024-04-25T16:51:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa2dce1b640 1 -- 172.21.15.94:0/4157092563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d815e230 con 0x7fa2d806c9a0 2024-04-25T16:51:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa2bffff640 1 -- 172.21.15.94:0/4157092563 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2cc0052d0 con 0x7fa2d806c9a0 2024-04-25T16:51:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa2bffff640 1 -- 172.21.15.94:0/4157092563 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0069e0 con 0x7fa2d806c9a0 2024-04-25T16:51:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa80b3e5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7fc057460 0x7fa7fc0d8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa80b3e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7fc0e37d0 0x7fa7fc0dd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa80b3e5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc03cf00 con 0x7fa7fc0e37d0 2024-04-25T16:51:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa809181640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7fc054620 0x7fa7fc0e06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa809181640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7fc054620 0x7fa7fc0e06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41210/0 (socket says 172.21.15.94:41210) 2024-04-25T16:51:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa809181640 1 -- 172.21.15.94:0/2191445057 learned_addr learned my addr 172.21.15.94:0/2191445057 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa80b3e5640 1 -- 172.21.15.94:0/2191445057 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc006ea0 con 0x7fa7fc054620 2024-04-25T16:51:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa80b3e5640 1 -- 172.21.15.94:0/2191445057 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc0f1d00 con 0x7fa7fc057460 2024-04-25T16:51:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa808980640 1 --2- 172.21.15.94:0/2191445057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7fc057460 0x7fa7fc0d8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f29099de640 1 -- 172.21.15.94:0/362928717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f290406a310 con 0x7f2904072f20 2024-04-25T16:51:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f86c0875640 1 -- 172.21.15.94:0/2430853029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86a00052a0 con 0x7f86bc094b80 2024-04-25T16:51:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa809181640 1 -- 172.21.15.94:0/2191445057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7fc057460 msgr2=0x7fa7fc0d8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa809181640 1 --2- 172.21.15.94:0/2191445057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7fc057460 0x7fa7fc0d8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa809181640 1 -- 172.21.15.94:0/2191445057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7fc0e37d0 msgr2=0x7fa7fc0dd160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T16:51:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa809181640 1 --2- 172.21.15.94:0/2191445057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7fc0e37d0 0x7fa7fc0dd160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa809181640 1 -- 172.21.15.94:0/2191445057 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc0dcab0 con 0x7fa7fc054620 2024-04-25T16:51:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa809181640 1 --2- 172.21.15.94:0/2191445057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7fc054620 0x7fa7fc0e06e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa80406b700 tx=0x7fa804069c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa7f9ffb640 1 -- 172.21.15.94:0/2191445057 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804075ec0 con 0x7fa7fc054620 2024-04-25T16:51:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f7402ccd640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73fc15b920 0x7f73fc156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f7401ccb640 1 -- 172.21.15.94:0/1104534029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73fc15b920 msgr2=0x7f73fc156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f7401ccb640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73fc15b920 0x7f73fc156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f7401ccb640 1 -- 172.21.15.94:0/1104534029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73fc06cae0 msgr2=0x7f73fc15bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f7401ccb640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73fc06cae0 0x7f73fc15bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f7401ccb640 1 -- 172.21.15.94:0/1104534029 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc209b40 con 0x7f73fc0731d0 2024-04-25T16:51:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f7401ccb640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73fc0731d0 0x7f73fc1644f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e8e0 tx=0x7f73f800eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f73f2ffd640 1 -- 172.21.15.94:0/1104534029 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8017070 con 0x7f73fc0731d0 2024-04-25T16:51:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f73f2ffd640 1 -- 172.21.15.94:0/1104534029 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f800c8e0 con 0x7f73fc0731d0 2024-04-25T16:51:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7f73f2ffd640 1 -- 172.21.15.94:0/1104534029 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f800ccd0 con 0x7f73fc0731d0 2024-04-25T16:51:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7f86aaffd640 1 -- 172.21.15.94:0/2430853029 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f86ac0bf070 con 0x7f86bc094b80 2024-04-25T16:51:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa80b3e5640 1 -- 172.21.15.94:0/2191445057 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc0dfc00 con 0x7fa7fc054620 2024-04-25T16:51:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa7f9ffb640 1 -- 172.21.15.94:0/2191445057 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa804076060 con 0x7fa7fc054620 2024-04-25T16:51:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa7f9ffb640 1 -- 172.21.15.94:0/2191445057 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80406c600 con 0x7fa7fc054620 2024-04-25T16:51:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.704+0000 7fa80b3e5640 1 -- 172.21.15.94:0/2191445057 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc0e01c0 con 0x7fa7fc054620 2024-04-25T16:51:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7f86aaffd640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f869c06f430 0x7f869c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7f74034ce640 1 -- 172.21.15.94:0/1104534029 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73fc209e20 con 0x7f73fc0731d0 2024-04-25T16:51:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7f74034ce640 1 -- 172.21.15.94:0/1104534029 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f73fc15c370 con 0x7f73fc0731d0 2024-04-25T16:51:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7fa80b3e5640 1 -- 172.21.15.94:0/2191445057 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7fc005580 con 0x7fa7fc054620 2024-04-25T16:51:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7f86ba59c640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f869c06f430 0x7f869c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7fa2bffff640 1 -- 172.21.15.94:0/4157092563 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa2cc01a070 con 0x7fa2d806c9a0 2024-04-25T16:51:23.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7f86ba59c640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f869c06f430 0x7f869c071940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f86b4004bd0 tx=0x7f86b4005260 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7f73f0ff9640 1 -- 172.21.15.94:0/1104534029 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d80052a0 con 0x7f73fc0731d0 2024-04-25T16:51:23.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7fa2bffff640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa2c4091440 0x7fa2c4093950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7fa7f9ffb640 1 -- 172.21.15.94:0/2191445057 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa80406c7a0 con 0x7fa7fc054620 2024-04-25T16:51:23.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7fa2d6ffd640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa2c4091440 0x7fa2c4093950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7fa7f9ffb640 1 --2- 172.21.15.94:0/2191445057 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa7e806f400 0x7fa7e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7fa808980640 1 --2- 172.21.15.94:0/2191445057 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa7e806f400 0x7fa7e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7fa7f9ffb640 1 -- 172.21.15.94:0/2191445057 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa80410b980 con 0x7fa7fc054620 2024-04-25T16:51:23.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7fa2d6ffd640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa2c4091440 0x7fa2c4093950 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8157ab0 tx=0x7fa2c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.708+0000 7fa808980640 1 --2- 172.21.15.94:0/2191445057 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa7e806f400 0x7fa7e8071910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa8000ae000 tx=0x7fa8000b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.712+0000 7f73f2ffd640 1 -- 172.21.15.94:0/1104534029 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f73f801a070 con 0x7f73fc0731d0 2024-04-25T16:51:23.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.712+0000 7f73f2ffd640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f73e006f400 0x7f73e0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.712+0000 7f73f2ffd640 1 -- 172.21.15.94:0/1104534029 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f73f808f8b0 con 0x7f73fc0731d0 2024-04-25T16:51:23.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.712+0000 7f74024cc640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f73e006f400 0x7f73e0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.712+0000 7f74024cc640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f73e006f400 0x7f73e0071910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f73ec005890 tx=0x7f73ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.712+0000 7f29017fa640 1 -- 172.21.15.94:0/362928717 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f28fc0b8a10 con 0x7f2904072f20 2024-04-25T16:51:23.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.712+0000 7f29017fa640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f28f006f400 0x7f28f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.712+0000 7f29089dc640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f28f006f400 0x7f28f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.712+0000 7f29089dc640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f28f006f400 0x7f28f0071910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009600 tx=0x7f28f8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.712+0000 7f86aaffd640 1 -- 172.21.15.94:0/2430853029 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f86ac127420 con 0x7f86bc094b80 2024-04-25T16:51:23.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.712+0000 7fa2bffff640 1 -- 172.21.15.94:0/4157092563 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa2cc08cbf0 con 0x7fa2d806c9a0 2024-04-25T16:51:23.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.712+0000 7f29017fa640 1 -- 172.21.15.94:0/362928717 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f28fc129450 con 0x7f2904072f20 2024-04-25T16:51:23.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.716+0000 7fa2dce1b640 1 -- 172.21.15.94:0/4157092563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b80052a0 con 0x7fa2d806c9a0 2024-04-25T16:51:23.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.720+0000 7f86aaffd640 1 -- 172.21.15.94:0/2430853029 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f86bc15fcf0 con 0x7f86bc094b80 2024-04-25T16:51:23.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.724+0000 7fa7f9ffb640 1 -- 172.21.15.94:0/2191445057 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa8040d7fc0 con 0x7fa7fc054620 2024-04-25T16:51:23.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.728+0000 7f29017fa640 1 -- 172.21.15.94:0/362928717 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f28fc016660 con 0x7f2904072f20 2024-04-25T16:51:23.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.724+0000 7f73f2ffd640 1 -- 172.21.15.94:0/1104534029 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f73f805be10 con 0x7f73fc0731d0 2024-04-25T16:51:23.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.736+0000 7fa2bffff640 1 -- 172.21.15.94:0/4157092563 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa2d80d0f90 con 0x7fa2d806c9a0 2024-04-25T16:51:23.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.756+0000 7fab6f651640 1 Processor -- start 2024-04-25T16:51:23.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.756+0000 7fab6f651640 1 -- start start 2024-04-25T16:51:23.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.760+0000 7fab6f651640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab6816ef60 0x7fab6809a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.760+0000 7fab6f651640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab6816d110 con 0x7fab6806c600 2024-04-25T16:51:23.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.760+0000 7fab6f651640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab6806d1e0 con 0x7fab680b2df0 2024-04-25T16:51:23.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.760+0000 7fab6f651640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fab6806ddf0 con 0x7fab6816ef60 2024-04-25T16:51:23.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.760+0000 7fab6cbec640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fab6806c600 0x7fab6806ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39138/0 (socket says 172.21.15.94:39138) 2024-04-25T16:51:23.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.760+0000 7fab6cbec640 1 -- 172.21.15.94:0/674563210 learned_addr learned my addr 172.21.15.94:0/674563210 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:23.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.760+0000 7fab6dbee640 1 --2- 172.21.15.94:0/674563210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab6816ef60 0x7fab6809a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.760+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1042706436 0 0) 0x7fab6816d110 con 0x7fab6806c600 2024-04-25T16:51:23.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab54003510 con 0x7fab6806c600 2024-04-25T16:51:23.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2714125012 0 0) 0x7fab6806d1e0 con 0x7fab680b2df0 2024-04-25T16:51:23.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab6816d110 con 0x7fab680b2df0 2024-04-25T16:51:23.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 651555417 0 0) 0x7fab54003510 con 0x7fab6806c600 2024-04-25T16:51:23.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab6816ef60 msgr2=0x7fab6809a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:23.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 --2- 172.21.15.94:0/674563210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab6816ef60 0x7fab6809a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 >> v1:172.21.15.138:6789/0 conn(0x7fab680b2df0 legacy=0x7fab680b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6806d4c0 con 0x7fab6806c600 2024-04-25T16:51:23.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fab580016a0 con 0x7fab6806c600 2024-04-25T16:51:23.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 >> v1:172.21.15.94:6789/0 conn(0x7fab6806c600 legacy=0x7fab6806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 --2- 172.21.15.94:0/674563210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab6816ef60 0x7fab54003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 --2- 172.21.15.94:0/674563210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab54004220 0x7fab54004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 --2- 172.21.15.94:0/674563210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab54008e60 0x7fab54013c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab6806ddf0 con 0x7fab54004220 2024-04-25T16:51:23.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab54014350 con 0x7fab54008e60 2024-04-25T16:51:23.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fab54014550 con 0x7fab6816ef60 2024-04-25T16:51:23.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab6cbec640 1 --2- 172.21.15.94:0/674563210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab54004220 0x7fab54004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab6d3ed640 1 --2- 172.21.15.94:0/674563210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab6816ef60 0x7fab54003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab6dbee640 1 --2- 172.21.15.94:0/674563210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab54008e60 0x7fab54013c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab6cbec640 1 -- 172.21.15.94:0/674563210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab6816ef60 msgr2=0x7fab54003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:23.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab6cbec640 1 --2- 172.21.15.94:0/674563210 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab6816ef60 0x7fab54003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab6cbec640 1 -- 172.21.15.94:0/674563210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab54008e60 msgr2=0x7fab54013c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:23.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab6cbec640 1 --2- 172.21.15.94:0/674563210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab54008e60 0x7fab54013c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.772+0000 7fab6cbec640 1 -- 172.21.15.94:0/674563210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab54003510 con 0x7fab54004220 2024-04-25T16:51:23.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.776+0000 7fab6cbec640 1 --2- 172.21.15.94:0/674563210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab54004220 0x7fab54004690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab680726f0 tx=0x7fab5801d1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c640e84a05a45f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.776+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58004c20 con 0x7fab54004220 2024-04-25T16:51:23.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.776+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab58005ae0 con 0x7fab54004220 2024-04-25T16:51:23.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.776+0000 7fab5ffff640 1 -- 172.21.15.94:0/674563210 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58008110 con 0x7fab54004220 2024-04-25T16:51:23.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.776+0000 7fab6f651640 1 -- 172.21.15.94:0/674563210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab54004220 msgr2=0x7fab54004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.776+0000 7fab6f651640 1 --2- 172.21.15.94:0/674563210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab54004220 0x7fab54004690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab680726f0 tx=0x7fab5801d1e0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.776+0000 7fab6dbee640 1 -- 172.21.15.94:0/674563210 reap_dead start 2024-04-25T16:51:23.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.776+0000 7fab6f651640 1 -- 172.21.15.94:0/674563210 shutdown_connections 2024-04-25T16:51:23.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.776+0000 7fab6f651640 1 -- 172.21.15.94:0/674563210 >> 172.21.15.94:0/674563210 conn(0x7fab68072970 msgr2=0x7fab680b6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:23.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.776+0000 7fab6f651640 1 -- 172.21.15.94:0/674563210 shutdown_connections 2024-04-25T16:51:23.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.776+0000 7fab6f651640 1 -- 172.21.15.94:0/674563210 wait complete. 2024-04-25T16:51:23.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.780+0000 7fab6f651640 1 Processor -- start 2024-04-25T16:51:23.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.780+0000 7fab6f651640 1 -- start start 2024-04-25T16:51:23.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6f651640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab68207090 0x7fab6808e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6d3ed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab68207090 0x7fab6808e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6d3ed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab68207090 0x7fab6808e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42462/0 (socket says 172.21.15.94:42462) 2024-04-25T16:51:23.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6d3ed640 1 -- 172.21.15.94:0/261467343 learned_addr learned my addr 172.21.15.94:0/261467343 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:23.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6f651640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab680b0400 0x7fab680b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6f651640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab6809a560 0x7fab6809a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6f651640 1 -- 172.21.15.94:0/261467343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab68170020 con 0x7fab68207090 2024-04-25T16:51:23.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6f651640 1 -- 172.21.15.94:0/261467343 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab6806da80 con 0x7fab6809a560 2024-04-25T16:51:23.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6f651640 1 -- 172.21.15.94:0/261467343 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fab6809a1d0 con 0x7fab680b0400 2024-04-25T16:51:23.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6cbec640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab680b0400 0x7fab680b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6dbee640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab6809a560 0x7fab6809a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6d3ed640 1 -- 172.21.15.94:0/261467343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab680b0400 msgr2=0x7fab680b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6d3ed640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab680b0400 0x7fab680b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6d3ed640 1 -- 172.21.15.94:0/261467343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab6809a560 msgr2=0x7fab6809a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6d3ed640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab6809a560 0x7fab6809a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6d3ed640 1 -- 172.21.15.94:0/261467343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6809aeb0 con 0x7fab68207090 2024-04-25T16:51:23.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6d3ed640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab68207090 0x7fab6808e3d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab60011820 tx=0x7fab60011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab5dffb640 1 -- 172.21.15.94:0/261467343 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab60019020 con 0x7fab68207090 2024-04-25T16:51:23.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6f651640 1 -- 172.21.15.94:0/261467343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab680b1130 con 0x7fab68207090 2024-04-25T16:51:23.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.784+0000 7fab6f651640 1 -- 172.21.15.94:0/261467343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab68206890 con 0x7fab68207090 2024-04-25T16:51:23.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.788+0000 7fab5dffb640 1 -- 172.21.15.94:0/261467343 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab60003db0 con 0x7fab68207090 2024-04-25T16:51:23.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.788+0000 7fab5dffb640 1 -- 172.21.15.94:0/261467343 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab60005310 con 0x7fab68207090 2024-04-25T16:51:23.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.792+0000 7fab5dffb640 1 -- 172.21.15.94:0/261467343 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fab6001c020 con 0x7fab68207090 2024-04-25T16:51:23.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.792+0000 7fab5dffb640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fab54079690 0x7fab5407bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:23.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.792+0000 7fab6cbec640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fab54079690 0x7fab5407bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:23.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.792+0000 7fab5dffb640 1 -- 172.21.15.94:0/261467343 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fab60092430 con 0x7fab68207090 2024-04-25T16:51:23.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.792+0000 7fab6f651640 1 -- 172.21.15.94:0/261467343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab480052a0 con 0x7fab68207090 2024-04-25T16:51:23.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.792+0000 7fab6cbec640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fab54079690 0x7fab5407bba0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fab58008fd0 tx=0x7fab58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:23.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.804+0000 7fab5dffb640 1 -- 172.21.15.94:0/261467343 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fab60023030 con 0x7fab68207090 2024-04-25T16:51:23.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.976+0000 7fcb8fa00640 1 -- 172.21.15.94:0/245591537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fcb881624e0 con 0x7fcb88164750 2024-04-25T16:51:23.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.976+0000 7fcb7dffb640 1 -- 172.21.15.94:0/245591537 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcb8405f1a0 con 0x7fcb88164750 2024-04-25T16:51:23.975 INFO:teuthology.orchestra.run.smithi094.stdout:55834574860 2024-04-25T16:51:23.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.980+0000 7fcb737fe640 1 -- 172.21.15.94:0/245591537 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcb6c06f500 msgr2=0x7fcb6c071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.980+0000 7fcb737fe640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcb6c06f500 0x7fcb6c071a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcb78006fd0 tx=0x7fcb78009060 comp rx=0 tx=0).stop 2024-04-25T16:51:23.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.980+0000 7fcb737fe640 1 -- 172.21.15.94:0/245591537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb88164750 msgr2=0x7fcb880aaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:23.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.980+0000 7fcb737fe640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb88164750 0x7fcb880aaa10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400c3c0 tx=0x7fcb8400c880 comp rx=0 tx=0).stop 2024-04-25T16:51:23.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.980+0000 7fcb737fe640 1 -- 172.21.15.94:0/245591537 shutdown_connections 2024-04-25T16:51:23.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.980+0000 7fcb737fe640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb88164750 0x7fcb880aaa10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.980+0000 7fcb737fe640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb88164340 0x7fcb880aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.980+0000 7fcb737fe640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcb6c06f500 0x7fcb6c071a10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.980+0000 7fcb737fe640 1 --2- 172.21.15.94:0/245591537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb880704b0 0x7fcb880709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:23.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.980+0000 7fcb737fe640 1 -- 172.21.15.94:0/245591537 >> 172.21.15.94:0/245591537 conn(0x7fcb8806f990 msgr2=0x7fcb8806fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:23.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.980+0000 7fcb737fe640 1 -- 172.21.15.94:0/245591537 shutdown_connections 2024-04-25T16:51:23.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:23.980+0000 7fcb737fe640 1 -- 172.21.15.94:0/245591537 wait complete. 2024-04-25T16:51:24.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.2 2024-04-25T16:51:24.004 DEBUG:teuthology.parallel:result is None 2024-04-25T16:51:24.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.012+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2887010740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb5b006a730 con 0x7fb5b00b09c0 2024-04-25T16:51:24.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.012+0000 7fb59effd640 1 -- 172.21.15.94:0/2887010740 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb5a005bca0 con 0x7fb5b00b09c0 2024-04-25T16:51:24.010 INFO:teuthology.orchestra.run.smithi094.stdout:55834574858 2024-04-25T16:51:24.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.016+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2887010740 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb59006f430 msgr2=0x7fb590071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.016+0000 7fb5b4c09640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb59006f430 0x7fb590071940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0071430 tx=0x7fb5a4008040 comp rx=0 tx=0).stop 2024-04-25T16:51:24.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.016+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2887010740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b00b09c0 msgr2=0x7fb5b0153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.016+0000 7fb5b4c09640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b00b09c0 0x7fb5b0153cf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000e8a0 tx=0x7fb5a000ed60 comp rx=0 tx=0).stop 2024-04-25T16:51:24.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.016+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2887010740 shutdown_connections 2024-04-25T16:51:24.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.016+0000 7fb5b4c09640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b01647c0 0x7fb5b00b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.016+0000 7fb5b4c09640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b00b09c0 0x7fb5b0153cf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.016+0000 7fb5b4c09640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb59006f430 0x7fb590071940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.016+0000 7fb5b4c09640 1 --2- 172.21.15.94:0/2887010740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5b006db50 0x7fb5b015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.016+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2887010740 >> 172.21.15.94:0/2887010740 conn(0x7fb5b00d1a30 msgr2=0x7fb5b0165540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:24.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.016+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2887010740 shutdown_connections 2024-04-25T16:51:24.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.016+0000 7fb5b4c09640 1 -- 172.21.15.94:0/2887010740 wait complete. 2024-04-25T16:51:24.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.036+0000 7f73f0ff9640 1 -- 172.21.15.94:0/1104534029 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f73d8005860 con 0x7f73fc0731d0 2024-04-25T16:51:24.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.036+0000 7f73f2ffd640 1 -- 172.21.15.94:0/1104534029 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f73f8060110 con 0x7f73fc0731d0 2024-04-25T16:51:24.034 INFO:teuthology.orchestra.run.smithi094.stdout:55834574860 2024-04-25T16:51:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.040+0000 7f73f0ff9640 1 -- 172.21.15.94:0/1104534029 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f73e006f400 msgr2=0x7f73e0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.040+0000 7f73f0ff9640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f73e006f400 0x7f73e0071910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f73ec005890 tx=0x7f73ec009040 comp rx=0 tx=0).stop 2024-04-25T16:51:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.040+0000 7f73f0ff9640 1 -- 172.21.15.94:0/1104534029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73fc0731d0 msgr2=0x7f73fc1644f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.040+0000 7f73f0ff9640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73fc0731d0 0x7f73fc1644f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e8e0 tx=0x7f73f800eda0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.040+0000 7f73f0ff9640 1 -- 172.21.15.94:0/1104534029 shutdown_connections 2024-04-25T16:51:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.040+0000 7f73f0ff9640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73fc15b920 0x7f73fc156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.040+0000 7f73f0ff9640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f73e006f400 0x7f73e0071910 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.040+0000 7f73f0ff9640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73fc0731d0 0x7f73fc1644f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.040+0000 7f73f0ff9640 1 --2- 172.21.15.94:0/1104534029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73fc06cae0 0x7f73fc15bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.040+0000 7f73f0ff9640 1 -- 172.21.15.94:0/1104534029 >> 172.21.15.94:0/1104534029 conn(0x7f73fc0d1020 msgr2=0x7f73fc0d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.040+0000 7f73f0ff9640 1 -- 172.21.15.94:0/1104534029 shutdown_connections 2024-04-25T16:51:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.040+0000 7f73f0ff9640 1 -- 172.21.15.94:0/1104534029 wait complete. 2024-04-25T16:51:24.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2024-04-25T16:51:24.040 DEBUG:teuthology.parallel:result is None 2024-04-25T16:51:24.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.048+0000 7fa2dce1b640 1 -- 172.21.15.94:0/4157092563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa2b8005860 con 0x7fa2d806c9a0 2024-04-25T16:51:24.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.048+0000 7fa2bffff640 1 -- 172.21.15.94:0/4157092563 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa2cc005480 con 0x7fa2d806c9a0 2024-04-25T16:51:24.047 INFO:teuthology.orchestra.run.smithi094.stdout:55834574860 2024-04-25T16:51:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7f86c0875640 1 -- 172.21.15.94:0/2430853029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f86a0005590 con 0x7f86bc094b80 2024-04-25T16:51:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7f86aaffd640 1 -- 172.21.15.94:0/2430853029 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f86ac0169b0 con 0x7f86bc094b80 2024-04-25T16:51:24.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7fa2dce1b640 1 -- 172.21.15.94:0/4157092563 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa2c4091440 msgr2=0x7fa2c4093950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7fa2dce1b640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa2c4091440 0x7fa2c4093950 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8157ab0 tx=0x7fa2c8008040 comp rx=0 tx=0).stop 2024-04-25T16:51:24.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7fa2dce1b640 1 -- 172.21.15.94:0/4157092563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d806c9a0 msgr2=0x7fa2d8150360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.049 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:51:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7fa2dce1b640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d806c9a0 0x7fa2d8150360 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00c3f0 tx=0x7fa2cc00c8b0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7fa2dce1b640 1 -- 172.21.15.94:0/4157092563 shutdown_connections 2024-04-25T16:51:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7fa2dce1b640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa2c4091440 0x7fa2c4093950 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7fa2dce1b640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d81607f0 0x7fa2d8160c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7fa2dce1b640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d80d0b80 0x7fa2d815ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7fa2dce1b640 1 --2- 172.21.15.94:0/4157092563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2d806c9a0 0x7fa2d8150360 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7fa2dce1b640 1 -- 172.21.15.94:0/4157092563 >> 172.21.15.94:0/4157092563 conn(0x7fa2d80b6ef0 msgr2=0x7fa2d80b96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:24.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7fa2dce1b640 1 -- 172.21.15.94:0/4157092563 shutdown_connections 2024-04-25T16:51:24.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.052+0000 7fa2dce1b640 1 -- 172.21.15.94:0/4157092563 wait complete. 2024-04-25T16:51:24.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f29099de640 1 -- 172.21.15.94:0/362928717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f29040cd690 con 0x7f2904072f20 2024-04-25T16:51:24.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7fa80b3e5640 1 -- 172.21.15.94:0/2191445057 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fa7fc04c4a0 con 0x7fa7fc054620 2024-04-25T16:51:24.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f29017fa640 1 -- 172.21.15.94:0/362928717 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f28fc0f77d0 con 0x7f2904072f20 2024-04-25T16:51:24.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f86c0875640 1 -- 172.21.15.94:0/2430853029 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f869c06f430 msgr2=0x7f869c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f86c0875640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f869c06f430 0x7f869c071940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f86b4004bd0 tx=0x7f86b4005260 comp rx=0 tx=0).stop 2024-04-25T16:51:24.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f86c0875640 1 -- 172.21.15.94:0/2430853029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc094b80 msgr2=0x7f86bc15f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f86c0875640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc094b80 0x7f86bc15f3a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f86ac000c10 tx=0x7f86a4003740 comp rx=0 tx=0).stop 2024-04-25T16:51:24.053 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:51:24.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f86c0875640 1 -- 172.21.15.94:0/2430853029 shutdown_connections 2024-04-25T16:51:24.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f86c0875640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f869c06f430 0x7f869c071940 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f86c0875640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86bc15f8e0 0x7f86bc159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f86c0875640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc094b80 0x7f86bc15f3a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f86c0875640 1 --2- 172.21.15.94:0/2430853029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a400ac60 0x7f86bc094640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f86c0875640 1 -- 172.21.15.94:0/2430853029 >> 172.21.15.94:0/2430853029 conn(0x7f86bc072970 msgr2=0x7f86bc156620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:24.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f86c0875640 1 -- 172.21.15.94:0/2430853029 shutdown_connections 2024-04-25T16:51:24.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f86c0875640 1 -- 172.21.15.94:0/2430853029 wait complete. 2024-04-25T16:51:24.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7fa7f9ffb640 1 -- 172.21.15.94:0/2191445057 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa8040dc2c0 con 0x7fa7fc054620 2024-04-25T16:51:24.055 INFO:teuthology.orchestra.run.smithi094.stdout:55834574860 2024-04-25T16:51:24.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f29099de640 1 -- 172.21.15.94:0/362928717 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f28f006f400 msgr2=0x7f28f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f29099de640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f28f006f400 0x7f28f0071910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009600 tx=0x7f28f8009190 comp rx=0 tx=0).stop 2024-04-25T16:51:24.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f29099de640 1 -- 172.21.15.94:0/362928717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2904072f20 msgr2=0x7f2904159bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.056+0000 7f29099de640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2904072f20 0x7f2904159bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0ab000 tx=0x7f28fc019730 comp rx=0 tx=0).stop 2024-04-25T16:51:24.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7fa80b3e5640 1 -- 172.21.15.94:0/2191445057 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa7e806f400 msgr2=0x7fa7e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7fa80b3e5640 1 --2- 172.21.15.94:0/2191445057 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa7e806f400 0x7fa7e8071910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa8000ae000 tx=0x7fa8000b5040 comp rx=0 tx=0).stop 2024-04-25T16:51:24.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7fa80b3e5640 1 -- 172.21.15.94:0/2191445057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7fc054620 msgr2=0x7fa7fc0e06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7fa80b3e5640 1 --2- 172.21.15.94:0/2191445057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7fc054620 0x7fa7fc0e06e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa80406b700 tx=0x7fa804069c20 comp rx=0 tx=0).stop 2024-04-25T16:51:24.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7f29099de640 1 -- 172.21.15.94:0/362928717 shutdown_connections 2024-04-25T16:51:24.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7f29099de640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f28f006f400 0x7f28f0071910 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7f29099de640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2904164fe0 0x7f2904206170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7f29099de640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2904072f20 0x7f2904159bf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7f29099de640 1 --2- 172.21.15.94:0/362928717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2904072b10 0x7f290415f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7f29099de640 1 -- 172.21.15.94:0/362928717 >> 172.21.15.94:0/362928717 conn(0x7f29040d1020 msgr2=0x7f29040b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:24.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7fa80b3e5640 1 -- 172.21.15.94:0/2191445057 shutdown_connections 2024-04-25T16:51:24.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7fa80b3e5640 1 --2- 172.21.15.94:0/2191445057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7fc0e37d0 0x7fa7fc0dd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7fa80b3e5640 1 --2- 172.21.15.94:0/2191445057 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa7e806f400 0x7fa7e8071910 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7fa80b3e5640 1 --2- 172.21.15.94:0/2191445057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7fc057460 0x7fa7fc0d8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7fa80b3e5640 1 --2- 172.21.15.94:0/2191445057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7fc054620 0x7fa7fc0e06e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7fa80b3e5640 1 -- 172.21.15.94:0/2191445057 >> 172.21.15.94:0/2191445057 conn(0x7fa7fc0591a0 msgr2=0x7fa7fc058860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:24.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7f29099de640 1 -- 172.21.15.94:0/362928717 shutdown_connections 2024-04-25T16:51:24.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7f29099de640 1 -- 172.21.15.94:0/362928717 wait complete. 2024-04-25T16:51:24.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7fa80b3e5640 1 -- 172.21.15.94:0/2191445057 shutdown_connections 2024-04-25T16:51:24.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.060+0000 7fa80b3e5640 1 -- 172.21.15.94:0/2191445057 wait complete. 2024-04-25T16:51:24.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.0 2024-04-25T16:51:24.063 DEBUG:teuthology.parallel:result is None 2024-04-25T16:51:24.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-04-25T16:51:24.077 DEBUG:teuthology.parallel:result is None 2024-04-25T16:51:24.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.084+0000 7fab6f651640 1 -- 172.21.15.94:0/261467343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fab48005590 con 0x7fab68207090 2024-04-25T16:51:24.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.084+0000 7fab5dffb640 1 -- 172.21.15.94:0/261467343 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fab6809aeb0 con 0x7fab68207090 2024-04-25T16:51:24.083 INFO:teuthology.orchestra.run.smithi094.stdout:55834574859 2024-04-25T16:51:24.086 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.5 2024-04-25T16:51:24.086 DEBUG:teuthology.parallel:result is None 2024-04-25T16:51:24.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.088+0000 7fab6f651640 1 -- 172.21.15.94:0/261467343 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fab54079690 msgr2=0x7fab5407bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.088+0000 7fab6f651640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fab54079690 0x7fab5407bba0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fab58008fd0 tx=0x7fab58008040 comp rx=0 tx=0).stop 2024-04-25T16:51:24.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.088+0000 7fab6f651640 1 -- 172.21.15.94:0/261467343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab68207090 msgr2=0x7fab6808e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.088+0000 7fab6f651640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab68207090 0x7fab6808e3d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab60011820 tx=0x7fab60011ce0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.088+0000 7fab6f651640 1 -- 172.21.15.94:0/261467343 shutdown_connections 2024-04-25T16:51:24.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.088+0000 7fab6f651640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab6809a560 0x7fab6809a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.088+0000 7fab6f651640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fab54079690 0x7fab5407bba0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.088+0000 7fab6f651640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab680b0400 0x7fab680b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.088+0000 7fab6f651640 1 --2- 172.21.15.94:0/261467343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab68207090 0x7fab6808e3d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.088+0000 7fab6f651640 1 -- 172.21.15.94:0/261467343 >> 172.21.15.94:0/261467343 conn(0x7fab68072970 msgr2=0x7fab680b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:24.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.088+0000 7fab6f651640 1 -- 172.21.15.94:0/261467343 shutdown_connections 2024-04-25T16:51:24.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.088+0000 7fab6f651640 1 -- 172.21.15.94:0/261467343 wait complete. 2024-04-25T16:51:24.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-04-25T16:51:24.090 DEBUG:teuthology.parallel:result is None 2024-04-25T16:51:24.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.7 2024-04-25T16:51:24.091 DEBUG:teuthology.parallel:result is None 2024-04-25T16:51:24.103 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2024-04-25T16:51:24.103 DEBUG:teuthology.parallel:result is None 2024-04-25T16:51:24.103 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T16:51:24.104 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:51:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.276+0000 7fe0395da640 1 Processor -- start 2024-04-25T16:51:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.276+0000 7fe0395da640 1 -- start start 2024-04-25T16:51:24.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.276+0000 7fe0395da640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe03406de20 0x7fe0340af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.276+0000 7fe0395da640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe034161300 con 0x7fe03415cf40 2024-04-25T16:51:24.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.276+0000 7fe0395da640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe03415d350 con 0x7fe034070180 2024-04-25T16:51:24.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.276+0000 7fe0395da640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe03415dec0 con 0x7fe03406de20 2024-04-25T16:51:24.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.276+0000 7fe0327fc640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fe03415cf40 0x7fe03406d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39142/0 (socket says 172.21.15.94:39142) 2024-04-25T16:51:24.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.276+0000 7fe0327fc640 1 -- 172.21.15.94:0/3003053371 learned_addr learned my addr 172.21.15.94:0/3003053371 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.276+0000 7fe0337fe640 1 --2- 172.21.15.94:0/3003053371 >> v2:172.21.15.94:3300/0 conn(0x7fe03406de20 0x7fe0340af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.276+0000 7fe0337fe640 1 -- 172.21.15.94:0/3003053371 >> v1:172.21.15.138:6789/0 conn(0x7fe034070180 legacy=0x7fe034160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.276+0000 7fe0337fe640 1 -- 172.21.15.94:0/3003053371 >> v1:172.21.15.94:6789/0 conn(0x7fe03415cf40 legacy=0x7fe03406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.276+0000 7fe0337fe640 1 -- 172.21.15.94:0/3003053371 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03415d620 con 0x7fe03406de20 2024-04-25T16:51:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.280+0000 7fe0337fe640 1 --2- 172.21.15.94:0/3003053371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03406de20 0x7fe0340af2c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009d90 tx=0x7fe02c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b210cd3291deebf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.280+0000 7fe031ffb640 1 -- 172.21.15.94:0/3003053371 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c0b2070 con 0x7fe03406de20 2024-04-25T16:51:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.280+0000 7fe031ffb640 1 -- 172.21.15.94:0/3003053371 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe02c0a58e0 con 0x7fe03406de20 2024-04-25T16:51:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.280+0000 7fe031ffb640 1 -- 172.21.15.94:0/3003053371 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c0a5cb0 con 0x7fe03406de20 2024-04-25T16:51:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.280+0000 7fe0395da640 1 -- 172.21.15.94:0/3003053371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03406de20 msgr2=0x7fe0340af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.280+0000 7fe0395da640 1 --2- 172.21.15.94:0/3003053371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03406de20 0x7fe0340af2c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009d90 tx=0x7fe02c01c930 comp rx=0 tx=0).stop 2024-04-25T16:51:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.280+0000 7fe0395da640 1 -- 172.21.15.94:0/3003053371 shutdown_connections 2024-04-25T16:51:24.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.280+0000 7fe0395da640 1 --2- 172.21.15.94:0/3003053371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03406de20 0x7fe0340af2c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.280+0000 7fe0395da640 1 -- 172.21.15.94:0/3003053371 >> 172.21.15.94:0/3003053371 conn(0x7fe0340b47f0 msgr2=0x7fe0340b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:24.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.280+0000 7fe0395da640 1 -- 172.21.15.94:0/3003053371 shutdown_connections 2024-04-25T16:51:24.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.280+0000 7fe0395da640 1 -- 172.21.15.94:0/3003053371 wait complete. 2024-04-25T16:51:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0395da640 1 Processor -- start 2024-04-25T16:51:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0395da640 1 -- start start 2024-04-25T16:51:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0395da640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe034070180 0x7fe03409ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0395da640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03415cf40 0x7fe034155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0395da640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe03415e070 0x7fe03415e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0395da640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe03416b500 con 0x7fe03415cf40 2024-04-25T16:51:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0395da640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0340afc10 con 0x7fe034070180 2024-04-25T16:51:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0395da640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe03416b680 con 0x7fe03415e070 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0327fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03415cf40 0x7fe034155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0327fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03415cf40 0x7fe034155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42468/0 (socket says 172.21.15.94:42468) 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0327fc640 1 -- 172.21.15.94:0/4168909414 learned_addr learned my addr 172.21.15.94:0/4168909414 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe032ffd640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe034070180 0x7fe03409ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0327fc640 1 -- 172.21.15.94:0/4168909414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe03415e070 msgr2=0x7fe03415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0327fc640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe03415e070 0x7fe03415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0327fc640 1 -- 172.21.15.94:0/4168909414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe034070180 msgr2=0x7fe03409ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0327fc640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe034070180 0x7fe03409ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0327fc640 1 -- 172.21.15.94:0/4168909414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03415e9c0 con 0x7fe03415cf40 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe032ffd640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe034070180 0x7fe03409ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0327fc640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03415cf40 0x7fe034155880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00c3d0 tx=0x7fe01c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe02b7fe640 1 -- 172.21.15.94:0/4168909414 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c017070 con 0x7fe03415cf40 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe02b7fe640 1 -- 172.21.15.94:0/4168909414 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe01c005cf0 con 0x7fe03415cf40 2024-04-25T16:51:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe02b7fe640 1 -- 172.21.15.94:0/4168909414 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0072d0 con 0x7fe03415cf40 2024-04-25T16:51:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0395da640 1 -- 172.21.15.94:0/4168909414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0340b1a10 con 0x7fe03415cf40 2024-04-25T16:51:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0395da640 1 -- 172.21.15.94:0/4168909414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0340b1cf0 con 0x7fe03415cf40 2024-04-25T16:51:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.284+0000 7fe0395da640 1 -- 172.21.15.94:0/4168909414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe034076830 con 0x7fe03415cf40 2024-04-25T16:51:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.288+0000 7fe02b7fe640 1 -- 172.21.15.94:0/4168909414 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe01c01a070 con 0x7fe03415cf40 2024-04-25T16:51:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.288+0000 7fe02b7fe640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe01406f430 0x7fe014071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.288+0000 7fe02b7fe640 1 -- 172.21.15.94:0/4168909414 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe01c08e780 con 0x7fe03415cf40 2024-04-25T16:51:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.288+0000 7fe032ffd640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe01406f430 0x7fe014071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.288+0000 7fe032ffd640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe01406f430 0x7fe014071940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe034071c30 tx=0x7fe024008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.292+0000 7fe02b7fe640 1 -- 172.21.15.94:0/4168909414 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe03415e9c0 con 0x7fe03415cf40 2024-04-25T16:51:24.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.436+0000 7fe0395da640 1 -- 172.21.15.94:0/4168909414 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe03406fa50 con 0x7fe01406f430 2024-04-25T16:51:24.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.456+0000 7fe02b7fe640 1 -- 172.21.15.94:0/4168909414 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332524 (secure 0 0 0) 0x7fe03406fa50 con 0x7fe01406f430 2024-04-25T16:51:24.456 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:51:24.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.460+0000 7fe0395da640 1 -- 172.21.15.94:0/4168909414 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe01406f430 msgr2=0x7fe014071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.460+0000 7fe0395da640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe01406f430 0x7fe014071940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe034071c30 tx=0x7fe024008040 comp rx=0 tx=0).stop 2024-04-25T16:51:24.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.460+0000 7fe0395da640 1 -- 172.21.15.94:0/4168909414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03415cf40 msgr2=0x7fe034155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.460+0000 7fe0395da640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03415cf40 0x7fe034155880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00c3d0 tx=0x7fe01c00c890 comp rx=0 tx=0).stop 2024-04-25T16:51:24.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.460+0000 7fe0395da640 1 -- 172.21.15.94:0/4168909414 shutdown_connections 2024-04-25T16:51:24.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.460+0000 7fe0395da640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe01406f430 0x7fe014071940 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.460+0000 7fe0395da640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe03415e070 0x7fe03415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.460+0000 7fe0395da640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03415cf40 0x7fe034155880 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.460+0000 7fe0395da640 1 --2- 172.21.15.94:0/4168909414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe034070180 0x7fe03409ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.460+0000 7fe0395da640 1 -- 172.21.15.94:0/4168909414 >> 172.21.15.94:0/4168909414 conn(0x7fe0340b47f0 msgr2=0x7fe03414f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:24.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.460+0000 7fe0395da640 1 -- 172.21.15.94:0/4168909414 shutdown_connections 2024-04-25T16:51:24.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.460+0000 7fe0395da640 1 -- 172.21.15.94:0/4168909414 wait complete. 2024-04-25T16:51:24.475 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-25T16:51:23.672042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240164,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734556,"statfs":{"total":773094113280,"available":772848185344,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005501"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018387+0000","last_change":"2024-04-25T16:50:54.047200+0000","last_active":"2024-04-25T16:51:01.018387+0000","last_peered":"2024-04-25T16:51:01.018387+0000","last_clean":"2024-04-25T16:51:01.018387+0000","last_became_active":"2024-04-25T16:50:54.046862+0000","last_became_peered":"2024-04-25T16:50:54.046862+0000","last_unstale":"2024-04-25T16:51:01.018387+0000","last_undegraded":"2024-04-25T16:51:01.018387+0000","last_fullsized":"2024-04-25T16:51:01.018387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.166309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311306+0000","last_change":"2024-04-25T16:50:55.060286+0000","last_active":"2024-04-25T16:51:00.311306+0000","last_peered":"2024-04-25T16:51:00.311306+0000","last_clean":"2024-04-25T16:51:00.311306+0000","last_became_active":"2024-04-25T16:50:55.059171+0000","last_became_peered":"2024-04-25T16:50:55.059171+0000","last_unstale":"2024-04-25T16:51:00.311306+0000","last_undegraded":"2024-04-25T16:51:00.311306+0000","last_fullsized":"2024-04-25T16:51:00.311306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:08.427466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310185+0000","last_change":"2024-04-25T16:50:59.181612+0000","last_active":"2024-04-25T16:51:00.310185+0000","last_peered":"2024-04-25T16:51:00.310185+0000","last_clean":"2024-04-25T16:51:00.310185+0000","last_became_active":"2024-04-25T16:50:59.180865+0000","last_became_peered":"2024-04-25T16:50:59.180865+0000","last_unstale":"2024-04-25T16:51:00.310185+0000","last_undegraded":"2024-04-25T16:51:00.310185+0000","last_fullsized":"2024-04-25T16:51:00.310185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:59.449855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311587+0000","last_change":"2024-04-25T16:50:54.040413+0000","last_active":"2024-04-25T16:51:00.311587+0000","last_peered":"2024-04-25T16:51:00.311587+0000","last_clean":"2024-04-25T16:51:00.311587+0000","last_became_active":"2024-04-25T16:50:54.040072+0000","last_became_peered":"2024-04-25T16:50:54.040072+0000","last_unstale":"2024-04-25T16:51:00.311587+0000","last_undegraded":"2024-04-25T16:51:00.311587+0000","last_fullsized":"2024-04-25T16:51:00.311587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:24.611706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018056+0000","last_change":"2024-04-25T16:50:55.083877+0000","last_active":"2024-04-25T16:51:01.018056+0000","last_peered":"2024-04-25T16:51:01.018056+0000","last_clean":"2024-04-25T16:51:01.018056+0000","last_became_active":"2024-04-25T16:50:55.083529+0000","last_became_peered":"2024-04-25T16:50:55.083529+0000","last_unstale":"2024-04-25T16:51:01.018056+0000","last_undegraded":"2024-04-25T16:51:01.018056+0000","last_fullsized":"2024-04-25T16:51:01.018056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152476+0000","last_change":"2024-04-25T16:50:59.143706+0000","last_active":"2024-04-25T16:51:00.152476+0000","last_peered":"2024-04-25T16:51:00.152476+0000","last_clean":"2024-04-25T16:51:00.152476+0000","last_became_active":"2024-04-25T16:50:59.142488+0000","last_became_peered":"2024-04-25T16:50:59.142488+0000","last_unstale":"2024-04-25T16:51:00.152476+0000","last_undegraded":"2024-04-25T16:51:00.152476+0000","last_fullsized":"2024-04-25T16:51:00.152476+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:20.203920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020308+0000","last_change":"2024-04-25T16:50:54.061175+0000","last_active":"2024-04-25T16:51:01.020308+0000","last_peered":"2024-04-25T16:51:01.020308+0000","last_clean":"2024-04-25T16:51:01.020308+0000","last_became_active":"2024-04-25T16:50:54.059991+0000","last_became_peered":"2024-04-25T16:50:54.059991+0000","last_unstale":"2024-04-25T16:51:01.020308+0000","last_undegraded":"2024-04-25T16:51:01.020308+0000","last_fullsized":"2024-04-25T16:51:01.020308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018692+0000","last_change":"2024-04-25T16:50:55.087495+0000","last_active":"2024-04-25T16:51:01.018692+0000","last_peered":"2024-04-25T16:51:01.018692+0000","last_clean":"2024-04-25T16:51:01.018692+0000","last_became_active":"2024-04-25T16:50:55.087177+0000","last_became_peered":"2024-04-25T16:50:55.087177+0000","last_unstale":"2024-04-25T16:51:01.018692+0000","last_undegraded":"2024-04-25T16:51:01.018692+0000","last_fullsized":"2024-04-25T16:51:01.018692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154110+0000","last_change":"2024-04-25T16:50:59.153145+0000","last_active":"2024-04-25T16:51:00.154110+0000","last_peered":"2024-04-25T16:51:00.154110+0000","last_clean":"2024-04-25T16:51:00.154110+0000","last_became_active":"2024-04-25T16:50:59.151443+0000","last_became_peered":"2024-04-25T16:50:59.151443+0000","last_unstale":"2024-04-25T16:51:00.154110+0000","last_undegraded":"2024-04-25T16:51:00.154110+0000","last_fullsized":"2024-04-25T16:51:00.154110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:08.117787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019564+0000","last_change":"2024-04-25T16:50:54.064208+0000","last_active":"2024-04-25T16:51:01.019564+0000","last_peered":"2024-04-25T16:51:01.019564+0000","last_clean":"2024-04-25T16:51:01.019564+0000","last_became_active":"2024-04-25T16:50:54.063734+0000","last_became_peered":"2024-04-25T16:50:54.063734+0000","last_unstale":"2024-04-25T16:51:01.019564+0000","last_undegraded":"2024-04-25T16:51:01.019564+0000","last_fullsized":"2024-04-25T16:51:01.019564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018436+0000","last_change":"2024-04-25T16:50:55.087482+0000","last_active":"2024-04-25T16:51:01.018436+0000","last_peered":"2024-04-25T16:51:01.018436+0000","last_clean":"2024-04-25T16:51:01.018436+0000","last_became_active":"2024-04-25T16:50:55.087139+0000","last_became_peered":"2024-04-25T16:50:55.087139+0000","last_unstale":"2024-04-25T16:51:01.018436+0000","last_undegraded":"2024-04-25T16:51:01.018436+0000","last_fullsized":"2024-04-25T16:51:01.018436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.304249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152921+0000","last_change":"2024-04-25T16:50:59.142658+0000","last_active":"2024-04-25T16:51:00.152921+0000","last_peered":"2024-04-25T16:51:00.152921+0000","last_clean":"2024-04-25T16:51:00.152921+0000","last_became_active":"2024-04-25T16:50:59.139343+0000","last_became_peered":"2024-04-25T16:50:59.139343+0000","last_unstale":"2024-04-25T16:51:00.152921+0000","last_undegraded":"2024-04-25T16:51:00.152921+0000","last_fullsized":"2024-04-25T16:51:00.152921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:45.390516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149327+0000","last_change":"2024-04-25T16:50:55.049255+0000","last_active":"2024-04-25T16:51:00.149327+0000","last_peered":"2024-04-25T16:51:00.149327+0000","last_clean":"2024-04-25T16:51:00.149327+0000","last_became_active":"2024-04-25T16:50:55.047635+0000","last_became_peered":"2024-04-25T16:50:55.047635+0000","last_unstale":"2024-04-25T16:51:00.149327+0000","last_undegraded":"2024-04-25T16:51:00.149327+0000","last_fullsized":"2024-04-25T16:51:00.149327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019735+0000","last_change":"2024-04-25T16:50:54.056721+0000","last_active":"2024-04-25T16:51:01.019735+0000","last_peered":"2024-04-25T16:51:01.019735+0000","last_clean":"2024-04-25T16:51:01.019735+0000","last_became_active":"2024-04-25T16:50:54.055845+0000","last_became_peered":"2024-04-25T16:50:54.055845+0000","last_unstale":"2024-04-25T16:51:01.019735+0000","last_undegraded":"2024-04-25T16:51:01.019735+0000","last_fullsized":"2024-04-25T16:51:01.019735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.077660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153494+0000","last_change":"2024-04-25T16:50:59.152269+0000","last_active":"2024-04-25T16:51:00.153494+0000","last_peered":"2024-04-25T16:51:00.153494+0000","last_clean":"2024-04-25T16:51:00.153494+0000","last_became_active":"2024-04-25T16:50:59.150778+0000","last_became_peered":"2024-04-25T16:50:59.150778+0000","last_unstale":"2024-04-25T16:51:00.153494+0000","last_undegraded":"2024-04-25T16:51:00.153494+0000","last_fullsized":"2024-04-25T16:51:00.153494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:12.513733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310585+0000","last_change":"2024-04-25T16:50:55.072126+0000","last_active":"2024-04-25T16:51:00.310585+0000","last_peered":"2024-04-25T16:51:00.310585+0000","last_clean":"2024-04-25T16:51:00.310585+0000","last_became_active":"2024-04-25T16:50:55.071191+0000","last_became_peered":"2024-04-25T16:50:55.071191+0000","last_unstale":"2024-04-25T16:51:00.310585+0000","last_undegraded":"2024-04-25T16:51:00.310585+0000","last_fullsized":"2024-04-25T16:51:00.310585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015930+0000","last_change":"2024-04-25T16:50:54.033554+0000","last_active":"2024-04-25T16:51:01.015930+0000","last_peered":"2024-04-25T16:51:01.015930+0000","last_clean":"2024-04-25T16:51:01.015930+0000","last_became_active":"2024-04-25T16:50:54.032467+0000","last_became_peered":"2024-04-25T16:50:54.032467+0000","last_unstale":"2024-04-25T16:51:01.015930+0000","last_undegraded":"2024-04-25T16:51:01.015930+0000","last_fullsized":"2024-04-25T16:51:01.015930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.077660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153884+0000","last_change":"2024-04-25T16:50:59.157126+0000","last_active":"2024-04-25T16:51:00.153884+0000","last_peered":"2024-04-25T16:51:00.153884+0000","last_clean":"2024-04-25T16:51:00.153884+0000","last_became_active":"2024-04-25T16:50:59.153352+0000","last_became_peered":"2024-04-25T16:50:59.153352+0000","last_unstale":"2024-04-25T16:51:00.153884+0000","last_undegraded":"2024-04-25T16:51:00.153884+0000","last_fullsized":"2024-04-25T16:51:00.153884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:18.428310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310685+0000","last_change":"2024-04-25T16:50:55.059831+0000","last_active":"2024-04-25T16:51:00.310685+0000","last_peered":"2024-04-25T16:51:00.310685+0000","last_clean":"2024-04-25T16:51:00.310685+0000","last_became_active":"2024-04-25T16:50:55.058958+0000","last_became_peered":"2024-04-25T16:50:55.058958+0000","last_unstale":"2024-04-25T16:51:00.310685+0000","last_undegraded":"2024-04-25T16:51:00.310685+0000","last_fullsized":"2024-04-25T16:51:00.310685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015662+0000","last_change":"2024-04-25T16:50:54.045690+0000","last_active":"2024-04-25T16:51:01.015662+0000","last_peered":"2024-04-25T16:51:01.015662+0000","last_clean":"2024-04-25T16:51:01.015662+0000","last_became_active":"2024-04-25T16:50:54.045057+0000","last_became_peered":"2024-04-25T16:50:54.045057+0000","last_unstale":"2024-04-25T16:51:01.015662+0000","last_undegraded":"2024-04-25T16:51:01.015662+0000","last_fullsized":"2024-04-25T16:51:01.015662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:27.893510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016104+0000","last_change":"2024-04-25T16:50:59.163126+0000","last_active":"2024-04-25T16:51:01.016104+0000","last_peered":"2024-04-25T16:51:01.016104+0000","last_clean":"2024-04-25T16:51:01.016104+0000","last_became_active":"2024-04-25T16:50:59.162606+0000","last_became_peered":"2024-04-25T16:50:59.162606+0000","last_unstale":"2024-04-25T16:51:01.016104+0000","last_undegraded":"2024-04-25T16:51:01.016104+0000","last_fullsized":"2024-04-25T16:51:01.016104+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.992576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153472+0000","last_change":"2024-04-25T16:50:55.079857+0000","last_active":"2024-04-25T16:51:00.153472+0000","last_peered":"2024-04-25T16:51:00.153472+0000","last_clean":"2024-04-25T16:51:00.153472+0000","last_became_active":"2024-04-25T16:50:55.070021+0000","last_became_peered":"2024-04-25T16:50:55.070021+0000","last_unstale":"2024-04-25T16:51:00.153472+0000","last_undegraded":"2024-04-25T16:51:00.153472+0000","last_fullsized":"2024-04-25T16:51:00.153472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:00.753427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017068+0000","last_change":"2024-04-25T16:50:54.050226+0000","last_active":"2024-04-25T16:51:01.017068+0000","last_peered":"2024-04-25T16:51:01.017068+0000","last_clean":"2024-04-25T16:51:01.017068+0000","last_became_active":"2024-04-25T16:50:54.049451+0000","last_became_peered":"2024-04-25T16:50:54.049451+0000","last_unstale":"2024-04-25T16:51:01.017068+0000","last_undegraded":"2024-04-25T16:51:01.017068+0000","last_fullsized":"2024-04-25T16:51:01.017068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:26.942742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154683+0000","last_change":"2024-04-25T16:50:59.152071+0000","last_active":"2024-04-25T16:51:00.154683+0000","last_peered":"2024-04-25T16:51:00.154683+0000","last_clean":"2024-04-25T16:51:00.154683+0000","last_became_active":"2024-04-25T16:50:59.149014+0000","last_became_peered":"2024-04-25T16:50:59.149014+0000","last_unstale":"2024-04-25T16:51:00.154683+0000","last_undegraded":"2024-04-25T16:51:00.154683+0000","last_fullsized":"2024-04-25T16:51:00.154683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:00.447099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018096+0000","last_change":"2024-04-25T16:50:55.057085+0000","last_active":"2024-04-25T16:51:01.018096+0000","last_peered":"2024-04-25T16:51:01.018096+0000","last_clean":"2024-04-25T16:51:01.018096+0000","last_became_active":"2024-04-25T16:50:55.056254+0000","last_became_peered":"2024-04-25T16:50:55.056254+0000","last_unstale":"2024-04-25T16:51:01.018096+0000","last_undegraded":"2024-04-25T16:51:01.018096+0000","last_fullsized":"2024-04-25T16:51:01.018096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151405+0000","last_change":"2024-04-25T16:50:54.047574+0000","last_active":"2024-04-25T16:51:00.151405+0000","last_peered":"2024-04-25T16:51:00.151405+0000","last_clean":"2024-04-25T16:51:00.151405+0000","last_became_active":"2024-04-25T16:50:54.047164+0000","last_became_peered":"2024-04-25T16:50:54.047164+0000","last_unstale":"2024-04-25T16:51:00.151405+0000","last_undegraded":"2024-04-25T16:51:00.151405+0000","last_fullsized":"2024-04-25T16:51:00.151405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152691+0000","last_change":"2024-04-25T16:50:59.181342+0000","last_active":"2024-04-25T16:51:00.152691+0000","last_peered":"2024-04-25T16:51:00.152691+0000","last_clean":"2024-04-25T16:51:00.152691+0000","last_became_active":"2024-04-25T16:50:59.180010+0000","last_became_peered":"2024-04-25T16:50:59.180010+0000","last_unstale":"2024-04-25T16:51:00.152691+0000","last_undegraded":"2024-04-25T16:51:00.152691+0000","last_fullsized":"2024-04-25T16:51:00.152691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:09.707352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.312597+0000","last_change":"2024-04-25T16:50:55.093718+0000","last_active":"2024-04-25T16:51:00.312597+0000","last_peered":"2024-04-25T16:51:00.312597+0000","last_clean":"2024-04-25T16:51:00.312597+0000","last_became_active":"2024-04-25T16:50:55.093376+0000","last_became_peered":"2024-04-25T16:50:55.093376+0000","last_unstale":"2024-04-25T16:51:00.312597+0000","last_undegraded":"2024-04-25T16:51:00.312597+0000","last_fullsized":"2024-04-25T16:51:00.312597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:04.031520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016325+0000","last_change":"2024-04-25T16:50:54.040703+0000","last_active":"2024-04-25T16:51:01.016325+0000","last_peered":"2024-04-25T16:51:01.016325+0000","last_clean":"2024-04-25T16:51:01.016325+0000","last_became_active":"2024-04-25T16:50:54.039961+0000","last_became_peered":"2024-04-25T16:50:54.039961+0000","last_unstale":"2024-04-25T16:51:01.016325+0000","last_undegraded":"2024-04-25T16:51:01.016325+0000","last_fullsized":"2024-04-25T16:51:01.016325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310069+0000","last_change":"2024-04-25T16:50:59.161678+0000","last_active":"2024-04-25T16:51:00.310069+0000","last_peered":"2024-04-25T16:51:00.310069+0000","last_clean":"2024-04-25T16:51:00.310069+0000","last_became_active":"2024-04-25T16:50:59.161064+0000","last_became_peered":"2024-04-25T16:50:59.161064+0000","last_unstale":"2024-04-25T16:51:00.310069+0000","last_undegraded":"2024-04-25T16:51:00.310069+0000","last_fullsized":"2024-04-25T16:51:00.310069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:19.854569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015648+0000","last_change":"2024-04-25T16:50:55.069738+0000","last_active":"2024-04-25T16:51:01.015648+0000","last_peered":"2024-04-25T16:51:01.015648+0000","last_clean":"2024-04-25T16:51:01.015648+0000","last_became_active":"2024-04-25T16:50:55.068864+0000","last_became_peered":"2024-04-25T16:50:55.068864+0000","last_unstale":"2024-04-25T16:51:01.015648+0000","last_undegraded":"2024-04-25T16:51:01.015648+0000","last_fullsized":"2024-04-25T16:51:01.015648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020503+0000","last_change":"2024-04-25T16:50:54.061758+0000","last_active":"2024-04-25T16:51:01.020503+0000","last_peered":"2024-04-25T16:51:01.020503+0000","last_clean":"2024-04-25T16:51:01.020503+0000","last_became_active":"2024-04-25T16:50:54.061267+0000","last_became_peered":"2024-04-25T16:50:54.061267+0000","last_unstale":"2024-04-25T16:51:01.020503+0000","last_undegraded":"2024-04-25T16:51:01.020503+0000","last_fullsized":"2024-04-25T16:51:01.020503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:21.043343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017543+0000","last_change":"2024-04-25T16:50:59.174520+0000","last_active":"2024-04-25T16:51:01.017543+0000","last_peered":"2024-04-25T16:51:01.017543+0000","last_clean":"2024-04-25T16:51:01.017543+0000","last_became_active":"2024-04-25T16:50:59.173812+0000","last_became_peered":"2024-04-25T16:50:59.173812+0000","last_unstale":"2024-04-25T16:51:01.017543+0000","last_undegraded":"2024-04-25T16:51:01.017543+0000","last_fullsized":"2024-04-25T16:51:01.017543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:29.939373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018748+0000","last_change":"2024-04-25T16:50:55.090616+0000","last_active":"2024-04-25T16:51:01.018748+0000","last_peered":"2024-04-25T16:51:01.018748+0000","last_clean":"2024-04-25T16:51:01.018748+0000","last_became_active":"2024-04-25T16:50:55.090287+0000","last_became_peered":"2024-04-25T16:50:55.090287+0000","last_unstale":"2024-04-25T16:51:01.018748+0000","last_undegraded":"2024-04-25T16:51:01.018748+0000","last_fullsized":"2024-04-25T16:51:01.018748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:04.031520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015756+0000","last_change":"2024-04-25T16:50:54.023906+0000","last_active":"2024-04-25T16:51:01.015756+0000","last_peered":"2024-04-25T16:51:01.015756+0000","last_clean":"2024-04-25T16:51:01.015756+0000","last_became_active":"2024-04-25T16:50:54.022198+0000","last_became_peered":"2024-04-25T16:50:54.022198+0000","last_unstale":"2024-04-25T16:51:01.015756+0000","last_undegraded":"2024-04-25T16:51:01.015756+0000","last_fullsized":"2024-04-25T16:51:01.015756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015375+0000","last_change":"2024-04-25T16:50:59.153325+0000","last_active":"2024-04-25T16:51:01.015375+0000","last_peered":"2024-04-25T16:51:01.015375+0000","last_clean":"2024-04-25T16:51:01.015375+0000","last_became_active":"2024-04-25T16:50:59.152431+0000","last_became_peered":"2024-04-25T16:50:59.152431+0000","last_unstale":"2024-04-25T16:51:01.015375+0000","last_undegraded":"2024-04-25T16:51:01.015375+0000","last_fullsized":"2024-04-25T16:51:01.015375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.819554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154818+0000","last_change":"2024-04-25T16:50:55.058098+0000","last_active":"2024-04-25T16:51:00.154818+0000","last_peered":"2024-04-25T16:51:00.154818+0000","last_clean":"2024-04-25T16:51:00.154818+0000","last_became_active":"2024-04-25T16:50:55.057233+0000","last_became_peered":"2024-04-25T16:50:55.057233+0000","last_unstale":"2024-04-25T16:51:00.154818+0000","last_undegraded":"2024-04-25T16:51:00.154818+0000","last_fullsized":"2024-04-25T16:51:00.154818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:36.088026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019426+0000","last_change":"2024-04-25T16:50:54.059732+0000","last_active":"2024-04-25T16:51:01.019426+0000","last_peered":"2024-04-25T16:51:01.019426+0000","last_clean":"2024-04-25T16:51:01.019426+0000","last_became_active":"2024-04-25T16:50:54.058805+0000","last_became_peered":"2024-04-25T16:50:54.058805+0000","last_unstale":"2024-04-25T16:51:01.019426+0000","last_undegraded":"2024-04-25T16:51:01.019426+0000","last_fullsized":"2024-04-25T16:51:01.019426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015551+0000","last_change":"2024-04-25T16:50:59.157551+0000","last_active":"2024-04-25T16:51:01.015551+0000","last_peered":"2024-04-25T16:51:01.015551+0000","last_clean":"2024-04-25T16:51:01.015551+0000","last_became_active":"2024-04-25T16:50:59.156077+0000","last_became_peered":"2024-04-25T16:50:59.156077+0000","last_unstale":"2024-04-25T16:51:01.015551+0000","last_undegraded":"2024-04-25T16:51:01.015551+0000","last_fullsized":"2024-04-25T16:51:01.015551+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:29.939373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015015+0000","last_change":"2024-04-25T16:50:55.072516+0000","last_active":"2024-04-25T16:51:01.015015+0000","last_peered":"2024-04-25T16:51:01.015015+0000","last_clean":"2024-04-25T16:51:01.015015+0000","last_became_active":"2024-04-25T16:50:55.071677+0000","last_became_peered":"2024-04-25T16:50:55.071677+0000","last_unstale":"2024-04-25T16:51:01.015015+0000","last_undegraded":"2024-04-25T16:51:01.015015+0000","last_fullsized":"2024-04-25T16:51:01.015015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148828+0000","last_change":"2024-04-25T16:50:54.026443+0000","last_active":"2024-04-25T16:51:00.148828+0000","last_peered":"2024-04-25T16:51:00.148828+0000","last_clean":"2024-04-25T16:51:00.148828+0000","last_became_active":"2024-04-25T16:50:54.024224+0000","last_became_peered":"2024-04-25T16:50:54.024224+0000","last_unstale":"2024-04-25T16:51:00.148828+0000","last_undegraded":"2024-04-25T16:51:00.148828+0000","last_fullsized":"2024-04-25T16:51:00.148828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016170+0000","last_change":"2024-04-25T16:50:59.149300+0000","last_active":"2024-04-25T16:51:01.016170+0000","last_peered":"2024-04-25T16:51:01.016170+0000","last_clean":"2024-04-25T16:51:01.016170+0000","last_became_active":"2024-04-25T16:50:59.147799+0000","last_became_peered":"2024-04-25T16:50:59.147799+0000","last_unstale":"2024-04-25T16:51:01.016170+0000","last_undegraded":"2024-04-25T16:51:01.016170+0000","last_fullsized":"2024-04-25T16:51:01.016170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:05:04.839576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016900+0000","last_change":"2024-04-25T16:50:55.048595+0000","last_active":"2024-04-25T16:51:01.016900+0000","last_peered":"2024-04-25T16:51:01.016900+0000","last_clean":"2024-04-25T16:51:01.016900+0000","last_became_active":"2024-04-25T16:50:55.047706+0000","last_became_peered":"2024-04-25T16:50:55.047706+0000","last_unstale":"2024-04-25T16:51:01.016900+0000","last_undegraded":"2024-04-25T16:51:01.016900+0000","last_fullsized":"2024-04-25T16:51:01.016900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:56.480058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017350+0000","last_change":"2024-04-25T16:50:54.053397+0000","last_active":"2024-04-25T16:51:01.017350+0000","last_peered":"2024-04-25T16:51:01.017350+0000","last_clean":"2024-04-25T16:51:01.017350+0000","last_became_active":"2024-04-25T16:50:54.052633+0000","last_became_peered":"2024-04-25T16:50:54.052633+0000","last_unstale":"2024-04-25T16:51:01.017350+0000","last_undegraded":"2024-04-25T16:51:01.017350+0000","last_fullsized":"2024-04-25T16:51:01.017350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018189+0000","last_change":"2024-04-25T16:50:59.162703+0000","last_active":"2024-04-25T16:51:01.018189+0000","last_peered":"2024-04-25T16:51:01.018189+0000","last_clean":"2024-04-25T16:51:01.018189+0000","last_became_active":"2024-04-25T16:50:59.161980+0000","last_became_peered":"2024-04-25T16:50:59.161980+0000","last_unstale":"2024-04-25T16:51:01.018189+0000","last_undegraded":"2024-04-25T16:51:01.018189+0000","last_fullsized":"2024-04-25T16:51:01.018189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:05:04.839576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310673+0000","last_change":"2024-04-25T16:50:55.090720+0000","last_active":"2024-04-25T16:51:00.310673+0000","last_peered":"2024-04-25T16:51:00.310673+0000","last_clean":"2024-04-25T16:51:00.310673+0000","last_became_active":"2024-04-25T16:50:55.090360+0000","last_became_peered":"2024-04-25T16:50:55.090360+0000","last_unstale":"2024-04-25T16:51:00.310673+0000","last_undegraded":"2024-04-25T16:51:00.310673+0000","last_fullsized":"2024-04-25T16:51:00.310673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018278+0000","last_change":"2024-04-25T16:50:54.050879+0000","last_active":"2024-04-25T16:51:01.018278+0000","last_peered":"2024-04-25T16:51:01.018278+0000","last_clean":"2024-04-25T16:51:01.018278+0000","last_became_active":"2024-04-25T16:50:54.049784+0000","last_became_peered":"2024-04-25T16:50:54.049784+0000","last_unstale":"2024-04-25T16:51:01.018278+0000","last_undegraded":"2024-04-25T16:51:01.018278+0000","last_fullsized":"2024-04-25T16:51:01.018278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":118,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017460+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:51:01.017460+0000","last_peered":"2024-04-25T16:51:01.017460+0000","last_clean":"2024-04-25T16:51:01.017460+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:51:01.017460+0000","last_undegraded":"2024-04-25T16:51:01.017460+0000","last_fullsized":"2024-04-25T16:51:01.017460+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150640+0000","last_change":"2024-04-25T16:50:59.175343+0000","last_active":"2024-04-25T16:51:00.150640+0000","last_peered":"2024-04-25T16:51:00.150640+0000","last_clean":"2024-04-25T16:51:00.150640+0000","last_became_active":"2024-04-25T16:50:59.173979+0000","last_became_peered":"2024-04-25T16:50:59.173979+0000","last_unstale":"2024-04-25T16:51:00.150640+0000","last_undegraded":"2024-04-25T16:51:00.150640+0000","last_fullsized":"2024-04-25T16:51:00.150640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:07.790324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019964+0000","last_change":"2024-04-25T16:50:55.092868+0000","last_active":"2024-04-25T16:51:01.019964+0000","last_peered":"2024-04-25T16:51:01.019964+0000","last_clean":"2024-04-25T16:51:01.019964+0000","last_became_active":"2024-04-25T16:50:55.092569+0000","last_became_peered":"2024-04-25T16:50:55.092569+0000","last_unstale":"2024-04-25T16:51:01.019964+0000","last_undegraded":"2024-04-25T16:51:01.019964+0000","last_fullsized":"2024-04-25T16:51:01.019964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:03.704057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149644+0000","last_change":"2024-04-25T16:50:54.056537+0000","last_active":"2024-04-25T16:51:00.149644+0000","last_peered":"2024-04-25T16:51:00.149644+0000","last_clean":"2024-04-25T16:51:00.149644+0000","last_became_active":"2024-04-25T16:50:54.056219+0000","last_became_peered":"2024-04-25T16:50:54.056219+0000","last_unstale":"2024-04-25T16:51:00.149644+0000","last_undegraded":"2024-04-25T16:51:00.149644+0000","last_fullsized":"2024-04-25T16:51:00.149644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149259+0000","last_change":"2024-04-25T16:50:59.168724+0000","last_active":"2024-04-25T16:51:00.149259+0000","last_peered":"2024-04-25T16:51:00.149259+0000","last_clean":"2024-04-25T16:51:00.149259+0000","last_became_active":"2024-04-25T16:50:59.167823+0000","last_became_peered":"2024-04-25T16:50:59.167823+0000","last_unstale":"2024-04-25T16:51:00.149259+0000","last_undegraded":"2024-04-25T16:51:00.149259+0000","last_fullsized":"2024-04-25T16:51:00.149259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:00.447099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018846+0000","last_change":"2024-04-25T16:50:55.091727+0000","last_active":"2024-04-25T16:51:01.018846+0000","last_peered":"2024-04-25T16:51:01.018846+0000","last_clean":"2024-04-25T16:51:01.018846+0000","last_became_active":"2024-04-25T16:50:55.091409+0000","last_became_peered":"2024-04-25T16:50:55.091409+0000","last_unstale":"2024-04-25T16:51:01.018846+0000","last_undegraded":"2024-04-25T16:51:01.018846+0000","last_fullsized":"2024-04-25T16:51:01.018846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:14.342043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015818+0000","last_change":"2024-04-25T16:50:54.051928+0000","last_active":"2024-04-25T16:51:01.015818+0000","last_peered":"2024-04-25T16:51:01.015818+0000","last_clean":"2024-04-25T16:51:01.015818+0000","last_became_active":"2024-04-25T16:50:54.045379+0000","last_became_peered":"2024-04-25T16:50:54.045379+0000","last_unstale":"2024-04-25T16:51:01.015818+0000","last_undegraded":"2024-04-25T16:51:01.015818+0000","last_fullsized":"2024-04-25T16:51:01.015818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017284+0000","last_change":"2024-04-25T16:50:59.149171+0000","last_active":"2024-04-25T16:51:01.017284+0000","last_peered":"2024-04-25T16:51:01.017284+0000","last_clean":"2024-04-25T16:51:01.017284+0000","last_became_active":"2024-04-25T16:50:59.148753+0000","last_became_peered":"2024-04-25T16:50:59.148753+0000","last_unstale":"2024-04-25T16:51:01.017284+0000","last_undegraded":"2024-04-25T16:51:01.017284+0000","last_fullsized":"2024-04-25T16:51:01.017284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.819554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.021346+0000","last_change":"2024-04-25T16:50:55.093340+0000","last_active":"2024-04-25T16:51:01.021346+0000","last_peered":"2024-04-25T16:51:01.021346+0000","last_clean":"2024-04-25T16:51:01.021346+0000","last_became_active":"2024-04-25T16:50:55.092870+0000","last_became_peered":"2024-04-25T16:50:55.092870+0000","last_unstale":"2024-04-25T16:51:01.021346+0000","last_undegraded":"2024-04-25T16:51:01.021346+0000","last_fullsized":"2024-04-25T16:51:01.021346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:25.853106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150507+0000","last_change":"2024-04-25T16:50:54.041300+0000","last_active":"2024-04-25T16:51:00.150507+0000","last_peered":"2024-04-25T16:51:00.150507+0000","last_clean":"2024-04-25T16:51:00.150507+0000","last_became_active":"2024-04-25T16:50:54.040987+0000","last_became_peered":"2024-04-25T16:50:54.040987+0000","last_unstale":"2024-04-25T16:51:00.150507+0000","last_undegraded":"2024-04-25T16:51:00.150507+0000","last_fullsized":"2024-04-25T16:51:00.150507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016995+0000","last_change":"2024-04-25T16:50:59.180254+0000","last_active":"2024-04-25T16:51:01.016995+0000","last_peered":"2024-04-25T16:51:01.016995+0000","last_clean":"2024-04-25T16:51:01.016995+0000","last_became_active":"2024-04-25T16:50:59.179432+0000","last_became_peered":"2024-04-25T16:50:59.179432+0000","last_unstale":"2024-04-25T16:51:01.016995+0000","last_undegraded":"2024-04-25T16:51:01.016995+0000","last_fullsized":"2024-04-25T16:51:01.016995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:44.752448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020598+0000","last_change":"2024-04-25T16:50:55.085874+0000","last_active":"2024-04-25T16:51:01.020598+0000","last_peered":"2024-04-25T16:51:01.020598+0000","last_clean":"2024-04-25T16:51:01.020598+0000","last_became_active":"2024-04-25T16:50:55.085347+0000","last_became_peered":"2024-04-25T16:50:55.085347+0000","last_unstale":"2024-04-25T16:51:01.020598+0000","last_undegraded":"2024-04-25T16:51:01.020598+0000","last_fullsized":"2024-04-25T16:51:01.020598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:14.906309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152960+0000","last_change":"2024-04-25T16:50:54.037711+0000","last_active":"2024-04-25T16:51:00.152960+0000","last_peered":"2024-04-25T16:51:00.152960+0000","last_clean":"2024-04-25T16:51:00.152960+0000","last_became_active":"2024-04-25T16:50:54.036847+0000","last_became_peered":"2024-04-25T16:50:54.036847+0000","last_unstale":"2024-04-25T16:51:00.152960+0000","last_undegraded":"2024-04-25T16:51:00.152960+0000","last_fullsized":"2024-04-25T16:51:00.152960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152479+0000","last_change":"2024-04-25T16:50:59.165076+0000","last_active":"2024-04-25T16:51:00.152479+0000","last_peered":"2024-04-25T16:51:00.152479+0000","last_clean":"2024-04-25T16:51:00.152479+0000","last_became_active":"2024-04-25T16:50:59.159200+0000","last_became_peered":"2024-04-25T16:50:59.159200+0000","last_unstale":"2024-04-25T16:51:00.152479+0000","last_undegraded":"2024-04-25T16:51:00.152479+0000","last_fullsized":"2024-04-25T16:51:00.152479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:11.722626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017585+0000","last_change":"2024-04-25T16:50:55.085792+0000","last_active":"2024-04-25T16:51:01.017585+0000","last_peered":"2024-04-25T16:51:01.017585+0000","last_clean":"2024-04-25T16:51:01.017585+0000","last_became_active":"2024-04-25T16:50:55.085212+0000","last_became_peered":"2024-04-25T16:50:55.085212+0000","last_unstale":"2024-04-25T16:51:01.017585+0000","last_undegraded":"2024-04-25T16:51:01.017585+0000","last_fullsized":"2024-04-25T16:51:01.017585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:08.427466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148299+0000","last_change":"2024-04-25T16:50:54.021199+0000","last_active":"2024-04-25T16:51:00.148299+0000","last_peered":"2024-04-25T16:51:00.148299+0000","last_clean":"2024-04-25T16:51:00.148299+0000","last_became_active":"2024-04-25T16:50:54.020716+0000","last_became_peered":"2024-04-25T16:50:54.020716+0000","last_unstale":"2024-04-25T16:51:00.148299+0000","last_undegraded":"2024-04-25T16:51:00.148299+0000","last_fullsized":"2024-04-25T16:51:00.148299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:27.893510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017880+0000","last_change":"2024-04-25T16:50:59.160368+0000","last_active":"2024-04-25T16:51:01.017880+0000","last_peered":"2024-04-25T16:51:01.017880+0000","last_clean":"2024-04-25T16:51:01.017880+0000","last_became_active":"2024-04-25T16:50:59.159232+0000","last_became_peered":"2024-04-25T16:50:59.159232+0000","last_unstale":"2024-04-25T16:51:01.017880+0000","last_undegraded":"2024-04-25T16:51:01.017880+0000","last_fullsized":"2024-04-25T16:51:01.017880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.992576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148948+0000","last_change":"2024-04-25T16:50:55.059427+0000","last_active":"2024-04-25T16:51:00.148948+0000","last_peered":"2024-04-25T16:51:00.148948+0000","last_clean":"2024-04-25T16:51:00.148948+0000","last_became_active":"2024-04-25T16:50:55.058611+0000","last_became_peered":"2024-04-25T16:50:55.058611+0000","last_unstale":"2024-04-25T16:51:00.148948+0000","last_undegraded":"2024-04-25T16:51:00.148948+0000","last_fullsized":"2024-04-25T16:51:00.148948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149289+0000","last_change":"2024-04-25T16:50:54.044858+0000","last_active":"2024-04-25T16:51:00.149289+0000","last_peered":"2024-04-25T16:51:00.149289+0000","last_clean":"2024-04-25T16:51:00.149289+0000","last_became_active":"2024-04-25T16:50:54.043345+0000","last_became_peered":"2024-04-25T16:50:54.043345+0000","last_unstale":"2024-04-25T16:51:00.149289+0000","last_undegraded":"2024-04-25T16:51:00.149289+0000","last_fullsized":"2024-04-25T16:51:00.149289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.312170+0000","last_change":"2024-04-25T16:50:59.173258+0000","last_active":"2024-04-25T16:51:00.312170+0000","last_peered":"2024-04-25T16:51:00.312170+0000","last_clean":"2024-04-25T16:51:00.312170+0000","last_became_active":"2024-04-25T16:50:59.172637+0000","last_became_peered":"2024-04-25T16:50:59.172637+0000","last_unstale":"2024-04-25T16:51:00.312170+0000","last_undegraded":"2024-04-25T16:51:00.312170+0000","last_fullsized":"2024-04-25T16:51:00.312170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:46.618416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019246+0000","last_change":"2024-04-25T16:50:55.085682+0000","last_active":"2024-04-25T16:51:01.019246+0000","last_peered":"2024-04-25T16:51:01.019246+0000","last_clean":"2024-04-25T16:51:01.019246+0000","last_became_active":"2024-04-25T16:50:55.084855+0000","last_became_peered":"2024-04-25T16:50:55.084855+0000","last_unstale":"2024-04-25T16:51:01.019246+0000","last_undegraded":"2024-04-25T16:51:01.019246+0000","last_fullsized":"2024-04-25T16:51:01.019246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:42.081728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019411+0000","last_change":"2024-04-25T16:50:54.059696+0000","last_active":"2024-04-25T16:51:01.019411+0000","last_peered":"2024-04-25T16:51:01.019411+0000","last_clean":"2024-04-25T16:51:01.019411+0000","last_became_active":"2024-04-25T16:50:54.058979+0000","last_became_peered":"2024-04-25T16:50:54.058979+0000","last_unstale":"2024-04-25T16:51:01.019411+0000","last_undegraded":"2024-04-25T16:51:01.019411+0000","last_fullsized":"2024-04-25T16:51:01.019411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311564+0000","last_change":"2024-04-25T16:50:59.163291+0000","last_active":"2024-04-25T16:51:00.311564+0000","last_peered":"2024-04-25T16:51:00.311564+0000","last_clean":"2024-04-25T16:51:00.311564+0000","last_became_active":"2024-04-25T16:50:59.162316+0000","last_became_peered":"2024-04-25T16:50:59.162316+0000","last_unstale":"2024-04-25T16:51:00.311564+0000","last_undegraded":"2024-04-25T16:51:00.311564+0000","last_fullsized":"2024-04-25T16:51:00.311564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:41.776182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018471+0000","last_change":"2024-04-25T16:50:55.067672+0000","last_active":"2024-04-25T16:51:01.018471+0000","last_peered":"2024-04-25T16:51:01.018471+0000","last_clean":"2024-04-25T16:51:01.018471+0000","last_became_active":"2024-04-25T16:50:55.066870+0000","last_became_peered":"2024-04-25T16:50:55.066870+0000","last_unstale":"2024-04-25T16:51:01.018471+0000","last_undegraded":"2024-04-25T16:51:01.018471+0000","last_fullsized":"2024-04-25T16:51:01.018471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:28:31.672027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017648+0000","last_change":"2024-04-25T16:50:54.036862+0000","last_active":"2024-04-25T16:51:01.017648+0000","last_peered":"2024-04-25T16:51:01.017648+0000","last_clean":"2024-04-25T16:51:01.017648+0000","last_became_active":"2024-04-25T16:50:54.036549+0000","last_became_peered":"2024-04-25T16:50:54.036549+0000","last_unstale":"2024-04-25T16:51:01.017648+0000","last_undegraded":"2024-04-25T16:51:01.017648+0000","last_fullsized":"2024-04-25T16:51:01.017648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:24.611706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018388+0000","last_change":"2024-04-25T16:50:59.175285+0000","last_active":"2024-04-25T16:51:01.018388+0000","last_peered":"2024-04-25T16:51:01.018388+0000","last_clean":"2024-04-25T16:51:01.018388+0000","last_became_active":"2024-04-25T16:50:59.174301+0000","last_became_peered":"2024-04-25T16:50:59.174301+0000","last_unstale":"2024-04-25T16:51:01.018388+0000","last_undegraded":"2024-04-25T16:51:01.018388+0000","last_fullsized":"2024-04-25T16:51:01.018388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:25.742371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019931+0000","last_change":"2024-04-25T16:50:55.076173+0000","last_active":"2024-04-25T16:51:01.019931+0000","last_peered":"2024-04-25T16:51:01.019931+0000","last_clean":"2024-04-25T16:51:01.019931+0000","last_became_active":"2024-04-25T16:50:55.075360+0000","last_became_peered":"2024-04-25T16:50:55.075360+0000","last_unstale":"2024-04-25T16:51:01.019931+0000","last_undegraded":"2024-04-25T16:51:01.019931+0000","last_fullsized":"2024-04-25T16:51:01.019931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:04.031520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:10.281685+0000","last_change":"2024-04-25T16:50:54.035115+0000","last_active":"2024-04-25T16:51:10.281685+0000","last_peered":"2024-04-25T16:51:10.281685+0000","last_clean":"2024-04-25T16:51:10.281685+0000","last_became_active":"2024-04-25T16:50:54.034629+0000","last_became_peered":"2024-04-25T16:50:54.034629+0000","last_unstale":"2024-04-25T16:51:10.281685+0000","last_undegraded":"2024-04-25T16:51:10.281685+0000","last_fullsized":"2024-04-25T16:51:10.281685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.077660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015930+0000","last_change":"2024-04-25T16:50:59.159337+0000","last_active":"2024-04-25T16:51:01.015930+0000","last_peered":"2024-04-25T16:51:01.015930+0000","last_clean":"2024-04-25T16:51:01.015930+0000","last_became_active":"2024-04-25T16:50:59.156207+0000","last_became_peered":"2024-04-25T16:50:59.156207+0000","last_unstale":"2024-04-25T16:51:01.015930+0000","last_undegraded":"2024-04-25T16:51:01.015930+0000","last_fullsized":"2024-04-25T16:51:01.015930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:39.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154043+0000","last_change":"2024-04-25T16:50:55.066237+0000","last_active":"2024-04-25T16:51:00.154043+0000","last_peered":"2024-04-25T16:51:00.154043+0000","last_clean":"2024-04-25T16:51:00.154043+0000","last_became_active":"2024-04-25T16:50:55.063200+0000","last_became_peered":"2024-04-25T16:50:55.063200+0000","last_unstale":"2024-04-25T16:51:00.154043+0000","last_undegraded":"2024-04-25T16:51:00.154043+0000","last_fullsized":"2024-04-25T16:51:00.154043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:22.053709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015709+0000","last_change":"2024-04-25T16:50:54.055594+0000","last_active":"2024-04-25T16:51:01.015709+0000","last_peered":"2024-04-25T16:51:01.015709+0000","last_clean":"2024-04-25T16:51:01.015709+0000","last_became_active":"2024-04-25T16:50:54.055027+0000","last_became_peered":"2024-04-25T16:50:54.055027+0000","last_unstale":"2024-04-25T16:51:01.015709+0000","last_undegraded":"2024-04-25T16:51:01.015709+0000","last_fullsized":"2024-04-25T16:51:01.015709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154662+0000","last_change":"2024-04-25T16:50:59.160116+0000","last_active":"2024-04-25T16:51:00.154662+0000","last_peered":"2024-04-25T16:51:00.154662+0000","last_clean":"2024-04-25T16:51:00.154662+0000","last_became_active":"2024-04-25T16:50:59.157923+0000","last_became_peered":"2024-04-25T16:50:59.157923+0000","last_unstale":"2024-04-25T16:51:00.154662+0000","last_undegraded":"2024-04-25T16:51:00.154662+0000","last_fullsized":"2024-04-25T16:51:00.154662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:46.167995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018271+0000","last_change":"2024-04-25T16:50:55.083898+0000","last_active":"2024-04-25T16:51:01.018271+0000","last_peered":"2024-04-25T16:51:01.018271+0000","last_clean":"2024-04-25T16:51:01.018271+0000","last_became_active":"2024-04-25T16:50:55.083533+0000","last_became_peered":"2024-04-25T16:50:55.083533+0000","last_unstale":"2024-04-25T16:51:01.018271+0000","last_undegraded":"2024-04-25T16:51:01.018271+0000","last_fullsized":"2024-04-25T16:51:01.018271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019088+0000","last_change":"2024-04-25T16:50:54.056906+0000","last_active":"2024-04-25T16:51:01.019088+0000","last_peered":"2024-04-25T16:51:01.019088+0000","last_clean":"2024-04-25T16:51:01.019088+0000","last_became_active":"2024-04-25T16:50:54.056102+0000","last_became_peered":"2024-04-25T16:50:54.056102+0000","last_unstale":"2024-04-25T16:51:01.019088+0000","last_undegraded":"2024-04-25T16:51:01.019088+0000","last_fullsized":"2024-04-25T16:51:01.019088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310494+0000","last_change":"2024-04-25T16:50:59.129868+0000","last_active":"2024-04-25T16:51:00.310494+0000","last_peered":"2024-04-25T16:51:00.310494+0000","last_clean":"2024-04-25T16:51:00.310494+0000","last_became_active":"2024-04-25T16:50:59.129447+0000","last_became_peered":"2024-04-25T16:50:59.129447+0000","last_unstale":"2024-04-25T16:51:00.310494+0000","last_undegraded":"2024-04-25T16:51:00.310494+0000","last_fullsized":"2024-04-25T16:51:00.310494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:07.826745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018797+0000","last_change":"2024-04-25T16:50:55.093338+0000","last_active":"2024-04-25T16:51:01.018797+0000","last_peered":"2024-04-25T16:51:01.018797+0000","last_clean":"2024-04-25T16:51:01.018797+0000","last_became_active":"2024-04-25T16:50:55.092985+0000","last_became_peered":"2024-04-25T16:50:55.092985+0000","last_unstale":"2024-04-25T16:51:01.018797+0000","last_undegraded":"2024-04-25T16:51:01.018797+0000","last_fullsized":"2024-04-25T16:51:01.018797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:03.704057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151183+0000","last_change":"2024-04-25T16:50:54.053862+0000","last_active":"2024-04-25T16:51:00.151183+0000","last_peered":"2024-04-25T16:51:00.151183+0000","last_clean":"2024-04-25T16:51:00.151183+0000","last_became_active":"2024-04-25T16:50:54.053156+0000","last_became_peered":"2024-04-25T16:50:54.053156+0000","last_unstale":"2024-04-25T16:51:00.151183+0000","last_undegraded":"2024-04-25T16:51:00.151183+0000","last_fullsized":"2024-04-25T16:51:00.151183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310744+0000","last_change":"2024-04-25T16:50:59.181080+0000","last_active":"2024-04-25T16:51:00.310744+0000","last_peered":"2024-04-25T16:51:00.310744+0000","last_clean":"2024-04-25T16:51:00.310744+0000","last_became_active":"2024-04-25T16:50:59.180638+0000","last_became_peered":"2024-04-25T16:50:59.180638+0000","last_unstale":"2024-04-25T16:51:00.310744+0000","last_undegraded":"2024-04-25T16:51:00.310744+0000","last_fullsized":"2024-04-25T16:51:00.310744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:47.904606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017640+0000","last_change":"2024-04-25T16:50:59.173382+0000","last_active":"2024-04-25T16:51:01.017640+0000","last_peered":"2024-04-25T16:51:01.017640+0000","last_clean":"2024-04-25T16:51:01.017640+0000","last_became_active":"2024-04-25T16:50:59.172921+0000","last_became_peered":"2024-04-25T16:50:59.172921+0000","last_unstale":"2024-04-25T16:51:01.017640+0000","last_undegraded":"2024-04-25T16:51:01.017640+0000","last_fullsized":"2024-04-25T16:51:01.017640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:39.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015213+0000","last_change":"2024-04-25T16:50:55.065015+0000","last_active":"2024-04-25T16:51:01.015213+0000","last_peered":"2024-04-25T16:51:01.015213+0000","last_clean":"2024-04-25T16:51:01.015213+0000","last_became_active":"2024-04-25T16:50:55.063038+0000","last_became_peered":"2024-04-25T16:50:55.063038+0000","last_unstale":"2024-04-25T16:51:01.015213+0000","last_undegraded":"2024-04-25T16:51:01.015213+0000","last_fullsized":"2024-04-25T16:51:01.015213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017330+0000","last_change":"2024-04-25T16:50:54.054787+0000","last_active":"2024-04-25T16:51:01.017330+0000","last_peered":"2024-04-25T16:51:01.017330+0000","last_clean":"2024-04-25T16:51:01.017330+0000","last_became_active":"2024-04-25T16:50:54.051984+0000","last_became_peered":"2024-04-25T16:50:54.051984+0000","last_unstale":"2024-04-25T16:51:01.017330+0000","last_undegraded":"2024-04-25T16:51:01.017330+0000","last_fullsized":"2024-04-25T16:51:01.017330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.166309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151032+0000","last_change":"2024-04-25T16:50:59.154675+0000","last_active":"2024-04-25T16:51:00.151032+0000","last_peered":"2024-04-25T16:51:00.151032+0000","last_clean":"2024-04-25T16:51:00.151032+0000","last_became_active":"2024-04-25T16:50:59.152969+0000","last_became_peered":"2024-04-25T16:50:59.152969+0000","last_unstale":"2024-04-25T16:51:00.151032+0000","last_undegraded":"2024-04-25T16:51:00.151032+0000","last_fullsized":"2024-04-25T16:51:00.151032+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.819554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310256+0000","last_change":"2024-04-25T16:50:55.069932+0000","last_active":"2024-04-25T16:51:00.310256+0000","last_peered":"2024-04-25T16:51:00.310256+0000","last_clean":"2024-04-25T16:51:00.310256+0000","last_became_active":"2024-04-25T16:50:55.060902+0000","last_became_peered":"2024-04-25T16:50:55.060902+0000","last_unstale":"2024-04-25T16:51:00.310256+0000","last_undegraded":"2024-04-25T16:51:00.310256+0000","last_fullsized":"2024-04-25T16:51:00.310256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310569+0000","last_change":"2024-04-25T16:50:54.043300+0000","last_active":"2024-04-25T16:51:00.310569+0000","last_peered":"2024-04-25T16:51:00.310569+0000","last_clean":"2024-04-25T16:51:00.310569+0000","last_became_active":"2024-04-25T16:50:54.042909+0000","last_became_peered":"2024-04-25T16:50:54.042909+0000","last_unstale":"2024-04-25T16:51:00.310569+0000","last_undegraded":"2024-04-25T16:51:00.310569+0000","last_fullsized":"2024-04-25T16:51:00.310569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:22.485560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149622+0000","last_change":"2024-04-25T16:50:54.044711+0000","last_active":"2024-04-25T16:51:00.149622+0000","last_peered":"2024-04-25T16:51:00.149622+0000","last_clean":"2024-04-25T16:51:00.149622+0000","last_became_active":"2024-04-25T16:50:54.043597+0000","last_became_peered":"2024-04-25T16:50:54.043597+0000","last_unstale":"2024-04-25T16:51:00.149622+0000","last_undegraded":"2024-04-25T16:51:00.149622+0000","last_fullsized":"2024-04-25T16:51:00.149622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017542+0000","last_change":"2024-04-25T16:50:55.061228+0000","last_active":"2024-04-25T16:51:01.017542+0000","last_peered":"2024-04-25T16:51:01.017542+0000","last_clean":"2024-04-25T16:51:01.017542+0000","last_became_active":"2024-04-25T16:50:55.060211+0000","last_became_peered":"2024-04-25T16:50:55.060211+0000","last_unstale":"2024-04-25T16:51:01.017542+0000","last_undegraded":"2024-04-25T16:51:01.017542+0000","last_fullsized":"2024-04-25T16:51:01.017542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149479+0000","last_change":"2024-04-25T16:50:59.151109+0000","last_active":"2024-04-25T16:51:00.149479+0000","last_peered":"2024-04-25T16:51:00.149479+0000","last_clean":"2024-04-25T16:51:00.149479+0000","last_became_active":"2024-04-25T16:50:59.150443+0000","last_became_peered":"2024-04-25T16:50:59.150443+0000","last_unstale":"2024-04-25T16:51:00.149479+0000","last_undegraded":"2024-04-25T16:51:00.149479+0000","last_fullsized":"2024-04-25T16:51:00.149479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:45.414606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310730+0000","last_change":"2024-04-25T16:50:54.047737+0000","last_active":"2024-04-25T16:51:00.310730+0000","last_peered":"2024-04-25T16:51:00.310730+0000","last_clean":"2024-04-25T16:51:00.310730+0000","last_became_active":"2024-04-25T16:50:54.047417+0000","last_became_peered":"2024-04-25T16:50:54.047417+0000","last_unstale":"2024-04-25T16:51:00.310730+0000","last_undegraded":"2024-04-25T16:51:00.310730+0000","last_fullsized":"2024-04-25T16:51:00.310730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.166309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153448+0000","last_change":"2024-04-25T16:50:55.064168+0000","last_active":"2024-04-25T16:51:00.153448+0000","last_peered":"2024-04-25T16:51:00.153448+0000","last_clean":"2024-04-25T16:51:00.153448+0000","last_became_active":"2024-04-25T16:50:55.062345+0000","last_became_peered":"2024-04-25T16:50:55.062345+0000","last_unstale":"2024-04-25T16:51:00.153448+0000","last_undegraded":"2024-04-25T16:51:00.153448+0000","last_fullsized":"2024-04-25T16:51:00.153448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:09.018645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311446+0000","last_change":"2024-04-25T16:50:59.143004+0000","last_active":"2024-04-25T16:51:00.311446+0000","last_peered":"2024-04-25T16:51:00.311446+0000","last_clean":"2024-04-25T16:51:00.311446+0000","last_became_active":"2024-04-25T16:50:59.142240+0000","last_became_peered":"2024-04-25T16:50:59.142240+0000","last_unstale":"2024-04-25T16:51:00.311446+0000","last_undegraded":"2024-04-25T16:51:00.311446+0000","last_fullsized":"2024-04-25T16:51:00.311446+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:19.230775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148672+0000","last_change":"2024-04-25T16:50:55.066558+0000","last_active":"2024-04-25T16:51:00.148672+0000","last_peered":"2024-04-25T16:51:00.148672+0000","last_clean":"2024-04-25T16:51:00.148672+0000","last_became_active":"2024-04-25T16:50:55.065201+0000","last_became_peered":"2024-04-25T16:50:55.065201+0000","last_unstale":"2024-04-25T16:51:00.148672+0000","last_undegraded":"2024-04-25T16:51:00.148672+0000","last_fullsized":"2024-04-25T16:51:00.148672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:12.518641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148555+0000","last_change":"2024-04-25T16:50:55.089086+0000","last_active":"2024-04-25T16:51:00.148555+0000","last_peered":"2024-04-25T16:51:00.148555+0000","last_clean":"2024-04-25T16:51:00.148555+0000","last_became_active":"2024-04-25T16:50:55.088788+0000","last_became_peered":"2024-04-25T16:50:55.088788+0000","last_unstale":"2024-04-25T16:51:00.148555+0000","last_undegraded":"2024-04-25T16:51:00.148555+0000","last_fullsized":"2024-04-25T16:51:00.148555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:22.053709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019230+0000","last_change":"2024-04-25T16:50:55.092706+0000","last_active":"2024-04-25T16:51:01.019230+0000","last_peered":"2024-04-25T16:51:01.019230+0000","last_clean":"2024-04-25T16:51:01.019230+0000","last_became_active":"2024-04-25T16:50:55.092411+0000","last_became_peered":"2024-04-25T16:50:55.092411+0000","last_unstale":"2024-04-25T16:51:01.019230+0000","last_undegraded":"2024-04-25T16:51:01.019230+0000","last_fullsized":"2024-04-25T16:51:01.019230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:56.360832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017429+0000","last_change":"2024-04-25T16:50:55.053386+0000","last_active":"2024-04-25T16:51:01.017429+0000","last_peered":"2024-04-25T16:51:01.017429+0000","last_clean":"2024-04-25T16:51:01.017429+0000","last_became_active":"2024-04-25T16:50:55.052555+0000","last_became_peered":"2024-04-25T16:50:55.052555+0000","last_unstale":"2024-04-25T16:51:01.017429+0000","last_undegraded":"2024-04-25T16:51:01.017429+0000","last_fullsized":"2024-04-25T16:51:01.017429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311617+0000","last_change":"2024-04-25T16:50:55.072013+0000","last_active":"2024-04-25T16:51:00.311617+0000","last_peered":"2024-04-25T16:51:00.311617+0000","last_clean":"2024-04-25T16:51:00.311617+0000","last_became_active":"2024-04-25T16:50:55.071089+0000","last_became_peered":"2024-04-25T16:50:55.071089+0000","last_unstale":"2024-04-25T16:51:00.311617+0000","last_undegraded":"2024-04-25T16:51:00.311617+0000","last_fullsized":"2024-04-25T16:51:00.311617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.304249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015430+0000","last_change":"2024-04-25T16:50:55.063809+0000","last_active":"2024-04-25T16:51:01.015430+0000","last_peered":"2024-04-25T16:51:01.015430+0000","last_clean":"2024-04-25T16:51:01.015430+0000","last_became_active":"2024-04-25T16:50:55.062913+0000","last_became_peered":"2024-04-25T16:50:55.062913+0000","last_unstale":"2024-04-25T16:51:01.015430+0000","last_undegraded":"2024-04-25T16:51:01.015430+0000","last_fullsized":"2024-04-25T16:51:01.015430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151698+0000","last_change":"2024-04-25T16:50:55.066745+0000","last_active":"2024-04-25T16:51:00.151698+0000","last_peered":"2024-04-25T16:51:00.151698+0000","last_clean":"2024-04-25T16:51:00.151698+0000","last_became_active":"2024-04-25T16:50:55.065075+0000","last_became_peered":"2024-04-25T16:50:55.065075+0000","last_unstale":"2024-04-25T16:51:00.151698+0000","last_undegraded":"2024-04-25T16:51:00.151698+0000","last_fullsized":"2024-04-25T16:51:00.151698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.312312+0000","last_change":"2024-04-25T16:50:55.081262+0000","last_active":"2024-04-25T16:51:00.312312+0000","last_peered":"2024-04-25T16:51:00.312312+0000","last_clean":"2024-04-25T16:51:00.312312+0000","last_became_active":"2024-04-25T16:50:55.080803+0000","last_became_peered":"2024-04-25T16:50:55.080803+0000","last_unstale":"2024-04-25T16:51:00.312312+0000","last_undegraded":"2024-04-25T16:51:00.312312+0000","last_fullsized":"2024-04-25T16:51:00.312312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:14.342043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017299+0000","last_change":"2024-04-25T16:50:55.090717+0000","last_active":"2024-04-25T16:51:01.017299+0000","last_peered":"2024-04-25T16:51:01.017299+0000","last_clean":"2024-04-25T16:51:01.017299+0000","last_became_active":"2024-04-25T16:50:55.090388+0000","last_became_peered":"2024-04-25T16:50:55.090388+0000","last_unstale":"2024-04-25T16:51:01.017299+0000","last_undegraded":"2024-04-25T16:51:01.017299+0000","last_fullsized":"2024-04-25T16:51:01.017299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310335+0000","last_change":"2024-04-25T16:50:55.072773+0000","last_active":"2024-04-25T16:51:00.310335+0000","last_peered":"2024-04-25T16:51:00.310335+0000","last_clean":"2024-04-25T16:51:00.310335+0000","last_became_active":"2024-04-25T16:50:55.071914+0000","last_became_peered":"2024-04-25T16:50:55.071914+0000","last_unstale":"2024-04-25T16:51:00.310335+0000","last_undegraded":"2024-04-25T16:51:00.310335+0000","last_fullsized":"2024-04-25T16:51:00.310335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018758+0000","last_change":"2024-04-25T16:50:55.093501+0000","last_active":"2024-04-25T16:51:01.018758+0000","last_peered":"2024-04-25T16:51:01.018758+0000","last_clean":"2024-04-25T16:51:01.018758+0000","last_became_active":"2024-04-25T16:50:55.093168+0000","last_became_peered":"2024-04-25T16:50:55.093168+0000","last_unstale":"2024-04-25T16:51:01.018758+0000","last_undegraded":"2024-04-25T16:51:01.018758+0000","last_fullsized":"2024-04-25T16:51:01.018758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:42.081728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017137+0000","last_change":"2024-04-25T16:50:55.066877+0000","last_active":"2024-04-25T16:51:01.017137+0000","last_peered":"2024-04-25T16:51:01.017137+0000","last_clean":"2024-04-25T16:51:01.017137+0000","last_became_active":"2024-04-25T16:50:55.066129+0000","last_became_peered":"2024-04-25T16:50:55.066129+0000","last_unstale":"2024-04-25T16:51:01.017137+0000","last_undegraded":"2024-04-25T16:51:01.017137+0000","last_fullsized":"2024-04-25T16:51:01.017137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:12.518641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150236+0000","last_change":"2024-04-25T16:50:55.052920+0000","last_active":"2024-04-25T16:51:00.150236+0000","last_peered":"2024-04-25T16:51:00.150236+0000","last_clean":"2024-04-25T16:51:00.150236+0000","last_became_active":"2024-04-25T16:50:55.047735+0000","last_became_peered":"2024-04-25T16:50:55.047735+0000","last_unstale":"2024-04-25T16:51:00.150236+0000","last_undegraded":"2024-04-25T16:51:00.150236+0000","last_fullsized":"2024-04-25T16:51:00.150236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020445+0000","last_change":"2024-04-25T16:50:55.090598+0000","last_active":"2024-04-25T16:51:01.020445+0000","last_peered":"2024-04-25T16:51:01.020445+0000","last_clean":"2024-04-25T16:51:01.020445+0000","last_became_active":"2024-04-25T16:50:55.090102+0000","last_became_peered":"2024-04-25T16:50:55.090102+0000","last_unstale":"2024-04-25T16:51:01.020445+0000","last_undegraded":"2024-04-25T16:51:01.020445+0000","last_fullsized":"2024-04-25T16:51:01.020445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:35.322477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150315+0000","last_change":"2024-04-25T16:50:55.081066+0000","last_active":"2024-04-25T16:51:00.150315+0000","last_peered":"2024-04-25T16:51:00.150315+0000","last_clean":"2024-04-25T16:51:00.150315+0000","last_became_active":"2024-04-25T16:50:55.069516+0000","last_became_peered":"2024-04-25T16:50:55.069516+0000","last_unstale":"2024-04-25T16:51:00.150315+0000","last_undegraded":"2024-04-25T16:51:00.150315+0000","last_fullsized":"2024-04-25T16:51:00.150315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:13.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019657+0000","last_change":"2024-04-25T16:50:55.093971+0000","last_active":"2024-04-25T16:51:01.019657+0000","last_peered":"2024-04-25T16:51:01.019657+0000","last_clean":"2024-04-25T16:51:01.019657+0000","last_became_active":"2024-04-25T16:50:55.093421+0000","last_became_peered":"2024-04-25T16:50:55.093421+0000","last_unstale":"2024-04-25T16:51:01.019657+0000","last_undegraded":"2024-04-25T16:51:01.019657+0000","last_fullsized":"2024-04-25T16:51:01.019657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:10.733287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016967+0000","last_change":"2024-04-25T16:50:55.075149+0000","last_active":"2024-04-25T16:51:01.016967+0000","last_peered":"2024-04-25T16:51:01.016967+0000","last_clean":"2024-04-25T16:51:01.016967+0000","last_became_active":"2024-04-25T16:50:55.074215+0000","last_became_peered":"2024-04-25T16:50:55.074215+0000","last_unstale":"2024-04-25T16:51:01.016967+0000","last_undegraded":"2024-04-25T16:51:01.016967+0000","last_fullsized":"2024-04-25T16:51:01.016967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015476+0000","last_change":"2024-04-25T16:50:55.072156+0000","last_active":"2024-04-25T16:51:01.015476+0000","last_peered":"2024-04-25T16:51:01.015476+0000","last_clean":"2024-04-25T16:51:01.015476+0000","last_became_active":"2024-04-25T16:50:55.071295+0000","last_became_peered":"2024-04-25T16:50:55.071295+0000","last_unstale":"2024-04-25T16:51:01.015476+0000","last_undegraded":"2024-04-25T16:51:01.015476+0000","last_fullsized":"2024-04-25T16:51:01.015476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018336+0000","last_change":"2024-04-25T16:50:55.062605+0000","last_active":"2024-04-25T16:51:01.018336+0000","last_peered":"2024-04-25T16:51:01.018336+0000","last_clean":"2024-04-25T16:51:01.018336+0000","last_became_active":"2024-04-25T16:50:55.061970+0000","last_became_peered":"2024-04-25T16:50:55.061970+0000","last_unstale":"2024-04-25T16:51:01.018336+0000","last_undegraded":"2024-04-25T16:51:01.018336+0000","last_fullsized":"2024-04-25T16:51:01.018336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:41.328339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310463+0000","last_change":"2024-04-25T16:50:55.062534+0000","last_active":"2024-04-25T16:51:00.310463+0000","last_peered":"2024-04-25T16:51:00.310463+0000","last_clean":"2024-04-25T16:51:00.310463+0000","last_became_active":"2024-04-25T16:50:55.061601+0000","last_became_peered":"2024-04-25T16:50:55.061601+0000","last_unstale":"2024-04-25T16:51:00.310463+0000","last_undegraded":"2024-04-25T16:51:00.310463+0000","last_fullsized":"2024-04-25T16:51:00.310463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154054+0000","last_change":"2024-04-25T16:50:55.067043+0000","last_active":"2024-04-25T16:51:00.154054+0000","last_peered":"2024-04-25T16:51:00.154054+0000","last_clean":"2024-04-25T16:51:00.154054+0000","last_became_active":"2024-04-25T16:50:55.066155+0000","last_became_peered":"2024-04-25T16:50:55.066155+0000","last_unstale":"2024-04-25T16:51:00.154054+0000","last_undegraded":"2024-04-25T16:51:00.154054+0000","last_fullsized":"2024-04-25T16:51:00.154054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:08.701059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150829+0000","last_change":"2024-04-25T16:50:55.084655+0000","last_active":"2024-04-25T16:51:00.150829+0000","last_peered":"2024-04-25T16:51:00.150829+0000","last_clean":"2024-04-25T16:51:00.150829+0000","last_became_active":"2024-04-25T16:50:55.083647+0000","last_became_peered":"2024-04-25T16:50:55.083647+0000","last_unstale":"2024-04-25T16:51:00.150829+0000","last_undegraded":"2024-04-25T16:51:00.150829+0000","last_fullsized":"2024-04-25T16:51:00.150829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151431+0000","last_change":"2024-04-25T16:50:55.085352+0000","last_active":"2024-04-25T16:51:00.151431+0000","last_peered":"2024-04-25T16:51:00.151431+0000","last_clean":"2024-04-25T16:51:00.151431+0000","last_became_active":"2024-04-25T16:50:55.083135+0000","last_became_peered":"2024-04-25T16:50:55.083135+0000","last_unstale":"2024-04-25T16:51:00.151431+0000","last_undegraded":"2024-04-25T16:51:00.151431+0000","last_fullsized":"2024-04-25T16:51:00.151431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.021207+0000","last_change":"2024-04-25T16:50:55.090981+0000","last_active":"2024-04-25T16:51:01.021207+0000","last_peered":"2024-04-25T16:51:01.021207+0000","last_clean":"2024-04-25T16:51:01.021207+0000","last_became_active":"2024-04-25T16:50:55.090507+0000","last_became_peered":"2024-04-25T16:50:55.090507+0000","last_unstale":"2024-04-25T16:51:01.021207+0000","last_undegraded":"2024-04-25T16:51:01.021207+0000","last_fullsized":"2024-04-25T16:51:01.021207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:10.733287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.309934+0000","last_change":"2024-04-25T16:50:55.090738+0000","last_active":"2024-04-25T16:51:00.309934+0000","last_peered":"2024-04-25T16:51:00.309934+0000","last_clean":"2024-04-25T16:51:00.309934+0000","last_became_active":"2024-04-25T16:50:55.090298+0000","last_became_peered":"2024-04-25T16:50:55.090298+0000","last_unstale":"2024-04-25T16:51:00.309934+0000","last_undegraded":"2024-04-25T16:51:00.309934+0000","last_fullsized":"2024-04-25T16:51:00.309934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:12.518641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017349+0000","last_change":"2024-04-25T16:50:55.083767+0000","last_active":"2024-04-25T16:51:01.017349+0000","last_peered":"2024-04-25T16:51:01.017349+0000","last_clean":"2024-04-25T16:51:01.017349+0000","last_became_active":"2024-04-25T16:50:55.083468+0000","last_became_peered":"2024-04-25T16:50:55.083468+0000","last_unstale":"2024-04-25T16:51:01.017349+0000","last_undegraded":"2024-04-25T16:51:01.017349+0000","last_fullsized":"2024-04-25T16:51:01.017349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.304249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018383+0000","last_change":"2024-04-25T16:50:55.083971+0000","last_active":"2024-04-25T16:51:01.018383+0000","last_peered":"2024-04-25T16:51:01.018383+0000","last_clean":"2024-04-25T16:51:01.018383+0000","last_became_active":"2024-04-25T16:50:55.083688+0000","last_became_peered":"2024-04-25T16:50:55.083688+0000","last_unstale":"2024-04-25T16:51:01.018383+0000","last_undegraded":"2024-04-25T16:51:01.018383+0000","last_fullsized":"2024-04-25T16:51:01.018383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:08.427466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018722+0000","last_change":"2024-04-25T16:50:55.090861+0000","last_active":"2024-04-25T16:51:01.018722+0000","last_peered":"2024-04-25T16:51:01.018722+0000","last_clean":"2024-04-25T16:51:01.018722+0000","last_became_active":"2024-04-25T16:50:55.089863+0000","last_became_peered":"2024-04-25T16:50:55.089863+0000","last_unstale":"2024-04-25T16:51:01.018722+0000","last_undegraded":"2024-04-25T16:51:01.018722+0000","last_fullsized":"2024-04-25T16:51:01.018722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:14.342043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018454+0000","last_change":"2024-04-25T16:50:55.091645+0000","last_active":"2024-04-25T16:51:01.018454+0000","last_peered":"2024-04-25T16:51:01.018454+0000","last_clean":"2024-04-25T16:51:01.018454+0000","last_became_active":"2024-04-25T16:50:55.091308+0000","last_became_peered":"2024-04-25T16:50:55.091308+0000","last_unstale":"2024-04-25T16:51:01.018454+0000","last_undegraded":"2024-04-25T16:51:01.018454+0000","last_fullsized":"2024-04-25T16:51:01.018454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017510+0000","last_change":"2024-04-25T16:50:55.074041+0000","last_active":"2024-04-25T16:51:01.017510+0000","last_peered":"2024-04-25T16:51:01.017510+0000","last_clean":"2024-04-25T16:51:01.017510+0000","last_became_active":"2024-04-25T16:50:55.073142+0000","last_became_peered":"2024-04-25T16:50:55.073142+0000","last_unstale":"2024-04-25T16:51:01.017510+0000","last_undegraded":"2024-04-25T16:51:01.017510+0000","last_fullsized":"2024-04-25T16:51:01.017510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016248+0000","last_change":"2024-04-25T16:50:55.065163+0000","last_active":"2024-04-25T16:51:01.016248+0000","last_peered":"2024-04-25T16:51:01.016248+0000","last_clean":"2024-04-25T16:51:01.016248+0000","last_became_active":"2024-04-25T16:50:55.064257+0000","last_became_peered":"2024-04-25T16:50:55.064257+0000","last_unstale":"2024-04-25T16:51:01.016248+0000","last_undegraded":"2024-04-25T16:51:01.016248+0000","last_fullsized":"2024-04-25T16:51:01.016248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020950+0000","last_change":"2024-04-25T16:50:55.082932+0000","last_active":"2024-04-25T16:51:01.020950+0000","last_peered":"2024-04-25T16:51:01.020950+0000","last_clean":"2024-04-25T16:51:01.020950+0000","last_became_active":"2024-04-25T16:50:55.082589+0000","last_became_peered":"2024-04-25T16:50:55.082589+0000","last_unstale":"2024-04-25T16:51:01.020950+0000","last_undegraded":"2024-04-25T16:51:01.020950+0000","last_fullsized":"2024-04-25T16:51:01.020950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:56.360832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335656,"statfs":{"total":96636764160,"available":96599711744,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:51:24.480 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:51:24.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f406bd640 1 Processor -- start 2024-04-25T16:51:24.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f406bd640 1 -- start start 2024-04-25T16:51:24.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f406bd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3815e990 0x7f9f38165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f406bd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f38165b90 0x7f9f38167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f406bd640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f3806d610 con 0x7f9f38070080 2024-04-25T16:51:24.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f406bd640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9f38158550 con 0x7f9f38165b90 2024-04-25T16:51:24.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f406bd640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3806f7d0 con 0x7f9f3815e990 2024-04-25T16:51:24.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3e459640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f9f38070080 0x7f9f3815e280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39154/0 (socket says 172.21.15.94:39154) 2024-04-25T16:51:24.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3e459640 1 -- 172.21.15.94:0/346187321 learned_addr learned my addr 172.21.15.94:0/346187321 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:24.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3ec5a640 1 --2- 172.21.15.94:0/346187321 >> v2:172.21.15.138:3300/0 conn(0x7f9f38165b90 0x7f9f38167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:24.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3dc58640 1 --2- 172.21.15.94:0/346187321 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3815e990 0x7f9f38165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:24.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1867480173 0 0) 0x7f9f34001120 con 0x7f9f38070080 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f24003530 con 0x7f9f38070080 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4170697299 0 0) 0x7f9f24003530 con 0x7f9f38070080 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3815e990 msgr2=0x7f9f38165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 --2- 172.21.15.94:0/346187321 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3815e990 0x7f9f38165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 >> v2:172.21.15.138:3300/0 conn(0x7f9f38165b90 msgr2=0x7f9f38167fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 --2- 172.21.15.94:0/346187321 >> v2:172.21.15.138:3300/0 conn(0x7f9f38165b90 0x7f9f38167fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3806fc60 con 0x7f9f38070080 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f9f34001850 con 0x7f9f38070080 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 >> v1:172.21.15.94:6789/0 conn(0x7f9f38070080 legacy=0x7f9f3815e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 --2- 172.21.15.94:0/346187321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f240039c0 0x7f9f24003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 --2- 172.21.15.94:0/346187321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f24004810 0x7f9f24004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 --2- 172.21.15.94:0/346187321 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3815e990 0x7f9f24005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38158550 con 0x7f9f24004810 2024-04-25T16:51:24.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3806f7d0 con 0x7f9f240039c0 2024-04-25T16:51:24.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2400b9d0 con 0x7f9f3815e990 2024-04-25T16:51:24.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3dc58640 1 --2- 172.21.15.94:0/346187321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f24004810 0x7f9f24004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:24.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3e459640 1 --2- 172.21.15.94:0/346187321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f240039c0 0x7f9f24003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:24.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3ec5a640 1 --2- 172.21.15.94:0/346187321 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3815e990 0x7f9f24005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:24.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3dc58640 1 -- 172.21.15.94:0/346187321 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3815e990 msgr2=0x7f9f24005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:24.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3dc58640 1 --2- 172.21.15.94:0/346187321 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f3815e990 0x7f9f24005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3dc58640 1 -- 172.21.15.94:0/346187321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f240039c0 msgr2=0x7f9f24003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:24.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3dc58640 1 --2- 172.21.15.94:0/346187321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f240039c0 0x7f9f24003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3dc58640 1 -- 172.21.15.94:0/346187321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f24003530 con 0x7f9f24004810 2024-04-25T16:51:24.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.668+0000 7f9f3e459640 1 --2- 172.21.15.94:0/346187321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f240039c0 0x7f9f24003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:24.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.672+0000 7f9f3dc58640 1 --2- 172.21.15.94:0/346187321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f24004810 0x7f9f24004c20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009ae0 tx=0x7f9f2c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83abd0effb9787fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:24.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.672+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0b2070 con 0x7f9f24004810 2024-04-25T16:51:24.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.672+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2c0a5570 con 0x7f9f24004810 2024-04-25T16:51:24.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.672+0000 7f9f3d457640 1 -- 172.21.15.94:0/346187321 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0096b0 con 0x7f9f24004810 2024-04-25T16:51:24.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.672+0000 7f9f406bd640 1 -- 172.21.15.94:0/346187321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f24004810 msgr2=0x7f9f24004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.672+0000 7f9f406bd640 1 --2- 172.21.15.94:0/346187321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f24004810 0x7f9f24004c20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009ae0 tx=0x7f9f2c01ca90 comp rx=0 tx=0).stop 2024-04-25T16:51:24.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.672+0000 7f9f3ec5a640 1 -- 172.21.15.94:0/346187321 reap_dead start 2024-04-25T16:51:24.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.672+0000 7f9f406bd640 1 -- 172.21.15.94:0/346187321 shutdown_connections 2024-04-25T16:51:24.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.672+0000 7f9f406bd640 1 -- 172.21.15.94:0/346187321 >> 172.21.15.94:0/346187321 conn(0x7f9f380b47f0 msgr2=0x7f9f380b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:24.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.672+0000 7f9f406bd640 1 -- 172.21.15.94:0/346187321 shutdown_connections 2024-04-25T16:51:24.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.672+0000 7f9f406bd640 1 -- 172.21.15.94:0/346187321 wait complete. 2024-04-25T16:51:24.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f406bd640 1 Processor -- start 2024-04-25T16:51:24.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f406bd640 1 -- start start 2024-04-25T16:51:24.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f406bd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f24004810 0x7f9f3809aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f406bd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f380cd670 0x7f9f38151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f406bd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f380cda80 0x7f9f381524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f406bd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f380ae3e0 con 0x7f9f380cda80 2024-04-25T16:51:24.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f406bd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f380ae100 con 0x7f9f380cd670 2024-04-25T16:51:24.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f406bd640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3806f460 con 0x7f9f24004810 2024-04-25T16:51:24.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f3ec5a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f380cda80 0x7f9f381524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:24.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f3ec5a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f380cda80 0x7f9f381524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42480/0 (socket says 172.21.15.94:42480) 2024-04-25T16:51:24.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f3ec5a640 1 -- 172.21.15.94:0/4154958113 learned_addr learned my addr 172.21.15.94:0/4154958113 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:24.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f3e459640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f24004810 0x7f9f3809aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:24.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f3dc58640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f380cd670 0x7f9f38151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:24.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f3ec5a640 1 -- 172.21.15.94:0/4154958113 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f24004810 msgr2=0x7f9f3809aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f3ec5a640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f24004810 0x7f9f3809aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f3ec5a640 1 -- 172.21.15.94:0/4154958113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f380cd670 msgr2=0x7f9f38151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f3ec5a640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f380cd670 0x7f9f38151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f3ec5a640 1 -- 172.21.15.94:0/4154958113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f380ce1e0 con 0x7f9f380cda80 2024-04-25T16:51:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f3ec5a640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f380cda80 0x7f9f381524c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9f380cd640 tx=0x7f9f3000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f2affd640 1 -- 172.21.15.94:0/4154958113 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30016420 con 0x7f9f380cda80 2024-04-25T16:51:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f2affd640 1 -- 172.21.15.94:0/4154958113 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f3001b070 con 0x7f9f380cda80 2024-04-25T16:51:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f406bd640 1 -- 172.21.15.94:0/4154958113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f380ce4c0 con 0x7f9f380cda80 2024-04-25T16:51:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f2affd640 1 -- 172.21.15.94:0/4154958113 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30016820 con 0x7f9f380cda80 2024-04-25T16:51:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f406bd640 1 -- 172.21.15.94:0/4154958113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f380cea80 con 0x7f9f380cda80 2024-04-25T16:51:24.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.676+0000 7f9f406bd640 1 -- 172.21.15.94:0/4154958113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f38076830 con 0x7f9f380cda80 2024-04-25T16:51:24.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.680+0000 7f9f2affd640 1 -- 172.21.15.94:0/4154958113 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9f300169c0 con 0x7f9f380cda80 2024-04-25T16:51:24.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.680+0000 7f9f2affd640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9f1c06f400 0x7f9f1c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:24.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.680+0000 7f9f3e459640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9f1c06f400 0x7f9f1c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:24.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.680+0000 7f9f2affd640 1 -- 172.21.15.94:0/4154958113 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9f300903b0 con 0x7f9f380cda80 2024-04-25T16:51:24.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.680+0000 7f9f3e459640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9f1c06f400 0x7f9f1c071910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9f34001a60 tx=0x7f9f34001220 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:24.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.684+0000 7f9f2affd640 1 -- 172.21.15.94:0/4154958113 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9f30025020 con 0x7f9f380cda80 2024-04-25T16:51:24.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.832+0000 7f9f406bd640 1 -- 172.21.15.94:0/4154958113 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9f380cf280 con 0x7f9f1c06f400 2024-04-25T16:51:24.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.856+0000 7f9f2affd640 1 -- 172.21.15.94:0/4154958113 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332524 (secure 0 0 0) 0x7f9f380cf280 con 0x7f9f1c06f400 2024-04-25T16:51:24.856 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:24.858 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:51:24.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.860+0000 7f9f406bd640 1 -- 172.21.15.94:0/4154958113 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9f1c06f400 msgr2=0x7f9f1c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.860+0000 7f9f406bd640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9f1c06f400 0x7f9f1c071910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9f34001a60 tx=0x7f9f34001220 comp rx=0 tx=0).stop 2024-04-25T16:51:24.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.860+0000 7f9f406bd640 1 -- 172.21.15.94:0/4154958113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f380cda80 msgr2=0x7f9f381524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:24.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.860+0000 7f9f406bd640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f380cda80 0x7f9f381524c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9f380cd640 tx=0x7f9f3000f520 comp rx=0 tx=0).stop 2024-04-25T16:51:24.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.860+0000 7f9f406bd640 1 -- 172.21.15.94:0/4154958113 shutdown_connections 2024-04-25T16:51:24.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.860+0000 7f9f406bd640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f380cda80 0x7f9f381524c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.860+0000 7f9f406bd640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f380cd670 0x7f9f38151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.860+0000 7f9f406bd640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9f1c06f400 0x7f9f1c071910 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.860+0000 7f9f406bd640 1 --2- 172.21.15.94:0/4154958113 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f24004810 0x7f9f3809aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:24.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.860+0000 7f9f406bd640 1 -- 172.21.15.94:0/4154958113 >> 172.21.15.94:0/4154958113 conn(0x7f9f380b47f0 msgr2=0x7f9f380d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:24.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.860+0000 7f9f406bd640 1 -- 172.21.15.94:0/4154958113 shutdown_connections 2024-04-25T16:51:24.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:24.860+0000 7f9f406bd640 1 -- 172.21.15.94:0/4154958113 wait complete. 2024-04-25T16:51:24.876 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-25T16:51:23.672042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240164,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734556,"statfs":{"total":773094113280,"available":772848185344,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005501"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018387+0000","last_change":"2024-04-25T16:50:54.047200+0000","last_active":"2024-04-25T16:51:01.018387+0000","last_peered":"2024-04-25T16:51:01.018387+0000","last_clean":"2024-04-25T16:51:01.018387+0000","last_became_active":"2024-04-25T16:50:54.046862+0000","last_became_peered":"2024-04-25T16:50:54.046862+0000","last_unstale":"2024-04-25T16:51:01.018387+0000","last_undegraded":"2024-04-25T16:51:01.018387+0000","last_fullsized":"2024-04-25T16:51:01.018387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.166309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311306+0000","last_change":"2024-04-25T16:50:55.060286+0000","last_active":"2024-04-25T16:51:00.311306+0000","last_peered":"2024-04-25T16:51:00.311306+0000","last_clean":"2024-04-25T16:51:00.311306+0000","last_became_active":"2024-04-25T16:50:55.059171+0000","last_became_peered":"2024-04-25T16:50:55.059171+0000","last_unstale":"2024-04-25T16:51:00.311306+0000","last_undegraded":"2024-04-25T16:51:00.311306+0000","last_fullsized":"2024-04-25T16:51:00.311306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:08.427466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310185+0000","last_change":"2024-04-25T16:50:59.181612+0000","last_active":"2024-04-25T16:51:00.310185+0000","last_peered":"2024-04-25T16:51:00.310185+0000","last_clean":"2024-04-25T16:51:00.310185+0000","last_became_active":"2024-04-25T16:50:59.180865+0000","last_became_peered":"2024-04-25T16:50:59.180865+0000","last_unstale":"2024-04-25T16:51:00.310185+0000","last_undegraded":"2024-04-25T16:51:00.310185+0000","last_fullsized":"2024-04-25T16:51:00.310185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:59.449855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311587+0000","last_change":"2024-04-25T16:50:54.040413+0000","last_active":"2024-04-25T16:51:00.311587+0000","last_peered":"2024-04-25T16:51:00.311587+0000","last_clean":"2024-04-25T16:51:00.311587+0000","last_became_active":"2024-04-25T16:50:54.040072+0000","last_became_peered":"2024-04-25T16:50:54.040072+0000","last_unstale":"2024-04-25T16:51:00.311587+0000","last_undegraded":"2024-04-25T16:51:00.311587+0000","last_fullsized":"2024-04-25T16:51:00.311587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:24.611706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018056+0000","last_change":"2024-04-25T16:50:55.083877+0000","last_active":"2024-04-25T16:51:01.018056+0000","last_peered":"2024-04-25T16:51:01.018056+0000","last_clean":"2024-04-25T16:51:01.018056+0000","last_became_active":"2024-04-25T16:50:55.083529+0000","last_became_peered":"2024-04-25T16:50:55.083529+0000","last_unstale":"2024-04-25T16:51:01.018056+0000","last_undegraded":"2024-04-25T16:51:01.018056+0000","last_fullsized":"2024-04-25T16:51:01.018056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152476+0000","last_change":"2024-04-25T16:50:59.143706+0000","last_active":"2024-04-25T16:51:00.152476+0000","last_peered":"2024-04-25T16:51:00.152476+0000","last_clean":"2024-04-25T16:51:00.152476+0000","last_became_active":"2024-04-25T16:50:59.142488+0000","last_became_peered":"2024-04-25T16:50:59.142488+0000","last_unstale":"2024-04-25T16:51:00.152476+0000","last_undegraded":"2024-04-25T16:51:00.152476+0000","last_fullsized":"2024-04-25T16:51:00.152476+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:20.203920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020308+0000","last_change":"2024-04-25T16:50:54.061175+0000","last_active":"2024-04-25T16:51:01.020308+0000","last_peered":"2024-04-25T16:51:01.020308+0000","last_clean":"2024-04-25T16:51:01.020308+0000","last_became_active":"2024-04-25T16:50:54.059991+0000","last_became_peered":"2024-04-25T16:50:54.059991+0000","last_unstale":"2024-04-25T16:51:01.020308+0000","last_undegraded":"2024-04-25T16:51:01.020308+0000","last_fullsized":"2024-04-25T16:51:01.020308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018692+0000","last_change":"2024-04-25T16:50:55.087495+0000","last_active":"2024-04-25T16:51:01.018692+0000","last_peered":"2024-04-25T16:51:01.018692+0000","last_clean":"2024-04-25T16:51:01.018692+0000","last_became_active":"2024-04-25T16:50:55.087177+0000","last_became_peered":"2024-04-25T16:50:55.087177+0000","last_unstale":"2024-04-25T16:51:01.018692+0000","last_undegraded":"2024-04-25T16:51:01.018692+0000","last_fullsized":"2024-04-25T16:51:01.018692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154110+0000","last_change":"2024-04-25T16:50:59.153145+0000","last_active":"2024-04-25T16:51:00.154110+0000","last_peered":"2024-04-25T16:51:00.154110+0000","last_clean":"2024-04-25T16:51:00.154110+0000","last_became_active":"2024-04-25T16:50:59.151443+0000","last_became_peered":"2024-04-25T16:50:59.151443+0000","last_unstale":"2024-04-25T16:51:00.154110+0000","last_undegraded":"2024-04-25T16:51:00.154110+0000","last_fullsized":"2024-04-25T16:51:00.154110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:08.117787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019564+0000","last_change":"2024-04-25T16:50:54.064208+0000","last_active":"2024-04-25T16:51:01.019564+0000","last_peered":"2024-04-25T16:51:01.019564+0000","last_clean":"2024-04-25T16:51:01.019564+0000","last_became_active":"2024-04-25T16:50:54.063734+0000","last_became_peered":"2024-04-25T16:50:54.063734+0000","last_unstale":"2024-04-25T16:51:01.019564+0000","last_undegraded":"2024-04-25T16:51:01.019564+0000","last_fullsized":"2024-04-25T16:51:01.019564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018436+0000","last_change":"2024-04-25T16:50:55.087482+0000","last_active":"2024-04-25T16:51:01.018436+0000","last_peered":"2024-04-25T16:51:01.018436+0000","last_clean":"2024-04-25T16:51:01.018436+0000","last_became_active":"2024-04-25T16:50:55.087139+0000","last_became_peered":"2024-04-25T16:50:55.087139+0000","last_unstale":"2024-04-25T16:51:01.018436+0000","last_undegraded":"2024-04-25T16:51:01.018436+0000","last_fullsized":"2024-04-25T16:51:01.018436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.304249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152921+0000","last_change":"2024-04-25T16:50:59.142658+0000","last_active":"2024-04-25T16:51:00.152921+0000","last_peered":"2024-04-25T16:51:00.152921+0000","last_clean":"2024-04-25T16:51:00.152921+0000","last_became_active":"2024-04-25T16:50:59.139343+0000","last_became_peered":"2024-04-25T16:50:59.139343+0000","last_unstale":"2024-04-25T16:51:00.152921+0000","last_undegraded":"2024-04-25T16:51:00.152921+0000","last_fullsized":"2024-04-25T16:51:00.152921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:45.390516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149327+0000","last_change":"2024-04-25T16:50:55.049255+0000","last_active":"2024-04-25T16:51:00.149327+0000","last_peered":"2024-04-25T16:51:00.149327+0000","last_clean":"2024-04-25T16:51:00.149327+0000","last_became_active":"2024-04-25T16:50:55.047635+0000","last_became_peered":"2024-04-25T16:50:55.047635+0000","last_unstale":"2024-04-25T16:51:00.149327+0000","last_undegraded":"2024-04-25T16:51:00.149327+0000","last_fullsized":"2024-04-25T16:51:00.149327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019735+0000","last_change":"2024-04-25T16:50:54.056721+0000","last_active":"2024-04-25T16:51:01.019735+0000","last_peered":"2024-04-25T16:51:01.019735+0000","last_clean":"2024-04-25T16:51:01.019735+0000","last_became_active":"2024-04-25T16:50:54.055845+0000","last_became_peered":"2024-04-25T16:50:54.055845+0000","last_unstale":"2024-04-25T16:51:01.019735+0000","last_undegraded":"2024-04-25T16:51:01.019735+0000","last_fullsized":"2024-04-25T16:51:01.019735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.077660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153494+0000","last_change":"2024-04-25T16:50:59.152269+0000","last_active":"2024-04-25T16:51:00.153494+0000","last_peered":"2024-04-25T16:51:00.153494+0000","last_clean":"2024-04-25T16:51:00.153494+0000","last_became_active":"2024-04-25T16:50:59.150778+0000","last_became_peered":"2024-04-25T16:50:59.150778+0000","last_unstale":"2024-04-25T16:51:00.153494+0000","last_undegraded":"2024-04-25T16:51:00.153494+0000","last_fullsized":"2024-04-25T16:51:00.153494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:12.513733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310585+0000","last_change":"2024-04-25T16:50:55.072126+0000","last_active":"2024-04-25T16:51:00.310585+0000","last_peered":"2024-04-25T16:51:00.310585+0000","last_clean":"2024-04-25T16:51:00.310585+0000","last_became_active":"2024-04-25T16:50:55.071191+0000","last_became_peered":"2024-04-25T16:50:55.071191+0000","last_unstale":"2024-04-25T16:51:00.310585+0000","last_undegraded":"2024-04-25T16:51:00.310585+0000","last_fullsized":"2024-04-25T16:51:00.310585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015930+0000","last_change":"2024-04-25T16:50:54.033554+0000","last_active":"2024-04-25T16:51:01.015930+0000","last_peered":"2024-04-25T16:51:01.015930+0000","last_clean":"2024-04-25T16:51:01.015930+0000","last_became_active":"2024-04-25T16:50:54.032467+0000","last_became_peered":"2024-04-25T16:50:54.032467+0000","last_unstale":"2024-04-25T16:51:01.015930+0000","last_undegraded":"2024-04-25T16:51:01.015930+0000","last_fullsized":"2024-04-25T16:51:01.015930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.077660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153884+0000","last_change":"2024-04-25T16:50:59.157126+0000","last_active":"2024-04-25T16:51:00.153884+0000","last_peered":"2024-04-25T16:51:00.153884+0000","last_clean":"2024-04-25T16:51:00.153884+0000","last_became_active":"2024-04-25T16:50:59.153352+0000","last_became_peered":"2024-04-25T16:50:59.153352+0000","last_unstale":"2024-04-25T16:51:00.153884+0000","last_undegraded":"2024-04-25T16:51:00.153884+0000","last_fullsized":"2024-04-25T16:51:00.153884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:18.428310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310685+0000","last_change":"2024-04-25T16:50:55.059831+0000","last_active":"2024-04-25T16:51:00.310685+0000","last_peered":"2024-04-25T16:51:00.310685+0000","last_clean":"2024-04-25T16:51:00.310685+0000","last_became_active":"2024-04-25T16:50:55.058958+0000","last_became_peered":"2024-04-25T16:50:55.058958+0000","last_unstale":"2024-04-25T16:51:00.310685+0000","last_undegraded":"2024-04-25T16:51:00.310685+0000","last_fullsized":"2024-04-25T16:51:00.310685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015662+0000","last_change":"2024-04-25T16:50:54.045690+0000","last_active":"2024-04-25T16:51:01.015662+0000","last_peered":"2024-04-25T16:51:01.015662+0000","last_clean":"2024-04-25T16:51:01.015662+0000","last_became_active":"2024-04-25T16:50:54.045057+0000","last_became_peered":"2024-04-25T16:50:54.045057+0000","last_unstale":"2024-04-25T16:51:01.015662+0000","last_undegraded":"2024-04-25T16:51:01.015662+0000","last_fullsized":"2024-04-25T16:51:01.015662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:27.893510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016104+0000","last_change":"2024-04-25T16:50:59.163126+0000","last_active":"2024-04-25T16:51:01.016104+0000","last_peered":"2024-04-25T16:51:01.016104+0000","last_clean":"2024-04-25T16:51:01.016104+0000","last_became_active":"2024-04-25T16:50:59.162606+0000","last_became_peered":"2024-04-25T16:50:59.162606+0000","last_unstale":"2024-04-25T16:51:01.016104+0000","last_undegraded":"2024-04-25T16:51:01.016104+0000","last_fullsized":"2024-04-25T16:51:01.016104+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.992576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153472+0000","last_change":"2024-04-25T16:50:55.079857+0000","last_active":"2024-04-25T16:51:00.153472+0000","last_peered":"2024-04-25T16:51:00.153472+0000","last_clean":"2024-04-25T16:51:00.153472+0000","last_became_active":"2024-04-25T16:50:55.070021+0000","last_became_peered":"2024-04-25T16:50:55.070021+0000","last_unstale":"2024-04-25T16:51:00.153472+0000","last_undegraded":"2024-04-25T16:51:00.153472+0000","last_fullsized":"2024-04-25T16:51:00.153472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:00.753427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017068+0000","last_change":"2024-04-25T16:50:54.050226+0000","last_active":"2024-04-25T16:51:01.017068+0000","last_peered":"2024-04-25T16:51:01.017068+0000","last_clean":"2024-04-25T16:51:01.017068+0000","last_became_active":"2024-04-25T16:50:54.049451+0000","last_became_peered":"2024-04-25T16:50:54.049451+0000","last_unstale":"2024-04-25T16:51:01.017068+0000","last_undegraded":"2024-04-25T16:51:01.017068+0000","last_fullsized":"2024-04-25T16:51:01.017068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:26.942742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154683+0000","last_change":"2024-04-25T16:50:59.152071+0000","last_active":"2024-04-25T16:51:00.154683+0000","last_peered":"2024-04-25T16:51:00.154683+0000","last_clean":"2024-04-25T16:51:00.154683+0000","last_became_active":"2024-04-25T16:50:59.149014+0000","last_became_peered":"2024-04-25T16:50:59.149014+0000","last_unstale":"2024-04-25T16:51:00.154683+0000","last_undegraded":"2024-04-25T16:51:00.154683+0000","last_fullsized":"2024-04-25T16:51:00.154683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:00.447099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018096+0000","last_change":"2024-04-25T16:50:55.057085+0000","last_active":"2024-04-25T16:51:01.018096+0000","last_peered":"2024-04-25T16:51:01.018096+0000","last_clean":"2024-04-25T16:51:01.018096+0000","last_became_active":"2024-04-25T16:50:55.056254+0000","last_became_peered":"2024-04-25T16:50:55.056254+0000","last_unstale":"2024-04-25T16:51:01.018096+0000","last_undegraded":"2024-04-25T16:51:01.018096+0000","last_fullsized":"2024-04-25T16:51:01.018096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151405+0000","last_change":"2024-04-25T16:50:54.047574+0000","last_active":"2024-04-25T16:51:00.151405+0000","last_peered":"2024-04-25T16:51:00.151405+0000","last_clean":"2024-04-25T16:51:00.151405+0000","last_became_active":"2024-04-25T16:50:54.047164+0000","last_became_peered":"2024-04-25T16:50:54.047164+0000","last_unstale":"2024-04-25T16:51:00.151405+0000","last_undegraded":"2024-04-25T16:51:00.151405+0000","last_fullsized":"2024-04-25T16:51:00.151405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152691+0000","last_change":"2024-04-25T16:50:59.181342+0000","last_active":"2024-04-25T16:51:00.152691+0000","last_peered":"2024-04-25T16:51:00.152691+0000","last_clean":"2024-04-25T16:51:00.152691+0000","last_became_active":"2024-04-25T16:50:59.180010+0000","last_became_peered":"2024-04-25T16:50:59.180010+0000","last_unstale":"2024-04-25T16:51:00.152691+0000","last_undegraded":"2024-04-25T16:51:00.152691+0000","last_fullsized":"2024-04-25T16:51:00.152691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:09.707352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.312597+0000","last_change":"2024-04-25T16:50:55.093718+0000","last_active":"2024-04-25T16:51:00.312597+0000","last_peered":"2024-04-25T16:51:00.312597+0000","last_clean":"2024-04-25T16:51:00.312597+0000","last_became_active":"2024-04-25T16:50:55.093376+0000","last_became_peered":"2024-04-25T16:50:55.093376+0000","last_unstale":"2024-04-25T16:51:00.312597+0000","last_undegraded":"2024-04-25T16:51:00.312597+0000","last_fullsized":"2024-04-25T16:51:00.312597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:04.031520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016325+0000","last_change":"2024-04-25T16:50:54.040703+0000","last_active":"2024-04-25T16:51:01.016325+0000","last_peered":"2024-04-25T16:51:01.016325+0000","last_clean":"2024-04-25T16:51:01.016325+0000","last_became_active":"2024-04-25T16:50:54.039961+0000","last_became_peered":"2024-04-25T16:50:54.039961+0000","last_unstale":"2024-04-25T16:51:01.016325+0000","last_undegraded":"2024-04-25T16:51:01.016325+0000","last_fullsized":"2024-04-25T16:51:01.016325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310069+0000","last_change":"2024-04-25T16:50:59.161678+0000","last_active":"2024-04-25T16:51:00.310069+0000","last_peered":"2024-04-25T16:51:00.310069+0000","last_clean":"2024-04-25T16:51:00.310069+0000","last_became_active":"2024-04-25T16:50:59.161064+0000","last_became_peered":"2024-04-25T16:50:59.161064+0000","last_unstale":"2024-04-25T16:51:00.310069+0000","last_undegraded":"2024-04-25T16:51:00.310069+0000","last_fullsized":"2024-04-25T16:51:00.310069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:19.854569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015648+0000","last_change":"2024-04-25T16:50:55.069738+0000","last_active":"2024-04-25T16:51:01.015648+0000","last_peered":"2024-04-25T16:51:01.015648+0000","last_clean":"2024-04-25T16:51:01.015648+0000","last_became_active":"2024-04-25T16:50:55.068864+0000","last_became_peered":"2024-04-25T16:50:55.068864+0000","last_unstale":"2024-04-25T16:51:01.015648+0000","last_undegraded":"2024-04-25T16:51:01.015648+0000","last_fullsized":"2024-04-25T16:51:01.015648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020503+0000","last_change":"2024-04-25T16:50:54.061758+0000","last_active":"2024-04-25T16:51:01.020503+0000","last_peered":"2024-04-25T16:51:01.020503+0000","last_clean":"2024-04-25T16:51:01.020503+0000","last_became_active":"2024-04-25T16:50:54.061267+0000","last_became_peered":"2024-04-25T16:50:54.061267+0000","last_unstale":"2024-04-25T16:51:01.020503+0000","last_undegraded":"2024-04-25T16:51:01.020503+0000","last_fullsized":"2024-04-25T16:51:01.020503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:21.043343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017543+0000","last_change":"2024-04-25T16:50:59.174520+0000","last_active":"2024-04-25T16:51:01.017543+0000","last_peered":"2024-04-25T16:51:01.017543+0000","last_clean":"2024-04-25T16:51:01.017543+0000","last_became_active":"2024-04-25T16:50:59.173812+0000","last_became_peered":"2024-04-25T16:50:59.173812+0000","last_unstale":"2024-04-25T16:51:01.017543+0000","last_undegraded":"2024-04-25T16:51:01.017543+0000","last_fullsized":"2024-04-25T16:51:01.017543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:29.939373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018748+0000","last_change":"2024-04-25T16:50:55.090616+0000","last_active":"2024-04-25T16:51:01.018748+0000","last_peered":"2024-04-25T16:51:01.018748+0000","last_clean":"2024-04-25T16:51:01.018748+0000","last_became_active":"2024-04-25T16:50:55.090287+0000","last_became_peered":"2024-04-25T16:50:55.090287+0000","last_unstale":"2024-04-25T16:51:01.018748+0000","last_undegraded":"2024-04-25T16:51:01.018748+0000","last_fullsized":"2024-04-25T16:51:01.018748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:04.031520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015756+0000","last_change":"2024-04-25T16:50:54.023906+0000","last_active":"2024-04-25T16:51:01.015756+0000","last_peered":"2024-04-25T16:51:01.015756+0000","last_clean":"2024-04-25T16:51:01.015756+0000","last_became_active":"2024-04-25T16:50:54.022198+0000","last_became_peered":"2024-04-25T16:50:54.022198+0000","last_unstale":"2024-04-25T16:51:01.015756+0000","last_undegraded":"2024-04-25T16:51:01.015756+0000","last_fullsized":"2024-04-25T16:51:01.015756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015375+0000","last_change":"2024-04-25T16:50:59.153325+0000","last_active":"2024-04-25T16:51:01.015375+0000","last_peered":"2024-04-25T16:51:01.015375+0000","last_clean":"2024-04-25T16:51:01.015375+0000","last_became_active":"2024-04-25T16:50:59.152431+0000","last_became_peered":"2024-04-25T16:50:59.152431+0000","last_unstale":"2024-04-25T16:51:01.015375+0000","last_undegraded":"2024-04-25T16:51:01.015375+0000","last_fullsized":"2024-04-25T16:51:01.015375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.819554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154818+0000","last_change":"2024-04-25T16:50:55.058098+0000","last_active":"2024-04-25T16:51:00.154818+0000","last_peered":"2024-04-25T16:51:00.154818+0000","last_clean":"2024-04-25T16:51:00.154818+0000","last_became_active":"2024-04-25T16:50:55.057233+0000","last_became_peered":"2024-04-25T16:50:55.057233+0000","last_unstale":"2024-04-25T16:51:00.154818+0000","last_undegraded":"2024-04-25T16:51:00.154818+0000","last_fullsized":"2024-04-25T16:51:00.154818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:36.088026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019426+0000","last_change":"2024-04-25T16:50:54.059732+0000","last_active":"2024-04-25T16:51:01.019426+0000","last_peered":"2024-04-25T16:51:01.019426+0000","last_clean":"2024-04-25T16:51:01.019426+0000","last_became_active":"2024-04-25T16:50:54.058805+0000","last_became_peered":"2024-04-25T16:50:54.058805+0000","last_unstale":"2024-04-25T16:51:01.019426+0000","last_undegraded":"2024-04-25T16:51:01.019426+0000","last_fullsized":"2024-04-25T16:51:01.019426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015551+0000","last_change":"2024-04-25T16:50:59.157551+0000","last_active":"2024-04-25T16:51:01.015551+0000","last_peered":"2024-04-25T16:51:01.015551+0000","last_clean":"2024-04-25T16:51:01.015551+0000","last_became_active":"2024-04-25T16:50:59.156077+0000","last_became_peered":"2024-04-25T16:50:59.156077+0000","last_unstale":"2024-04-25T16:51:01.015551+0000","last_undegraded":"2024-04-25T16:51:01.015551+0000","last_fullsized":"2024-04-25T16:51:01.015551+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:29.939373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015015+0000","last_change":"2024-04-25T16:50:55.072516+0000","last_active":"2024-04-25T16:51:01.015015+0000","last_peered":"2024-04-25T16:51:01.015015+0000","last_clean":"2024-04-25T16:51:01.015015+0000","last_became_active":"2024-04-25T16:50:55.071677+0000","last_became_peered":"2024-04-25T16:50:55.071677+0000","last_unstale":"2024-04-25T16:51:01.015015+0000","last_undegraded":"2024-04-25T16:51:01.015015+0000","last_fullsized":"2024-04-25T16:51:01.015015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148828+0000","last_change":"2024-04-25T16:50:54.026443+0000","last_active":"2024-04-25T16:51:00.148828+0000","last_peered":"2024-04-25T16:51:00.148828+0000","last_clean":"2024-04-25T16:51:00.148828+0000","last_became_active":"2024-04-25T16:50:54.024224+0000","last_became_peered":"2024-04-25T16:50:54.024224+0000","last_unstale":"2024-04-25T16:51:00.148828+0000","last_undegraded":"2024-04-25T16:51:00.148828+0000","last_fullsized":"2024-04-25T16:51:00.148828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016170+0000","last_change":"2024-04-25T16:50:59.149300+0000","last_active":"2024-04-25T16:51:01.016170+0000","last_peered":"2024-04-25T16:51:01.016170+0000","last_clean":"2024-04-25T16:51:01.016170+0000","last_became_active":"2024-04-25T16:50:59.147799+0000","last_became_peered":"2024-04-25T16:50:59.147799+0000","last_unstale":"2024-04-25T16:51:01.016170+0000","last_undegraded":"2024-04-25T16:51:01.016170+0000","last_fullsized":"2024-04-25T16:51:01.016170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:05:04.839576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016900+0000","last_change":"2024-04-25T16:50:55.048595+0000","last_active":"2024-04-25T16:51:01.016900+0000","last_peered":"2024-04-25T16:51:01.016900+0000","last_clean":"2024-04-25T16:51:01.016900+0000","last_became_active":"2024-04-25T16:50:55.047706+0000","last_became_peered":"2024-04-25T16:50:55.047706+0000","last_unstale":"2024-04-25T16:51:01.016900+0000","last_undegraded":"2024-04-25T16:51:01.016900+0000","last_fullsized":"2024-04-25T16:51:01.016900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:56.480058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017350+0000","last_change":"2024-04-25T16:50:54.053397+0000","last_active":"2024-04-25T16:51:01.017350+0000","last_peered":"2024-04-25T16:51:01.017350+0000","last_clean":"2024-04-25T16:51:01.017350+0000","last_became_active":"2024-04-25T16:50:54.052633+0000","last_became_peered":"2024-04-25T16:50:54.052633+0000","last_unstale":"2024-04-25T16:51:01.017350+0000","last_undegraded":"2024-04-25T16:51:01.017350+0000","last_fullsized":"2024-04-25T16:51:01.017350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018189+0000","last_change":"2024-04-25T16:50:59.162703+0000","last_active":"2024-04-25T16:51:01.018189+0000","last_peered":"2024-04-25T16:51:01.018189+0000","last_clean":"2024-04-25T16:51:01.018189+0000","last_became_active":"2024-04-25T16:50:59.161980+0000","last_became_peered":"2024-04-25T16:50:59.161980+0000","last_unstale":"2024-04-25T16:51:01.018189+0000","last_undegraded":"2024-04-25T16:51:01.018189+0000","last_fullsized":"2024-04-25T16:51:01.018189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:05:04.839576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310673+0000","last_change":"2024-04-25T16:50:55.090720+0000","last_active":"2024-04-25T16:51:00.310673+0000","last_peered":"2024-04-25T16:51:00.310673+0000","last_clean":"2024-04-25T16:51:00.310673+0000","last_became_active":"2024-04-25T16:50:55.090360+0000","last_became_peered":"2024-04-25T16:50:55.090360+0000","last_unstale":"2024-04-25T16:51:00.310673+0000","last_undegraded":"2024-04-25T16:51:00.310673+0000","last_fullsized":"2024-04-25T16:51:00.310673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018278+0000","last_change":"2024-04-25T16:50:54.050879+0000","last_active":"2024-04-25T16:51:01.018278+0000","last_peered":"2024-04-25T16:51:01.018278+0000","last_clean":"2024-04-25T16:51:01.018278+0000","last_became_active":"2024-04-25T16:50:54.049784+0000","last_became_peered":"2024-04-25T16:50:54.049784+0000","last_unstale":"2024-04-25T16:51:01.018278+0000","last_undegraded":"2024-04-25T16:51:01.018278+0000","last_fullsized":"2024-04-25T16:51:01.018278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":118,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017460+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:51:01.017460+0000","last_peered":"2024-04-25T16:51:01.017460+0000","last_clean":"2024-04-25T16:51:01.017460+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:51:01.017460+0000","last_undegraded":"2024-04-25T16:51:01.017460+0000","last_fullsized":"2024-04-25T16:51:01.017460+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150640+0000","last_change":"2024-04-25T16:50:59.175343+0000","last_active":"2024-04-25T16:51:00.150640+0000","last_peered":"2024-04-25T16:51:00.150640+0000","last_clean":"2024-04-25T16:51:00.150640+0000","last_became_active":"2024-04-25T16:50:59.173979+0000","last_became_peered":"2024-04-25T16:50:59.173979+0000","last_unstale":"2024-04-25T16:51:00.150640+0000","last_undegraded":"2024-04-25T16:51:00.150640+0000","last_fullsized":"2024-04-25T16:51:00.150640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:07.790324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019964+0000","last_change":"2024-04-25T16:50:55.092868+0000","last_active":"2024-04-25T16:51:01.019964+0000","last_peered":"2024-04-25T16:51:01.019964+0000","last_clean":"2024-04-25T16:51:01.019964+0000","last_became_active":"2024-04-25T16:50:55.092569+0000","last_became_peered":"2024-04-25T16:50:55.092569+0000","last_unstale":"2024-04-25T16:51:01.019964+0000","last_undegraded":"2024-04-25T16:51:01.019964+0000","last_fullsized":"2024-04-25T16:51:01.019964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:03.704057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149644+0000","last_change":"2024-04-25T16:50:54.056537+0000","last_active":"2024-04-25T16:51:00.149644+0000","last_peered":"2024-04-25T16:51:00.149644+0000","last_clean":"2024-04-25T16:51:00.149644+0000","last_became_active":"2024-04-25T16:50:54.056219+0000","last_became_peered":"2024-04-25T16:50:54.056219+0000","last_unstale":"2024-04-25T16:51:00.149644+0000","last_undegraded":"2024-04-25T16:51:00.149644+0000","last_fullsized":"2024-04-25T16:51:00.149644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149259+0000","last_change":"2024-04-25T16:50:59.168724+0000","last_active":"2024-04-25T16:51:00.149259+0000","last_peered":"2024-04-25T16:51:00.149259+0000","last_clean":"2024-04-25T16:51:00.149259+0000","last_became_active":"2024-04-25T16:50:59.167823+0000","last_became_peered":"2024-04-25T16:50:59.167823+0000","last_unstale":"2024-04-25T16:51:00.149259+0000","last_undegraded":"2024-04-25T16:51:00.149259+0000","last_fullsized":"2024-04-25T16:51:00.149259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:00.447099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018846+0000","last_change":"2024-04-25T16:50:55.091727+0000","last_active":"2024-04-25T16:51:01.018846+0000","last_peered":"2024-04-25T16:51:01.018846+0000","last_clean":"2024-04-25T16:51:01.018846+0000","last_became_active":"2024-04-25T16:50:55.091409+0000","last_became_peered":"2024-04-25T16:50:55.091409+0000","last_unstale":"2024-04-25T16:51:01.018846+0000","last_undegraded":"2024-04-25T16:51:01.018846+0000","last_fullsized":"2024-04-25T16:51:01.018846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:14.342043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015818+0000","last_change":"2024-04-25T16:50:54.051928+0000","last_active":"2024-04-25T16:51:01.015818+0000","last_peered":"2024-04-25T16:51:01.015818+0000","last_clean":"2024-04-25T16:51:01.015818+0000","last_became_active":"2024-04-25T16:50:54.045379+0000","last_became_peered":"2024-04-25T16:50:54.045379+0000","last_unstale":"2024-04-25T16:51:01.015818+0000","last_undegraded":"2024-04-25T16:51:01.015818+0000","last_fullsized":"2024-04-25T16:51:01.015818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017284+0000","last_change":"2024-04-25T16:50:59.149171+0000","last_active":"2024-04-25T16:51:01.017284+0000","last_peered":"2024-04-25T16:51:01.017284+0000","last_clean":"2024-04-25T16:51:01.017284+0000","last_became_active":"2024-04-25T16:50:59.148753+0000","last_became_peered":"2024-04-25T16:50:59.148753+0000","last_unstale":"2024-04-25T16:51:01.017284+0000","last_undegraded":"2024-04-25T16:51:01.017284+0000","last_fullsized":"2024-04-25T16:51:01.017284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.819554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.021346+0000","last_change":"2024-04-25T16:50:55.093340+0000","last_active":"2024-04-25T16:51:01.021346+0000","last_peered":"2024-04-25T16:51:01.021346+0000","last_clean":"2024-04-25T16:51:01.021346+0000","last_became_active":"2024-04-25T16:50:55.092870+0000","last_became_peered":"2024-04-25T16:50:55.092870+0000","last_unstale":"2024-04-25T16:51:01.021346+0000","last_undegraded":"2024-04-25T16:51:01.021346+0000","last_fullsized":"2024-04-25T16:51:01.021346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:25.853106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150507+0000","last_change":"2024-04-25T16:50:54.041300+0000","last_active":"2024-04-25T16:51:00.150507+0000","last_peered":"2024-04-25T16:51:00.150507+0000","last_clean":"2024-04-25T16:51:00.150507+0000","last_became_active":"2024-04-25T16:50:54.040987+0000","last_became_peered":"2024-04-25T16:50:54.040987+0000","last_unstale":"2024-04-25T16:51:00.150507+0000","last_undegraded":"2024-04-25T16:51:00.150507+0000","last_fullsized":"2024-04-25T16:51:00.150507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016995+0000","last_change":"2024-04-25T16:50:59.180254+0000","last_active":"2024-04-25T16:51:01.016995+0000","last_peered":"2024-04-25T16:51:01.016995+0000","last_clean":"2024-04-25T16:51:01.016995+0000","last_became_active":"2024-04-25T16:50:59.179432+0000","last_became_peered":"2024-04-25T16:50:59.179432+0000","last_unstale":"2024-04-25T16:51:01.016995+0000","last_undegraded":"2024-04-25T16:51:01.016995+0000","last_fullsized":"2024-04-25T16:51:01.016995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:44.752448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020598+0000","last_change":"2024-04-25T16:50:55.085874+0000","last_active":"2024-04-25T16:51:01.020598+0000","last_peered":"2024-04-25T16:51:01.020598+0000","last_clean":"2024-04-25T16:51:01.020598+0000","last_became_active":"2024-04-25T16:50:55.085347+0000","last_became_peered":"2024-04-25T16:50:55.085347+0000","last_unstale":"2024-04-25T16:51:01.020598+0000","last_undegraded":"2024-04-25T16:51:01.020598+0000","last_fullsized":"2024-04-25T16:51:01.020598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:14.906309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152960+0000","last_change":"2024-04-25T16:50:54.037711+0000","last_active":"2024-04-25T16:51:00.152960+0000","last_peered":"2024-04-25T16:51:00.152960+0000","last_clean":"2024-04-25T16:51:00.152960+0000","last_became_active":"2024-04-25T16:50:54.036847+0000","last_became_peered":"2024-04-25T16:50:54.036847+0000","last_unstale":"2024-04-25T16:51:00.152960+0000","last_undegraded":"2024-04-25T16:51:00.152960+0000","last_fullsized":"2024-04-25T16:51:00.152960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:21.644845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.152479+0000","last_change":"2024-04-25T16:50:59.165076+0000","last_active":"2024-04-25T16:51:00.152479+0000","last_peered":"2024-04-25T16:51:00.152479+0000","last_clean":"2024-04-25T16:51:00.152479+0000","last_became_active":"2024-04-25T16:50:59.159200+0000","last_became_peered":"2024-04-25T16:50:59.159200+0000","last_unstale":"2024-04-25T16:51:00.152479+0000","last_undegraded":"2024-04-25T16:51:00.152479+0000","last_fullsized":"2024-04-25T16:51:00.152479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:11.722626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017585+0000","last_change":"2024-04-25T16:50:55.085792+0000","last_active":"2024-04-25T16:51:01.017585+0000","last_peered":"2024-04-25T16:51:01.017585+0000","last_clean":"2024-04-25T16:51:01.017585+0000","last_became_active":"2024-04-25T16:50:55.085212+0000","last_became_peered":"2024-04-25T16:50:55.085212+0000","last_unstale":"2024-04-25T16:51:01.017585+0000","last_undegraded":"2024-04-25T16:51:01.017585+0000","last_fullsized":"2024-04-25T16:51:01.017585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:08.427466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148299+0000","last_change":"2024-04-25T16:50:54.021199+0000","last_active":"2024-04-25T16:51:00.148299+0000","last_peered":"2024-04-25T16:51:00.148299+0000","last_clean":"2024-04-25T16:51:00.148299+0000","last_became_active":"2024-04-25T16:50:54.020716+0000","last_became_peered":"2024-04-25T16:50:54.020716+0000","last_unstale":"2024-04-25T16:51:00.148299+0000","last_undegraded":"2024-04-25T16:51:00.148299+0000","last_fullsized":"2024-04-25T16:51:00.148299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:27.893510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017880+0000","last_change":"2024-04-25T16:50:59.160368+0000","last_active":"2024-04-25T16:51:01.017880+0000","last_peered":"2024-04-25T16:51:01.017880+0000","last_clean":"2024-04-25T16:51:01.017880+0000","last_became_active":"2024-04-25T16:50:59.159232+0000","last_became_peered":"2024-04-25T16:50:59.159232+0000","last_unstale":"2024-04-25T16:51:01.017880+0000","last_undegraded":"2024-04-25T16:51:01.017880+0000","last_fullsized":"2024-04-25T16:51:01.017880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.992576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148948+0000","last_change":"2024-04-25T16:50:55.059427+0000","last_active":"2024-04-25T16:51:00.148948+0000","last_peered":"2024-04-25T16:51:00.148948+0000","last_clean":"2024-04-25T16:51:00.148948+0000","last_became_active":"2024-04-25T16:50:55.058611+0000","last_became_peered":"2024-04-25T16:50:55.058611+0000","last_unstale":"2024-04-25T16:51:00.148948+0000","last_undegraded":"2024-04-25T16:51:00.148948+0000","last_fullsized":"2024-04-25T16:51:00.148948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149289+0000","last_change":"2024-04-25T16:50:54.044858+0000","last_active":"2024-04-25T16:51:00.149289+0000","last_peered":"2024-04-25T16:51:00.149289+0000","last_clean":"2024-04-25T16:51:00.149289+0000","last_became_active":"2024-04-25T16:50:54.043345+0000","last_became_peered":"2024-04-25T16:50:54.043345+0000","last_unstale":"2024-04-25T16:51:00.149289+0000","last_undegraded":"2024-04-25T16:51:00.149289+0000","last_fullsized":"2024-04-25T16:51:00.149289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:58.336991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.312170+0000","last_change":"2024-04-25T16:50:59.173258+0000","last_active":"2024-04-25T16:51:00.312170+0000","last_peered":"2024-04-25T16:51:00.312170+0000","last_clean":"2024-04-25T16:51:00.312170+0000","last_became_active":"2024-04-25T16:50:59.172637+0000","last_became_peered":"2024-04-25T16:50:59.172637+0000","last_unstale":"2024-04-25T16:51:00.312170+0000","last_undegraded":"2024-04-25T16:51:00.312170+0000","last_fullsized":"2024-04-25T16:51:00.312170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:46.618416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019246+0000","last_change":"2024-04-25T16:50:55.085682+0000","last_active":"2024-04-25T16:51:01.019246+0000","last_peered":"2024-04-25T16:51:01.019246+0000","last_clean":"2024-04-25T16:51:01.019246+0000","last_became_active":"2024-04-25T16:50:55.084855+0000","last_became_peered":"2024-04-25T16:50:55.084855+0000","last_unstale":"2024-04-25T16:51:01.019246+0000","last_undegraded":"2024-04-25T16:51:01.019246+0000","last_fullsized":"2024-04-25T16:51:01.019246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:42.081728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019411+0000","last_change":"2024-04-25T16:50:54.059696+0000","last_active":"2024-04-25T16:51:01.019411+0000","last_peered":"2024-04-25T16:51:01.019411+0000","last_clean":"2024-04-25T16:51:01.019411+0000","last_became_active":"2024-04-25T16:50:54.058979+0000","last_became_peered":"2024-04-25T16:50:54.058979+0000","last_unstale":"2024-04-25T16:51:01.019411+0000","last_undegraded":"2024-04-25T16:51:01.019411+0000","last_fullsized":"2024-04-25T16:51:01.019411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:08.008279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311564+0000","last_change":"2024-04-25T16:50:59.163291+0000","last_active":"2024-04-25T16:51:00.311564+0000","last_peered":"2024-04-25T16:51:00.311564+0000","last_clean":"2024-04-25T16:51:00.311564+0000","last_became_active":"2024-04-25T16:50:59.162316+0000","last_became_peered":"2024-04-25T16:50:59.162316+0000","last_unstale":"2024-04-25T16:51:00.311564+0000","last_undegraded":"2024-04-25T16:51:00.311564+0000","last_fullsized":"2024-04-25T16:51:00.311564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:41.776182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018471+0000","last_change":"2024-04-25T16:50:55.067672+0000","last_active":"2024-04-25T16:51:01.018471+0000","last_peered":"2024-04-25T16:51:01.018471+0000","last_clean":"2024-04-25T16:51:01.018471+0000","last_became_active":"2024-04-25T16:50:55.066870+0000","last_became_peered":"2024-04-25T16:50:55.066870+0000","last_unstale":"2024-04-25T16:51:01.018471+0000","last_undegraded":"2024-04-25T16:51:01.018471+0000","last_fullsized":"2024-04-25T16:51:01.018471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:28:31.672027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017648+0000","last_change":"2024-04-25T16:50:54.036862+0000","last_active":"2024-04-25T16:51:01.017648+0000","last_peered":"2024-04-25T16:51:01.017648+0000","last_clean":"2024-04-25T16:51:01.017648+0000","last_became_active":"2024-04-25T16:50:54.036549+0000","last_became_peered":"2024-04-25T16:50:54.036549+0000","last_unstale":"2024-04-25T16:51:01.017648+0000","last_undegraded":"2024-04-25T16:51:01.017648+0000","last_fullsized":"2024-04-25T16:51:01.017648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:24.611706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018388+0000","last_change":"2024-04-25T16:50:59.175285+0000","last_active":"2024-04-25T16:51:01.018388+0000","last_peered":"2024-04-25T16:51:01.018388+0000","last_clean":"2024-04-25T16:51:01.018388+0000","last_became_active":"2024-04-25T16:50:59.174301+0000","last_became_peered":"2024-04-25T16:50:59.174301+0000","last_unstale":"2024-04-25T16:51:01.018388+0000","last_undegraded":"2024-04-25T16:51:01.018388+0000","last_fullsized":"2024-04-25T16:51:01.018388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:25.742371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019931+0000","last_change":"2024-04-25T16:50:55.076173+0000","last_active":"2024-04-25T16:51:01.019931+0000","last_peered":"2024-04-25T16:51:01.019931+0000","last_clean":"2024-04-25T16:51:01.019931+0000","last_became_active":"2024-04-25T16:50:55.075360+0000","last_became_peered":"2024-04-25T16:50:55.075360+0000","last_unstale":"2024-04-25T16:51:01.019931+0000","last_undegraded":"2024-04-25T16:51:01.019931+0000","last_fullsized":"2024-04-25T16:51:01.019931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:04.031520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:10.281685+0000","last_change":"2024-04-25T16:50:54.035115+0000","last_active":"2024-04-25T16:51:10.281685+0000","last_peered":"2024-04-25T16:51:10.281685+0000","last_clean":"2024-04-25T16:51:10.281685+0000","last_became_active":"2024-04-25T16:50:54.034629+0000","last_became_peered":"2024-04-25T16:50:54.034629+0000","last_unstale":"2024-04-25T16:51:10.281685+0000","last_undegraded":"2024-04-25T16:51:10.281685+0000","last_fullsized":"2024-04-25T16:51:10.281685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:35.077660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015930+0000","last_change":"2024-04-25T16:50:59.159337+0000","last_active":"2024-04-25T16:51:01.015930+0000","last_peered":"2024-04-25T16:51:01.015930+0000","last_clean":"2024-04-25T16:51:01.015930+0000","last_became_active":"2024-04-25T16:50:59.156207+0000","last_became_peered":"2024-04-25T16:50:59.156207+0000","last_unstale":"2024-04-25T16:51:01.015930+0000","last_undegraded":"2024-04-25T16:51:01.015930+0000","last_fullsized":"2024-04-25T16:51:01.015930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:39.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154043+0000","last_change":"2024-04-25T16:50:55.066237+0000","last_active":"2024-04-25T16:51:00.154043+0000","last_peered":"2024-04-25T16:51:00.154043+0000","last_clean":"2024-04-25T16:51:00.154043+0000","last_became_active":"2024-04-25T16:50:55.063200+0000","last_became_peered":"2024-04-25T16:50:55.063200+0000","last_unstale":"2024-04-25T16:51:00.154043+0000","last_undegraded":"2024-04-25T16:51:00.154043+0000","last_fullsized":"2024-04-25T16:51:00.154043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:22.053709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015709+0000","last_change":"2024-04-25T16:50:54.055594+0000","last_active":"2024-04-25T16:51:01.015709+0000","last_peered":"2024-04-25T16:51:01.015709+0000","last_clean":"2024-04-25T16:51:01.015709+0000","last_became_active":"2024-04-25T16:50:54.055027+0000","last_became_peered":"2024-04-25T16:50:54.055027+0000","last_unstale":"2024-04-25T16:51:01.015709+0000","last_undegraded":"2024-04-25T16:51:01.015709+0000","last_fullsized":"2024-04-25T16:51:01.015709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154662+0000","last_change":"2024-04-25T16:50:59.160116+0000","last_active":"2024-04-25T16:51:00.154662+0000","last_peered":"2024-04-25T16:51:00.154662+0000","last_clean":"2024-04-25T16:51:00.154662+0000","last_became_active":"2024-04-25T16:50:59.157923+0000","last_became_peered":"2024-04-25T16:50:59.157923+0000","last_unstale":"2024-04-25T16:51:00.154662+0000","last_undegraded":"2024-04-25T16:51:00.154662+0000","last_fullsized":"2024-04-25T16:51:00.154662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:46.167995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018271+0000","last_change":"2024-04-25T16:50:55.083898+0000","last_active":"2024-04-25T16:51:01.018271+0000","last_peered":"2024-04-25T16:51:01.018271+0000","last_clean":"2024-04-25T16:51:01.018271+0000","last_became_active":"2024-04-25T16:50:55.083533+0000","last_became_peered":"2024-04-25T16:50:55.083533+0000","last_unstale":"2024-04-25T16:51:01.018271+0000","last_undegraded":"2024-04-25T16:51:01.018271+0000","last_fullsized":"2024-04-25T16:51:01.018271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019088+0000","last_change":"2024-04-25T16:50:54.056906+0000","last_active":"2024-04-25T16:51:01.019088+0000","last_peered":"2024-04-25T16:51:01.019088+0000","last_clean":"2024-04-25T16:51:01.019088+0000","last_became_active":"2024-04-25T16:50:54.056102+0000","last_became_peered":"2024-04-25T16:50:54.056102+0000","last_unstale":"2024-04-25T16:51:01.019088+0000","last_undegraded":"2024-04-25T16:51:01.019088+0000","last_fullsized":"2024-04-25T16:51:01.019088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310494+0000","last_change":"2024-04-25T16:50:59.129868+0000","last_active":"2024-04-25T16:51:00.310494+0000","last_peered":"2024-04-25T16:51:00.310494+0000","last_clean":"2024-04-25T16:51:00.310494+0000","last_became_active":"2024-04-25T16:50:59.129447+0000","last_became_peered":"2024-04-25T16:50:59.129447+0000","last_unstale":"2024-04-25T16:51:00.310494+0000","last_undegraded":"2024-04-25T16:51:00.310494+0000","last_fullsized":"2024-04-25T16:51:00.310494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:07.826745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018797+0000","last_change":"2024-04-25T16:50:55.093338+0000","last_active":"2024-04-25T16:51:01.018797+0000","last_peered":"2024-04-25T16:51:01.018797+0000","last_clean":"2024-04-25T16:51:01.018797+0000","last_became_active":"2024-04-25T16:50:55.092985+0000","last_became_peered":"2024-04-25T16:50:55.092985+0000","last_unstale":"2024-04-25T16:51:01.018797+0000","last_undegraded":"2024-04-25T16:51:01.018797+0000","last_fullsized":"2024-04-25T16:51:01.018797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:03.704057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151183+0000","last_change":"2024-04-25T16:50:54.053862+0000","last_active":"2024-04-25T16:51:00.151183+0000","last_peered":"2024-04-25T16:51:00.151183+0000","last_clean":"2024-04-25T16:51:00.151183+0000","last_became_active":"2024-04-25T16:50:54.053156+0000","last_became_peered":"2024-04-25T16:50:54.053156+0000","last_unstale":"2024-04-25T16:51:00.151183+0000","last_undegraded":"2024-04-25T16:51:00.151183+0000","last_fullsized":"2024-04-25T16:51:00.151183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.690693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310744+0000","last_change":"2024-04-25T16:50:59.181080+0000","last_active":"2024-04-25T16:51:00.310744+0000","last_peered":"2024-04-25T16:51:00.310744+0000","last_clean":"2024-04-25T16:51:00.310744+0000","last_became_active":"2024-04-25T16:50:59.180638+0000","last_became_peered":"2024-04-25T16:50:59.180638+0000","last_unstale":"2024-04-25T16:51:00.310744+0000","last_undegraded":"2024-04-25T16:51:00.310744+0000","last_fullsized":"2024-04-25T16:51:00.310744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:47.904606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017640+0000","last_change":"2024-04-25T16:50:59.173382+0000","last_active":"2024-04-25T16:51:01.017640+0000","last_peered":"2024-04-25T16:51:01.017640+0000","last_clean":"2024-04-25T16:51:01.017640+0000","last_became_active":"2024-04-25T16:50:59.172921+0000","last_became_peered":"2024-04-25T16:50:59.172921+0000","last_unstale":"2024-04-25T16:51:01.017640+0000","last_undegraded":"2024-04-25T16:51:01.017640+0000","last_fullsized":"2024-04-25T16:51:01.017640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:39.408743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015213+0000","last_change":"2024-04-25T16:50:55.065015+0000","last_active":"2024-04-25T16:51:01.015213+0000","last_peered":"2024-04-25T16:51:01.015213+0000","last_clean":"2024-04-25T16:51:01.015213+0000","last_became_active":"2024-04-25T16:50:55.063038+0000","last_became_peered":"2024-04-25T16:50:55.063038+0000","last_unstale":"2024-04-25T16:51:01.015213+0000","last_undegraded":"2024-04-25T16:51:01.015213+0000","last_fullsized":"2024-04-25T16:51:01.015213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017330+0000","last_change":"2024-04-25T16:50:54.054787+0000","last_active":"2024-04-25T16:51:01.017330+0000","last_peered":"2024-04-25T16:51:01.017330+0000","last_clean":"2024-04-25T16:51:01.017330+0000","last_became_active":"2024-04-25T16:50:54.051984+0000","last_became_peered":"2024-04-25T16:50:54.051984+0000","last_unstale":"2024-04-25T16:51:01.017330+0000","last_undegraded":"2024-04-25T16:51:01.017330+0000","last_fullsized":"2024-04-25T16:51:01.017330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.166309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151032+0000","last_change":"2024-04-25T16:50:59.154675+0000","last_active":"2024-04-25T16:51:00.151032+0000","last_peered":"2024-04-25T16:51:00.151032+0000","last_clean":"2024-04-25T16:51:00.151032+0000","last_became_active":"2024-04-25T16:50:59.152969+0000","last_became_peered":"2024-04-25T16:50:59.152969+0000","last_unstale":"2024-04-25T16:51:00.151032+0000","last_undegraded":"2024-04-25T16:51:00.151032+0000","last_fullsized":"2024-04-25T16:51:00.151032+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.819554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310256+0000","last_change":"2024-04-25T16:50:55.069932+0000","last_active":"2024-04-25T16:51:00.310256+0000","last_peered":"2024-04-25T16:51:00.310256+0000","last_clean":"2024-04-25T16:51:00.310256+0000","last_became_active":"2024-04-25T16:50:55.060902+0000","last_became_peered":"2024-04-25T16:50:55.060902+0000","last_unstale":"2024-04-25T16:51:00.310256+0000","last_undegraded":"2024-04-25T16:51:00.310256+0000","last_fullsized":"2024-04-25T16:51:00.310256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310569+0000","last_change":"2024-04-25T16:50:54.043300+0000","last_active":"2024-04-25T16:51:00.310569+0000","last_peered":"2024-04-25T16:51:00.310569+0000","last_clean":"2024-04-25T16:51:00.310569+0000","last_became_active":"2024-04-25T16:50:54.042909+0000","last_became_peered":"2024-04-25T16:50:54.042909+0000","last_unstale":"2024-04-25T16:51:00.310569+0000","last_undegraded":"2024-04-25T16:51:00.310569+0000","last_fullsized":"2024-04-25T16:51:00.310569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:22.485560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149622+0000","last_change":"2024-04-25T16:50:54.044711+0000","last_active":"2024-04-25T16:51:00.149622+0000","last_peered":"2024-04-25T16:51:00.149622+0000","last_clean":"2024-04-25T16:51:00.149622+0000","last_became_active":"2024-04-25T16:50:54.043597+0000","last_became_peered":"2024-04-25T16:50:54.043597+0000","last_unstale":"2024-04-25T16:51:00.149622+0000","last_undegraded":"2024-04-25T16:51:00.149622+0000","last_fullsized":"2024-04-25T16:51:00.149622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.743061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017542+0000","last_change":"2024-04-25T16:50:55.061228+0000","last_active":"2024-04-25T16:51:01.017542+0000","last_peered":"2024-04-25T16:51:01.017542+0000","last_clean":"2024-04-25T16:51:01.017542+0000","last_became_active":"2024-04-25T16:50:55.060211+0000","last_became_peered":"2024-04-25T16:50:55.060211+0000","last_unstale":"2024-04-25T16:51:01.017542+0000","last_undegraded":"2024-04-25T16:51:01.017542+0000","last_fullsized":"2024-04-25T16:51:01.017542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.149479+0000","last_change":"2024-04-25T16:50:59.151109+0000","last_active":"2024-04-25T16:51:00.149479+0000","last_peered":"2024-04-25T16:51:00.149479+0000","last_clean":"2024-04-25T16:51:00.149479+0000","last_became_active":"2024-04-25T16:50:59.150443+0000","last_became_peered":"2024-04-25T16:50:59.150443+0000","last_unstale":"2024-04-25T16:51:00.149479+0000","last_undegraded":"2024-04-25T16:51:00.149479+0000","last_fullsized":"2024-04-25T16:51:00.149479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:45.414606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310730+0000","last_change":"2024-04-25T16:50:54.047737+0000","last_active":"2024-04-25T16:51:00.310730+0000","last_peered":"2024-04-25T16:51:00.310730+0000","last_clean":"2024-04-25T16:51:00.310730+0000","last_became_active":"2024-04-25T16:50:54.047417+0000","last_became_peered":"2024-04-25T16:50:54.047417+0000","last_unstale":"2024-04-25T16:51:00.310730+0000","last_undegraded":"2024-04-25T16:51:00.310730+0000","last_fullsized":"2024-04-25T16:51:00.310730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:52.993286+0000","last_clean_scrub_stamp":"2024-04-25T16:50:52.993286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.166309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.153448+0000","last_change":"2024-04-25T16:50:55.064168+0000","last_active":"2024-04-25T16:51:00.153448+0000","last_peered":"2024-04-25T16:51:00.153448+0000","last_clean":"2024-04-25T16:51:00.153448+0000","last_became_active":"2024-04-25T16:50:55.062345+0000","last_became_peered":"2024-04-25T16:50:55.062345+0000","last_unstale":"2024-04-25T16:51:00.153448+0000","last_undegraded":"2024-04-25T16:51:00.153448+0000","last_fullsized":"2024-04-25T16:51:00.153448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:09.018645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311446+0000","last_change":"2024-04-25T16:50:59.143004+0000","last_active":"2024-04-25T16:51:00.311446+0000","last_peered":"2024-04-25T16:51:00.311446+0000","last_clean":"2024-04-25T16:51:00.311446+0000","last_became_active":"2024-04-25T16:50:59.142240+0000","last_became_peered":"2024-04-25T16:50:59.142240+0000","last_unstale":"2024-04-25T16:51:00.311446+0000","last_undegraded":"2024-04-25T16:51:00.311446+0000","last_fullsized":"2024-04-25T16:51:00.311446+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:58.089919+0000","last_clean_scrub_stamp":"2024-04-25T16:50:58.089919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:19.230775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148672+0000","last_change":"2024-04-25T16:50:55.066558+0000","last_active":"2024-04-25T16:51:00.148672+0000","last_peered":"2024-04-25T16:51:00.148672+0000","last_clean":"2024-04-25T16:51:00.148672+0000","last_became_active":"2024-04-25T16:50:55.065201+0000","last_became_peered":"2024-04-25T16:50:55.065201+0000","last_unstale":"2024-04-25T16:51:00.148672+0000","last_undegraded":"2024-04-25T16:51:00.148672+0000","last_fullsized":"2024-04-25T16:51:00.148672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:12.518641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.148555+0000","last_change":"2024-04-25T16:50:55.089086+0000","last_active":"2024-04-25T16:51:00.148555+0000","last_peered":"2024-04-25T16:51:00.148555+0000","last_clean":"2024-04-25T16:51:00.148555+0000","last_became_active":"2024-04-25T16:50:55.088788+0000","last_became_peered":"2024-04-25T16:50:55.088788+0000","last_unstale":"2024-04-25T16:51:00.148555+0000","last_undegraded":"2024-04-25T16:51:00.148555+0000","last_fullsized":"2024-04-25T16:51:00.148555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:22.053709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019230+0000","last_change":"2024-04-25T16:50:55.092706+0000","last_active":"2024-04-25T16:51:01.019230+0000","last_peered":"2024-04-25T16:51:01.019230+0000","last_clean":"2024-04-25T16:51:01.019230+0000","last_became_active":"2024-04-25T16:50:55.092411+0000","last_became_peered":"2024-04-25T16:50:55.092411+0000","last_unstale":"2024-04-25T16:51:01.019230+0000","last_undegraded":"2024-04-25T16:51:01.019230+0000","last_fullsized":"2024-04-25T16:51:01.019230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:56.360832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017429+0000","last_change":"2024-04-25T16:50:55.053386+0000","last_active":"2024-04-25T16:51:01.017429+0000","last_peered":"2024-04-25T16:51:01.017429+0000","last_clean":"2024-04-25T16:51:01.017429+0000","last_became_active":"2024-04-25T16:50:55.052555+0000","last_became_peered":"2024-04-25T16:50:55.052555+0000","last_unstale":"2024-04-25T16:51:01.017429+0000","last_undegraded":"2024-04-25T16:51:01.017429+0000","last_fullsized":"2024-04-25T16:51:01.017429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:31.376803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.311617+0000","last_change":"2024-04-25T16:50:55.072013+0000","last_active":"2024-04-25T16:51:00.311617+0000","last_peered":"2024-04-25T16:51:00.311617+0000","last_clean":"2024-04-25T16:51:00.311617+0000","last_became_active":"2024-04-25T16:50:55.071089+0000","last_became_peered":"2024-04-25T16:50:55.071089+0000","last_unstale":"2024-04-25T16:51:00.311617+0000","last_undegraded":"2024-04-25T16:51:00.311617+0000","last_fullsized":"2024-04-25T16:51:00.311617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.304249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015430+0000","last_change":"2024-04-25T16:50:55.063809+0000","last_active":"2024-04-25T16:51:01.015430+0000","last_peered":"2024-04-25T16:51:01.015430+0000","last_clean":"2024-04-25T16:51:01.015430+0000","last_became_active":"2024-04-25T16:50:55.062913+0000","last_became_peered":"2024-04-25T16:50:55.062913+0000","last_unstale":"2024-04-25T16:51:01.015430+0000","last_undegraded":"2024-04-25T16:51:01.015430+0000","last_fullsized":"2024-04-25T16:51:01.015430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151698+0000","last_change":"2024-04-25T16:50:55.066745+0000","last_active":"2024-04-25T16:51:00.151698+0000","last_peered":"2024-04-25T16:51:00.151698+0000","last_clean":"2024-04-25T16:51:00.151698+0000","last_became_active":"2024-04-25T16:50:55.065075+0000","last_became_peered":"2024-04-25T16:50:55.065075+0000","last_unstale":"2024-04-25T16:51:00.151698+0000","last_undegraded":"2024-04-25T16:51:00.151698+0000","last_fullsized":"2024-04-25T16:51:00.151698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.312312+0000","last_change":"2024-04-25T16:50:55.081262+0000","last_active":"2024-04-25T16:51:00.312312+0000","last_peered":"2024-04-25T16:51:00.312312+0000","last_clean":"2024-04-25T16:51:00.312312+0000","last_became_active":"2024-04-25T16:50:55.080803+0000","last_became_peered":"2024-04-25T16:50:55.080803+0000","last_unstale":"2024-04-25T16:51:00.312312+0000","last_undegraded":"2024-04-25T16:51:00.312312+0000","last_fullsized":"2024-04-25T16:51:00.312312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:14.342043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017299+0000","last_change":"2024-04-25T16:50:55.090717+0000","last_active":"2024-04-25T16:51:01.017299+0000","last_peered":"2024-04-25T16:51:01.017299+0000","last_clean":"2024-04-25T16:51:01.017299+0000","last_became_active":"2024-04-25T16:50:55.090388+0000","last_became_peered":"2024-04-25T16:50:55.090388+0000","last_unstale":"2024-04-25T16:51:01.017299+0000","last_undegraded":"2024-04-25T16:51:01.017299+0000","last_fullsized":"2024-04-25T16:51:01.017299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310335+0000","last_change":"2024-04-25T16:50:55.072773+0000","last_active":"2024-04-25T16:51:00.310335+0000","last_peered":"2024-04-25T16:51:00.310335+0000","last_clean":"2024-04-25T16:51:00.310335+0000","last_became_active":"2024-04-25T16:50:55.071914+0000","last_became_peered":"2024-04-25T16:50:55.071914+0000","last_unstale":"2024-04-25T16:51:00.310335+0000","last_undegraded":"2024-04-25T16:51:00.310335+0000","last_fullsized":"2024-04-25T16:51:00.310335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018758+0000","last_change":"2024-04-25T16:50:55.093501+0000","last_active":"2024-04-25T16:51:01.018758+0000","last_peered":"2024-04-25T16:51:01.018758+0000","last_clean":"2024-04-25T16:51:01.018758+0000","last_became_active":"2024-04-25T16:50:55.093168+0000","last_became_peered":"2024-04-25T16:50:55.093168+0000","last_unstale":"2024-04-25T16:51:01.018758+0000","last_undegraded":"2024-04-25T16:51:01.018758+0000","last_fullsized":"2024-04-25T16:51:01.018758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:42.081728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017137+0000","last_change":"2024-04-25T16:50:55.066877+0000","last_active":"2024-04-25T16:51:01.017137+0000","last_peered":"2024-04-25T16:51:01.017137+0000","last_clean":"2024-04-25T16:51:01.017137+0000","last_became_active":"2024-04-25T16:50:55.066129+0000","last_became_peered":"2024-04-25T16:50:55.066129+0000","last_unstale":"2024-04-25T16:51:01.017137+0000","last_undegraded":"2024-04-25T16:51:01.017137+0000","last_fullsized":"2024-04-25T16:51:01.017137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:12.518641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150236+0000","last_change":"2024-04-25T16:50:55.052920+0000","last_active":"2024-04-25T16:51:00.150236+0000","last_peered":"2024-04-25T16:51:00.150236+0000","last_clean":"2024-04-25T16:51:00.150236+0000","last_became_active":"2024-04-25T16:50:55.047735+0000","last_became_peered":"2024-04-25T16:50:55.047735+0000","last_unstale":"2024-04-25T16:51:00.150236+0000","last_undegraded":"2024-04-25T16:51:00.150236+0000","last_fullsized":"2024-04-25T16:51:00.150236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020445+0000","last_change":"2024-04-25T16:50:55.090598+0000","last_active":"2024-04-25T16:51:01.020445+0000","last_peered":"2024-04-25T16:51:01.020445+0000","last_clean":"2024-04-25T16:51:01.020445+0000","last_became_active":"2024-04-25T16:50:55.090102+0000","last_became_peered":"2024-04-25T16:50:55.090102+0000","last_unstale":"2024-04-25T16:51:01.020445+0000","last_undegraded":"2024-04-25T16:51:01.020445+0000","last_fullsized":"2024-04-25T16:51:01.020445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:35.322477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150315+0000","last_change":"2024-04-25T16:50:55.081066+0000","last_active":"2024-04-25T16:51:00.150315+0000","last_peered":"2024-04-25T16:51:00.150315+0000","last_clean":"2024-04-25T16:51:00.150315+0000","last_became_active":"2024-04-25T16:50:55.069516+0000","last_became_peered":"2024-04-25T16:50:55.069516+0000","last_unstale":"2024-04-25T16:51:00.150315+0000","last_undegraded":"2024-04-25T16:51:00.150315+0000","last_fullsized":"2024-04-25T16:51:00.150315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:13.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.019657+0000","last_change":"2024-04-25T16:50:55.093971+0000","last_active":"2024-04-25T16:51:01.019657+0000","last_peered":"2024-04-25T16:51:01.019657+0000","last_clean":"2024-04-25T16:51:01.019657+0000","last_became_active":"2024-04-25T16:50:55.093421+0000","last_became_peered":"2024-04-25T16:50:55.093421+0000","last_unstale":"2024-04-25T16:51:01.019657+0000","last_undegraded":"2024-04-25T16:51:01.019657+0000","last_fullsized":"2024-04-25T16:51:01.019657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:10.733287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016967+0000","last_change":"2024-04-25T16:50:55.075149+0000","last_active":"2024-04-25T16:51:01.016967+0000","last_peered":"2024-04-25T16:51:01.016967+0000","last_clean":"2024-04-25T16:51:01.016967+0000","last_became_active":"2024-04-25T16:50:55.074215+0000","last_became_peered":"2024-04-25T16:50:55.074215+0000","last_unstale":"2024-04-25T16:51:01.016967+0000","last_undegraded":"2024-04-25T16:51:01.016967+0000","last_fullsized":"2024-04-25T16:51:01.016967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.015476+0000","last_change":"2024-04-25T16:50:55.072156+0000","last_active":"2024-04-25T16:51:01.015476+0000","last_peered":"2024-04-25T16:51:01.015476+0000","last_clean":"2024-04-25T16:51:01.015476+0000","last_became_active":"2024-04-25T16:50:55.071295+0000","last_became_peered":"2024-04-25T16:50:55.071295+0000","last_unstale":"2024-04-25T16:51:01.015476+0000","last_undegraded":"2024-04-25T16:51:01.015476+0000","last_fullsized":"2024-04-25T16:51:01.015476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:04.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018336+0000","last_change":"2024-04-25T16:50:55.062605+0000","last_active":"2024-04-25T16:51:01.018336+0000","last_peered":"2024-04-25T16:51:01.018336+0000","last_clean":"2024-04-25T16:51:01.018336+0000","last_became_active":"2024-04-25T16:50:55.061970+0000","last_became_peered":"2024-04-25T16:50:55.061970+0000","last_unstale":"2024-04-25T16:51:01.018336+0000","last_undegraded":"2024-04-25T16:51:01.018336+0000","last_fullsized":"2024-04-25T16:51:01.018336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:41.328339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.310463+0000","last_change":"2024-04-25T16:50:55.062534+0000","last_active":"2024-04-25T16:51:00.310463+0000","last_peered":"2024-04-25T16:51:00.310463+0000","last_clean":"2024-04-25T16:51:00.310463+0000","last_became_active":"2024-04-25T16:50:55.061601+0000","last_became_peered":"2024-04-25T16:50:55.061601+0000","last_unstale":"2024-04-25T16:51:00.310463+0000","last_undegraded":"2024-04-25T16:51:00.310463+0000","last_fullsized":"2024-04-25T16:51:00.310463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.154054+0000","last_change":"2024-04-25T16:50:55.067043+0000","last_active":"2024-04-25T16:51:00.154054+0000","last_peered":"2024-04-25T16:51:00.154054+0000","last_clean":"2024-04-25T16:51:00.154054+0000","last_became_active":"2024-04-25T16:50:55.066155+0000","last_became_peered":"2024-04-25T16:50:55.066155+0000","last_unstale":"2024-04-25T16:51:00.154054+0000","last_undegraded":"2024-04-25T16:51:00.154054+0000","last_fullsized":"2024-04-25T16:51:00.154054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:08.701059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.150829+0000","last_change":"2024-04-25T16:50:55.084655+0000","last_active":"2024-04-25T16:51:00.150829+0000","last_peered":"2024-04-25T16:51:00.150829+0000","last_clean":"2024-04-25T16:51:00.150829+0000","last_became_active":"2024-04-25T16:50:55.083647+0000","last_became_peered":"2024-04-25T16:50:55.083647+0000","last_unstale":"2024-04-25T16:51:00.150829+0000","last_undegraded":"2024-04-25T16:51:00.150829+0000","last_fullsized":"2024-04-25T16:51:00.150829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.151431+0000","last_change":"2024-04-25T16:50:55.085352+0000","last_active":"2024-04-25T16:51:00.151431+0000","last_peered":"2024-04-25T16:51:00.151431+0000","last_clean":"2024-04-25T16:51:00.151431+0000","last_became_active":"2024-04-25T16:50:55.083135+0000","last_became_peered":"2024-04-25T16:50:55.083135+0000","last_unstale":"2024-04-25T16:51:00.151431+0000","last_undegraded":"2024-04-25T16:51:00.151431+0000","last_fullsized":"2024-04-25T16:51:00.151431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:07.636359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.021207+0000","last_change":"2024-04-25T16:50:55.090981+0000","last_active":"2024-04-25T16:51:01.021207+0000","last_peered":"2024-04-25T16:51:01.021207+0000","last_clean":"2024-04-25T16:51:01.021207+0000","last_became_active":"2024-04-25T16:50:55.090507+0000","last_became_peered":"2024-04-25T16:50:55.090507+0000","last_unstale":"2024-04-25T16:51:01.021207+0000","last_undegraded":"2024-04-25T16:51:01.021207+0000","last_fullsized":"2024-04-25T16:51:01.021207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:10.733287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:00.309934+0000","last_change":"2024-04-25T16:50:55.090738+0000","last_active":"2024-04-25T16:51:00.309934+0000","last_peered":"2024-04-25T16:51:00.309934+0000","last_clean":"2024-04-25T16:51:00.309934+0000","last_became_active":"2024-04-25T16:50:55.090298+0000","last_became_peered":"2024-04-25T16:50:55.090298+0000","last_unstale":"2024-04-25T16:51:00.309934+0000","last_undegraded":"2024-04-25T16:51:00.309934+0000","last_fullsized":"2024-04-25T16:51:00.309934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:12.518641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017349+0000","last_change":"2024-04-25T16:50:55.083767+0000","last_active":"2024-04-25T16:51:01.017349+0000","last_peered":"2024-04-25T16:51:01.017349+0000","last_clean":"2024-04-25T16:51:01.017349+0000","last_became_active":"2024-04-25T16:50:55.083468+0000","last_became_peered":"2024-04-25T16:50:55.083468+0000","last_unstale":"2024-04-25T16:51:01.017349+0000","last_undegraded":"2024-04-25T16:51:01.017349+0000","last_fullsized":"2024-04-25T16:51:01.017349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.304249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018383+0000","last_change":"2024-04-25T16:50:55.083971+0000","last_active":"2024-04-25T16:51:01.018383+0000","last_peered":"2024-04-25T16:51:01.018383+0000","last_clean":"2024-04-25T16:51:01.018383+0000","last_became_active":"2024-04-25T16:50:55.083688+0000","last_became_peered":"2024-04-25T16:50:55.083688+0000","last_unstale":"2024-04-25T16:51:01.018383+0000","last_undegraded":"2024-04-25T16:51:01.018383+0000","last_fullsized":"2024-04-25T16:51:01.018383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:08.427466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018722+0000","last_change":"2024-04-25T16:50:55.090861+0000","last_active":"2024-04-25T16:51:01.018722+0000","last_peered":"2024-04-25T16:51:01.018722+0000","last_clean":"2024-04-25T16:51:01.018722+0000","last_became_active":"2024-04-25T16:50:55.089863+0000","last_became_peered":"2024-04-25T16:50:55.089863+0000","last_unstale":"2024-04-25T16:51:01.018722+0000","last_undegraded":"2024-04-25T16:51:01.018722+0000","last_fullsized":"2024-04-25T16:51:01.018722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:14.342043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.018454+0000","last_change":"2024-04-25T16:50:55.091645+0000","last_active":"2024-04-25T16:51:01.018454+0000","last_peered":"2024-04-25T16:51:01.018454+0000","last_clean":"2024-04-25T16:51:01.018454+0000","last_became_active":"2024-04-25T16:50:55.091308+0000","last_became_peered":"2024-04-25T16:50:55.091308+0000","last_unstale":"2024-04-25T16:51:01.018454+0000","last_undegraded":"2024-04-25T16:51:01.018454+0000","last_fullsized":"2024-04-25T16:51:01.018454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:16.117653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.017510+0000","last_change":"2024-04-25T16:50:55.074041+0000","last_active":"2024-04-25T16:51:01.017510+0000","last_peered":"2024-04-25T16:51:01.017510+0000","last_clean":"2024-04-25T16:51:01.017510+0000","last_became_active":"2024-04-25T16:50:55.073142+0000","last_became_peered":"2024-04-25T16:50:55.073142+0000","last_unstale":"2024-04-25T16:51:01.017510+0000","last_undegraded":"2024-04-25T16:51:01.017510+0000","last_fullsized":"2024-04-25T16:51:01.017510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.219061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.016248+0000","last_change":"2024-04-25T16:50:55.065163+0000","last_active":"2024-04-25T16:51:01.016248+0000","last_peered":"2024-04-25T16:51:01.016248+0000","last_clean":"2024-04-25T16:51:01.016248+0000","last_became_active":"2024-04-25T16:50:55.064257+0000","last_became_peered":"2024-04-25T16:50:55.064257+0000","last_unstale":"2024-04-25T16:51:01.016248+0000","last_undegraded":"2024-04-25T16:51:01.016248+0000","last_fullsized":"2024-04-25T16:51:01.016248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.621085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T16:51:01.020950+0000","last_change":"2024-04-25T16:50:55.082932+0000","last_active":"2024-04-25T16:51:01.020950+0000","last_peered":"2024-04-25T16:51:01.020950+0000","last_clean":"2024-04-25T16:51:01.020950+0000","last_became_active":"2024-04-25T16:50:55.082589+0000","last_became_peered":"2024-04-25T16:50:55.082589+0000","last_unstale":"2024-04-25T16:51:01.020950+0000","last_undegraded":"2024-04-25T16:51:01.020950+0000","last_fullsized":"2024-04-25T16:51:01.020950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:54.003652+0000","last_clean_scrub_stamp":"2024-04-25T16:50:54.003652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:56.360832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335656,"statfs":{"total":96636764160,"available":96599711744,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:51:24.881 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T16:51:24.881 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T16:51:24.881 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T16:51:24.881 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T16:51:25.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b3d938640 1 Processor -- start 2024-04-25T16:51:25.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b3d938640 1 -- start start 2024-04-25T16:51:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b3d938640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4b3815da80 0x7f4b380d2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b3d938640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4b380d2570 0x7f4b38070220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b3d938640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b380d2980 con 0x7f4b38070760 2024-04-25T16:51:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b3d938640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f4b380cf350 con 0x7f4b380d2570 2024-04-25T16:51:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b3d938640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4b3815de90 con 0x7f4b3815da80 2024-04-25T16:51:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b367fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4b380d2570 0x7f4b38070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b367fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4b380d2570 0x7f4b38070220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42484/0 (socket says 172.21.15.94:42484) 2024-04-25T16:51:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b367fc640 1 -- 172.21.15.94:0/1711286503 learned_addr learned my addr 172.21.15.94:0/1711286503 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b36ffd640 1 --2- 172.21.15.94:0/1711286503 >> v2:172.21.15.138:3300/0 conn(0x7f4b3815da80 0x7f4b380d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b367fc640 1 -- 172.21.15.94:0/1711286503 >> v2:172.21.15.138:3300/0 conn(0x7f4b3815da80 msgr2=0x7f4b380d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b367fc640 1 --2- 172.21.15.94:0/1711286503 >> v2:172.21.15.138:3300/0 conn(0x7f4b3815da80 0x7f4b380d2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b367fc640 1 -- 172.21.15.94:0/1711286503 >> v1:172.21.15.138:6789/0 conn(0x7f4b38070760 legacy=0x7f4b38070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:25.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.064+0000 7f4b367fc640 1 -- 172.21.15.94:0/1711286503 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b380b7020 con 0x7f4b380d2570 2024-04-25T16:51:25.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.068+0000 7f4b367fc640 1 --2- 172.21.15.94:0/1711286503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b380d2570 0x7f4b38070220 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b380d0ab0 tx=0x7f4b2001c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a852ed39194423 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:25.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.068+0000 7f4b35ffb640 1 -- 172.21.15.94:0/1711286503 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b200b0070 con 0x7f4b380d2570 2024-04-25T16:51:25.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.068+0000 7f4b35ffb640 1 -- 172.21.15.94:0/1711286503 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b200ab600 con 0x7f4b380d2570 2024-04-25T16:51:25.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.068+0000 7f4b35ffb640 1 -- 172.21.15.94:0/1711286503 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b200ab9d0 con 0x7f4b380d2570 2024-04-25T16:51:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.068+0000 7f4b3d938640 1 -- 172.21.15.94:0/1711286503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b380d2570 msgr2=0x7f4b38070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.068+0000 7f4b3d938640 1 --2- 172.21.15.94:0/1711286503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b380d2570 0x7f4b38070220 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b380d0ab0 tx=0x7f4b2001c2d0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.068+0000 7f4b3d938640 1 -- 172.21.15.94:0/1711286503 shutdown_connections 2024-04-25T16:51:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.068+0000 7f4b3d938640 1 --2- 172.21.15.94:0/1711286503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b380d2570 0x7f4b38070220 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.068+0000 7f4b3d938640 1 --2- 172.21.15.94:0/1711286503 >> v2:172.21.15.138:3300/0 conn(0x7f4b3815da80 0x7f4b380d2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.068+0000 7f4b3d938640 1 -- 172.21.15.94:0/1711286503 >> 172.21.15.94:0/1711286503 conn(0x7f4b380cfed0 msgr2=0x7f4b380d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.068+0000 7f4b3d938640 1 -- 172.21.15.94:0/1711286503 shutdown_connections 2024-04-25T16:51:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.068+0000 7f4b3d938640 1 -- 172.21.15.94:0/1711286503 wait complete. 2024-04-25T16:51:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b3d938640 1 Processor -- start 2024-04-25T16:51:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b3d938640 1 -- start start 2024-04-25T16:51:25.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b3d938640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b38070760 0x7f4b381528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b3d938640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b380d2570 0x7f4b380cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b3d938640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3815da80 0x7f4b380cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b3d938640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4b38071280 con 0x7f4b3815da80 2024-04-25T16:51:25.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b3d938640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b380b6cd0 con 0x7f4b38070760 2024-04-25T16:51:25.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b3d938640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4b380b80c0 con 0x7f4b380d2570 2024-04-25T16:51:25.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3815da80 0x7f4b380cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3815da80 0x7f4b380cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42498/0 (socket says 172.21.15.94:42498) 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b377fe640 1 -- 172.21.15.94:0/1956944896 learned_addr learned my addr 172.21.15.94:0/1956944896 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b36ffd640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b38070760 0x7f4b381528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b377fe640 1 -- 172.21.15.94:0/1956944896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b380d2570 msgr2=0x7f4b380cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b377fe640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b380d2570 0x7f4b380cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b377fe640 1 -- 172.21.15.94:0/1956944896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b38070760 msgr2=0x7f4b381528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b377fe640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b38070760 0x7f4b381528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b377fe640 1 -- 172.21.15.94:0/1956944896 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b380c9c50 con 0x7f4b3815da80 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b36ffd640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b38070760 0x7f4b381528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b377fe640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3815da80 0x7f4b380cdf30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800ba10 tx=0x7f4b2800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b2ffff640 1 -- 172.21.15.94:0/1956944896 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28019070 con 0x7f4b3815da80 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b2ffff640 1 -- 172.21.15.94:0/1956944896 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b2800cba0 con 0x7f4b3815da80 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b2ffff640 1 -- 172.21.15.94:0/1956944896 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28014440 con 0x7f4b3815da80 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b3d938640 1 -- 172.21.15.94:0/1956944896 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b380ca720 con 0x7f4b3815da80 2024-04-25T16:51:25.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b3d938640 1 -- 172.21.15.94:0/1956944896 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b380cbfa0 con 0x7f4b3815da80 2024-04-25T16:51:25.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.072+0000 7f4b3d938640 1 -- 172.21.15.94:0/1956944896 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b381532e0 con 0x7f4b3815da80 2024-04-25T16:51:25.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.076+0000 7f4b2ffff640 1 -- 172.21.15.94:0/1956944896 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4b280145e0 con 0x7f4b3815da80 2024-04-25T16:51:25.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.076+0000 7f4b2ffff640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4b1806f430 0x7f4b18071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.076+0000 7f4b2ffff640 1 -- 172.21.15.94:0/1956944896 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4b2808f250 con 0x7f4b3815da80 2024-04-25T16:51:25.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.076+0000 7f4b36ffd640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4b1806f430 0x7f4b18071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.076+0000 7f4b36ffd640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4b1806f430 0x7f4b18071940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4b30006fd0 tx=0x7f4b30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:25.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.080+0000 7f4b2ffff640 1 -- 172.21.15.94:0/1956944896 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4b2805be00 con 0x7f4b3815da80 2024-04-25T16:51:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.260+0000 7f4b3d938640 1 -- 172.21.15.94:0/1956944896 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4b38076830 con 0x7f4b3815da80 2024-04-25T16:51:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.264+0000 7f4b2ffff640 1 -- 172.21.15.94:0/1956944896 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f4b2805b700 con 0x7f4b3815da80 2024-04-25T16:51:25.260 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:25.260 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T16:51:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.264+0000 7f4b3d938640 1 -- 172.21.15.94:0/1956944896 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4b1806f430 msgr2=0x7f4b18071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.264+0000 7f4b3d938640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4b1806f430 0x7f4b18071940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4b30006fd0 tx=0x7f4b30008040 comp rx=0 tx=0).stop 2024-04-25T16:51:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.264+0000 7f4b3d938640 1 -- 172.21.15.94:0/1956944896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3815da80 msgr2=0x7f4b380cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.264+0000 7f4b3d938640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3815da80 0x7f4b380cdf30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800ba10 tx=0x7f4b2800bed0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.264+0000 7f4b3d938640 1 -- 172.21.15.94:0/1956944896 shutdown_connections 2024-04-25T16:51:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.264+0000 7f4b3d938640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3815da80 0x7f4b380cdf30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.264+0000 7f4b3d938640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4b1806f430 0x7f4b18071940 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.264+0000 7f4b3d938640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b380d2570 0x7f4b380cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.264+0000 7f4b3d938640 1 --2- 172.21.15.94:0/1956944896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b38070760 0x7f4b381528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.264+0000 7f4b3d938640 1 -- 172.21.15.94:0/1956944896 >> 172.21.15.94:0/1956944896 conn(0x7f4b380cfed0 msgr2=0x7f4b380b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.264+0000 7f4b3d938640 1 -- 172.21.15.94:0/1956944896 shutdown_connections 2024-04-25T16:51:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.264+0000 7f4b3d938640 1 -- 172.21.15.94:0/1956944896 wait complete. 2024-04-25T16:51:25.280 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T16:51:25.281 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:25.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.416+0000 7f3455413640 1 Processor -- start 2024-04-25T16:51:25.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f3455413640 1 -- start start 2024-04-25T16:51:25.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f3455413640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3450070180 0x7f345015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f3455413640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f345006de20 0x7f34500af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f3455413640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f345015d620 con 0x7f3450160bf0 2024-04-25T16:51:25.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f3455413640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f345015dec0 con 0x7f3450070180 2024-04-25T16:51:25.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f3455413640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3450160760 con 0x7f345006de20 2024-04-25T16:51:25.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f344effd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3450070180 0x7f345015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f344effd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3450070180 0x7f345015cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42510/0 (socket says 172.21.15.94:42510) 2024-04-25T16:51:25.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f344effd640 1 -- 172.21.15.94:0/1357276713 learned_addr learned my addr 172.21.15.94:0/1357276713 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:25.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f344f7fe640 1 --2- 172.21.15.94:0/1357276713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f345006de20 0x7f34500af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f344effd640 1 -- 172.21.15.94:0/1357276713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f345006de20 msgr2=0x7f34500af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:25.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f344effd640 1 --2- 172.21.15.94:0/1357276713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f345006de20 0x7f34500af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f344effd640 1 -- 172.21.15.94:0/1357276713 >> v1:172.21.15.94:6789/0 conn(0x7f3450160bf0 legacy=0x7f345006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f344effd640 1 -- 172.21.15.94:0/1357276713 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345015d900 con 0x7f3450070180 2024-04-25T16:51:25.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.420+0000 7f344f7fe640 1 --2- 172.21.15.94:0/1357276713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f345006de20 0x7f34500af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:25.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f344effd640 1 --2- 172.21.15.94:0/1357276713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3450070180 0x7f345015cf40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3440009870 tx=0x7f344001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea12bb553f5eaa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:25.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f344e7fc640 1 -- 172.21.15.94:0/1357276713 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34400ab640 con 0x7f3450070180 2024-04-25T16:51:25.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f344e7fc640 1 -- 172.21.15.94:0/1357276713 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34400b0070 con 0x7f3450070180 2024-04-25T16:51:25.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f344e7fc640 1 -- 172.21.15.94:0/1357276713 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34400aba00 con 0x7f3450070180 2024-04-25T16:51:25.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f3455413640 1 -- 172.21.15.94:0/1357276713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3450070180 msgr2=0x7f345015cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f3455413640 1 --2- 172.21.15.94:0/1357276713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3450070180 0x7f345015cf40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3440009870 tx=0x7f344001c530 comp rx=0 tx=0).stop 2024-04-25T16:51:25.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f3455413640 1 -- 172.21.15.94:0/1357276713 shutdown_connections 2024-04-25T16:51:25.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f3455413640 1 --2- 172.21.15.94:0/1357276713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f345006de20 0x7f34500af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f3455413640 1 --2- 172.21.15.94:0/1357276713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3450070180 0x7f345015cf40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f3455413640 1 -- 172.21.15.94:0/1357276713 >> 172.21.15.94:0/1357276713 conn(0x7f34500b47f0 msgr2=0x7f34500b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:25.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f3455413640 1 -- 172.21.15.94:0/1357276713 shutdown_connections 2024-04-25T16:51:25.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f3455413640 1 -- 172.21.15.94:0/1357276713 wait complete. 2024-04-25T16:51:25.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f3455413640 1 Processor -- start 2024-04-25T16:51:25.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.424+0000 7f3455413640 1 -- start start 2024-04-25T16:51:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3455413640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3450070180 0x7f345014ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3455413640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3450160bf0 0x7f3450152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3455413640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f345009c870 0x7f345009cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3455413640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f345016a260 con 0x7f3450160bf0 2024-04-25T16:51:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3455413640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f345016a0e0 con 0x7f3450070180 2024-04-25T16:51:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3455413640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f345016a3e0 con 0x7f345009c870 2024-04-25T16:51:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3447fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3450160bf0 0x7f3450152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3447fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3450160bf0 0x7f3450152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42514/0 (socket says 172.21.15.94:42514) 2024-04-25T16:51:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3447fff640 1 -- 172.21.15.94:0/2601271548 learned_addr learned my addr 172.21.15.94:0/2601271548 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3447fff640 1 -- 172.21.15.94:0/2601271548 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f345009c870 msgr2=0x7f345009cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f344effd640 1 --2- 172.21.15.94:0/2601271548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3450070180 0x7f345014ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3447fff640 1 --2- 172.21.15.94:0/2601271548 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f345009c870 0x7f345009cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3447fff640 1 -- 172.21.15.94:0/2601271548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3450070180 msgr2=0x7f345014ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3447fff640 1 --2- 172.21.15.94:0/2601271548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3450070180 0x7f345014ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3447fff640 1 -- 172.21.15.94:0/2601271548 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345009d1c0 con 0x7f3450160bf0 2024-04-25T16:51:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3447fff640 1 --2- 172.21.15.94:0/2601271548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3450160bf0 0x7f3450152040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f343800c3a0 tx=0x7f343800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f34477fe640 1 -- 172.21.15.94:0/2601271548 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34380052d0 con 0x7f3450160bf0 2024-04-25T16:51:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f34477fe640 1 -- 172.21.15.94:0/2601271548 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3438005470 con 0x7f3450160bf0 2024-04-25T16:51:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3455413640 1 -- 172.21.15.94:0/2601271548 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345014f110 con 0x7f3450160bf0 2024-04-25T16:51:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f34477fe640 1 -- 172.21.15.94:0/2601271548 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438007260 con 0x7f3450160bf0 2024-04-25T16:51:25.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3455413640 1 -- 172.21.15.94:0/2601271548 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345009b710 con 0x7f3450160bf0 2024-04-25T16:51:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f3455413640 1 -- 172.21.15.94:0/2601271548 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3450076830 con 0x7f3450160bf0 2024-04-25T16:51:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f34477fe640 1 -- 172.21.15.94:0/2601271548 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3438007400 con 0x7f3450160bf0 2024-04-25T16:51:25.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.428+0000 7f34477fe640 1 --2- 172.21.15.94:0/2601271548 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f343406f430 0x7f3434071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.432+0000 7f34477fe640 1 -- 172.21.15.94:0/2601271548 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f343808eb00 con 0x7f3450160bf0 2024-04-25T16:51:25.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.432+0000 7f344effd640 1 --2- 172.21.15.94:0/2601271548 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f343406f430 0x7f3434071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.432+0000 7f344effd640 1 --2- 172.21.15.94:0/2601271548 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f343406f430 0x7f3434071940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f34400af000 tx=0x7f3440008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:25.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.436+0000 7f34477fe640 1 -- 172.21.15.94:0/2601271548 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f345009d1c0 con 0x7f3450160bf0 2024-04-25T16:51:25.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.584+0000 7f3455413640 1 -- 172.21.15.94:0/2601271548 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f345006fa50 con 0x7f3450160bf0 2024-04-25T16:51:25.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.588+0000 7f34477fe640 1 -- 172.21.15.94:0/2601271548 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f343805b630 con 0x7f3450160bf0 2024-04-25T16:51:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:25.586 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:59.528743+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:25.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.588+0000 7f3455413640 1 -- 172.21.15.94:0/2601271548 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f343406f430 msgr2=0x7f3434071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.588+0000 7f3455413640 1 --2- 172.21.15.94:0/2601271548 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f343406f430 0x7f3434071940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f34400af000 tx=0x7f3440008040 comp rx=0 tx=0).stop 2024-04-25T16:51:25.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.588+0000 7f3455413640 1 -- 172.21.15.94:0/2601271548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3450160bf0 msgr2=0x7f3450152040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.588+0000 7f3455413640 1 --2- 172.21.15.94:0/2601271548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3450160bf0 0x7f3450152040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f343800c3a0 tx=0x7f343800c860 comp rx=0 tx=0).stop 2024-04-25T16:51:25.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.592+0000 7f3455413640 1 -- 172.21.15.94:0/2601271548 shutdown_connections 2024-04-25T16:51:25.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.592+0000 7f3455413640 1 --2- 172.21.15.94:0/2601271548 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f343406f430 0x7f3434071940 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.592+0000 7f3455413640 1 --2- 172.21.15.94:0/2601271548 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f345009c870 0x7f345009cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.592+0000 7f3455413640 1 --2- 172.21.15.94:0/2601271548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3450160bf0 0x7f3450152040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.592+0000 7f3455413640 1 --2- 172.21.15.94:0/2601271548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3450070180 0x7f345014ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.592+0000 7f3455413640 1 -- 172.21.15.94:0/2601271548 >> 172.21.15.94:0/2601271548 conn(0x7f34500b47f0 msgr2=0x7f34500d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:25.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.592+0000 7f3455413640 1 -- 172.21.15.94:0/2601271548 shutdown_connections 2024-04-25T16:51:25.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.592+0000 7f3455413640 1 -- 172.21.15.94:0/2601271548 wait complete. 2024-04-25T16:51:25.606 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:51:25.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:51:25.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2ee64640 1 Processor -- start 2024-04-25T16:51:25.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2ee64640 1 -- start start 2024-04-25T16:51:25.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2ee64640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2815c7e0 0x7f5e280afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2ee64640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5e280b0330 0x7f5e2806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2ee64640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5e2806dc50 0x7f5e2806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2ee64640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f5e28074380 con 0x7f5e2806dc50 2024-04-25T16:51:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2ee64640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5e2806e060 con 0x7f5e280b0330 2024-04-25T16:51:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2ee64640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5e280727e0 con 0x7f5e2815c7e0 2024-04-25T16:51:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2d401640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5e2806dc50 0x7f5e2806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2d401640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5e2806dc50 0x7f5e2806e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42518/0 (socket says 172.21.15.94:42518) 2024-04-25T16:51:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2d401640 1 -- 172.21.15.94:0/1168340855 learned_addr learned my addr 172.21.15.94:0/1168340855 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2d401640 1 -- 172.21.15.94:0/1168340855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2815c7e0 msgr2=0x7f5e280afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2d401640 1 --2- 172.21.15.94:0/1168340855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2815c7e0 0x7f5e280afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2d401640 1 -- 172.21.15.94:0/1168340855 >> v2:172.21.15.138:3300/0 conn(0x7f5e280b0330 msgr2=0x7f5e2806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e27fff640 1 --2- 172.21.15.94:0/1168340855 >> v2:172.21.15.138:3300/0 conn(0x7f5e280b0330 0x7f5e2806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2d401640 1 --2- 172.21.15.94:0/1168340855 >> v2:172.21.15.138:3300/0 conn(0x7f5e280b0330 0x7f5e2806d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.744+0000 7f5e2d401640 1 -- 172.21.15.94:0/1168340855 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e28072ac0 con 0x7f5e2806dc50 2024-04-25T16:51:25.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e2d401640 1 --2- 172.21.15.94:0/1168340855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e2806dc50 0x7f5e2806e7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5e280b5050 tx=0x7f5e2001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2036c31b55e6530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:25.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e277fe640 1 -- 172.21.15.94:0/1168340855 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e200a5840 con 0x7f5e2806dc50 2024-04-25T16:51:25.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e277fe640 1 -- 172.21.15.94:0/1168340855 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e200a59e0 con 0x7f5e2806dc50 2024-04-25T16:51:25.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e277fe640 1 -- 172.21.15.94:0/1168340855 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e200a5db0 con 0x7f5e2806dc50 2024-04-25T16:51:25.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e2ee64640 1 -- 172.21.15.94:0/1168340855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e2806dc50 msgr2=0x7f5e2806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e2ee64640 1 --2- 172.21.15.94:0/1168340855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e2806dc50 0x7f5e2806e7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5e280b5050 tx=0x7f5e2001c9c0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e2ee64640 1 -- 172.21.15.94:0/1168340855 shutdown_connections 2024-04-25T16:51:25.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e2ee64640 1 --2- 172.21.15.94:0/1168340855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e2806dc50 0x7f5e2806e7a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e2ee64640 1 --2- 172.21.15.94:0/1168340855 >> v2:172.21.15.138:3300/0 conn(0x7f5e280b0330 0x7f5e2806d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e2ee64640 1 --2- 172.21.15.94:0/1168340855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2815c7e0 0x7f5e280afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e2ee64640 1 -- 172.21.15.94:0/1168340855 >> 172.21.15.94:0/1168340855 conn(0x7f5e280b47f0 msgr2=0x7f5e280b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:25.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e2ee64640 1 -- 172.21.15.94:0/1168340855 shutdown_connections 2024-04-25T16:51:25.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e2ee64640 1 -- 172.21.15.94:0/1168340855 wait complete. 2024-04-25T16:51:25.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e2ee64640 1 Processor -- start 2024-04-25T16:51:25.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.748+0000 7f5e2ee64640 1 -- start start 2024-04-25T16:51:25.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e2ee64640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2806dc50 0x7f5e2814d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e2ee64640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e280b0330 0x7f5e28158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e2ee64640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2815c7e0 0x7f5e28159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e2ee64640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2815cbf0 con 0x7f5e280b0330 2024-04-25T16:51:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e2ee64640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28073ca0 con 0x7f5e2815c7e0 2024-04-25T16:51:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e2ee64640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5e28160460 con 0x7f5e2806dc50 2024-04-25T16:51:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e27fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e280b0330 0x7f5e28158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e27fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e280b0330 0x7f5e28158cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42520/0 (socket says 172.21.15.94:42520) 2024-04-25T16:51:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e27fff640 1 -- 172.21.15.94:0/854417156 learned_addr learned my addr 172.21.15.94:0/854417156 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e27fff640 1 -- 172.21.15.94:0/854417156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2806dc50 msgr2=0x7f5e2814d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e27fff640 1 --2- 172.21.15.94:0/854417156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2806dc50 0x7f5e2814d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e2d401640 1 --2- 172.21.15.94:0/854417156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2815c7e0 0x7f5e28159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e27fff640 1 -- 172.21.15.94:0/854417156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2815c7e0 msgr2=0x7f5e28159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e27fff640 1 --2- 172.21.15.94:0/854417156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2815c7e0 0x7f5e28159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e27fff640 1 -- 172.21.15.94:0/854417156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2809b380 con 0x7f5e280b0330 2024-04-25T16:51:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e27fff640 1 --2- 172.21.15.94:0/854417156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e280b0330 0x7f5e28158cd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00ea30 tx=0x7f5e1c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e257fa640 1 -- 172.21.15.94:0/854417156 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c004220 con 0x7f5e280b0330 2024-04-25T16:51:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e257fa640 1 -- 172.21.15.94:0/854417156 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e1c0043c0 con 0x7f5e280b0330 2024-04-25T16:51:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e2ee64640 1 -- 172.21.15.94:0/854417156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2809b620 con 0x7f5e280b0330 2024-04-25T16:51:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e257fa640 1 -- 172.21.15.94:0/854417156 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c007bd0 con 0x7f5e280b0330 2024-04-25T16:51:25.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e2ee64640 1 -- 172.21.15.94:0/854417156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2809bbe0 con 0x7f5e280b0330 2024-04-25T16:51:25.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e2ee64640 1 -- 172.21.15.94:0/854417156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e28076830 con 0x7f5e280b0330 2024-04-25T16:51:25.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e257fa640 1 -- 172.21.15.94:0/854417156 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5e1c00b030 con 0x7f5e280b0330 2024-04-25T16:51:25.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e257fa640 1 --2- 172.21.15.94:0/854417156 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5e1406f400 0x7f5e14071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:25.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e257fa640 1 -- 172.21.15.94:0/854417156 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5e1c0918b0 con 0x7f5e280b0330 2024-04-25T16:51:25.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.752+0000 7f5e2cc00640 1 --2- 172.21.15.94:0/854417156 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5e1406f400 0x7f5e14071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:25.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.756+0000 7f5e2cc00640 1 --2- 172.21.15.94:0/854417156 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5e1406f400 0x7f5e14071910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009690 tx=0x7f5e18006c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:25.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.760+0000 7f5e257fa640 1 -- 172.21.15.94:0/854417156 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5e1c05e460 con 0x7f5e280b0330 2024-04-25T16:51:25.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e2ee64640 1 -- 172.21.15.94:0/854417156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5e281601d0 con 0x7f5e280b0330 2024-04-25T16:51:25.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e257fa640 1 -- 172.21.15.94:0/854417156 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f5e1c05dd60 con 0x7f5e280b0330 2024-04-25T16:51:25.909 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:51:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e2ee64640 1 -- 172.21.15.94:0/854417156 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5e1406f400 msgr2=0x7f5e14071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e2ee64640 1 --2- 172.21.15.94:0/854417156 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5e1406f400 0x7f5e14071910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009690 tx=0x7f5e18006c00 comp rx=0 tx=0).stop 2024-04-25T16:51:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e2ee64640 1 -- 172.21.15.94:0/854417156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e280b0330 msgr2=0x7f5e28158cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e2ee64640 1 --2- 172.21.15.94:0/854417156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e280b0330 0x7f5e28158cd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00ea30 tx=0x7f5e1c00eef0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e2ee64640 1 -- 172.21.15.94:0/854417156 shutdown_connections 2024-04-25T16:51:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e2ee64640 1 --2- 172.21.15.94:0/854417156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2815c7e0 0x7f5e28159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e2ee64640 1 --2- 172.21.15.94:0/854417156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e280b0330 0x7f5e28158cd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e2ee64640 1 --2- 172.21.15.94:0/854417156 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5e1406f400 0x7f5e14071910 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e2ee64640 1 --2- 172.21.15.94:0/854417156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e2806dc50 0x7f5e2814d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e2ee64640 1 -- 172.21.15.94:0/854417156 >> 172.21.15.94:0/854417156 conn(0x7f5e280b47f0 msgr2=0x7f5e2815bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e2ee64640 1 -- 172.21.15.94:0/854417156 shutdown_connections 2024-04-25T16:51:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:25.912+0000 7f5e2ee64640 1 -- 172.21.15.94:0/854417156 wait complete. 2024-04-25T16:51:25.929 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:51:26.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871f617640 1 Processor -- start 2024-04-25T16:51:26.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871f617640 1 -- start start 2024-04-25T16:51:26.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871f617640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f871815c7e0 0x7f87180afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871f617640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f871806de20 0x7f871806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871f617640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8718074380 con 0x7f87180b0330 2024-04-25T16:51:26.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871f617640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f87180727e0 con 0x7f871815c7e0 2024-04-25T16:51:26.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871f617640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8718072ac0 con 0x7f871806de20 2024-04-25T16:51:26.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871d3b3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f871815c7e0 0x7f87180afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871d3b3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f871815c7e0 0x7f87180afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42530/0 (socket says 172.21.15.94:42530) 2024-04-25T16:51:26.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871d3b3640 1 -- 172.21.15.94:0/224012336 learned_addr learned my addr 172.21.15.94:0/224012336 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:26.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871d3b3640 1 -- 172.21.15.94:0/224012336 >> v2:172.21.15.138:3300/0 conn(0x7f871806de20 msgr2=0x7f871806e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:26.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871d3b3640 1 --2- 172.21.15.94:0/224012336 >> v2:172.21.15.138:3300/0 conn(0x7f871806de20 0x7f871806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871d3b3640 1 -- 172.21.15.94:0/224012336 >> v1:172.21.15.138:6789/0 conn(0x7f87180b0330 legacy=0x7f871806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:26.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.072+0000 7f871d3b3640 1 -- 172.21.15.94:0/224012336 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8718072da0 con 0x7f871815c7e0 2024-04-25T16:51:26.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.076+0000 7f871d3b3640 1 --2- 172.21.15.94:0/224012336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f871815c7e0 0x7f87180afdf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8718072510 tx=0x7f871401c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727a86f94a7e6e4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:26.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.076+0000 7f870ffff640 1 -- 172.21.15.94:0/224012336 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87140b0070 con 0x7f871815c7e0 2024-04-25T16:51:26.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.076+0000 7f870ffff640 1 -- 172.21.15.94:0/224012336 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87140ab600 con 0x7f871815c7e0 2024-04-25T16:51:26.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.076+0000 7f870ffff640 1 -- 172.21.15.94:0/224012336 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87140ab9d0 con 0x7f871815c7e0 2024-04-25T16:51:26.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.076+0000 7f871f617640 1 -- 172.21.15.94:0/224012336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f871815c7e0 msgr2=0x7f87180afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.076+0000 7f871f617640 1 --2- 172.21.15.94:0/224012336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f871815c7e0 0x7f87180afdf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8718072510 tx=0x7f871401c240 comp rx=0 tx=0).stop 2024-04-25T16:51:26.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.076+0000 7f871f617640 1 -- 172.21.15.94:0/224012336 shutdown_connections 2024-04-25T16:51:26.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.076+0000 7f871f617640 1 --2- 172.21.15.94:0/224012336 >> v2:172.21.15.138:3300/0 conn(0x7f871806de20 0x7f871806e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.076+0000 7f871f617640 1 --2- 172.21.15.94:0/224012336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f871815c7e0 0x7f87180afdf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.076+0000 7f871f617640 1 -- 172.21.15.94:0/224012336 >> 172.21.15.94:0/224012336 conn(0x7f87180b47f0 msgr2=0x7f87180b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:26.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.076+0000 7f871f617640 1 -- 172.21.15.94:0/224012336 shutdown_connections 2024-04-25T16:51:26.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.076+0000 7f871f617640 1 -- 172.21.15.94:0/224012336 wait complete. 2024-04-25T16:51:26.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871f617640 1 Processor -- start 2024-04-25T16:51:26.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871f617640 1 -- start start 2024-04-25T16:51:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871f617640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87180b0330 0x7f8718151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871f617640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f871815c7e0 0x7f87181fc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871f617640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87181f94a0 0x7f871814df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871f617640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87181695d0 con 0x7f871815c7e0 2024-04-25T16:51:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871f617640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f871815cbf0 con 0x7f87180b0330 2024-04-25T16:51:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871f617640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8718169750 con 0x7f87181f94a0 2024-04-25T16:51:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871cbb2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f871815c7e0 0x7f87181fc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871cbb2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f871815c7e0 0x7f87181fc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42532/0 (socket says 172.21.15.94:42532) 2024-04-25T16:51:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871cbb2640 1 -- 172.21.15.94:0/1209756397 learned_addr learned my addr 172.21.15.94:0/1209756397 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871dbb4640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87181f94a0 0x7f871814df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871cbb2640 1 -- 172.21.15.94:0/1209756397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87181f94a0 msgr2=0x7f871814df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871cbb2640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87181f94a0 0x7f871814df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871d3b3640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87180b0330 0x7f8718151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871cbb2640 1 -- 172.21.15.94:0/1209756397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87180b0330 msgr2=0x7f8718151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871cbb2640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87180b0330 0x7f8718151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871cbb2640 1 -- 172.21.15.94:0/1209756397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87181fd2c0 con 0x7f871815c7e0 2024-04-25T16:51:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f871cbb2640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f871815c7e0 0x7f87181fc900 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f870800b410 tx=0x7f870800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.080+0000 7f870dffb640 1 -- 172.21.15.94:0/1209756397 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708015020 con 0x7f871815c7e0 2024-04-25T16:51:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.084+0000 7f870dffb640 1 -- 172.21.15.94:0/1209756397 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8708011850 con 0x7f871815c7e0 2024-04-25T16:51:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.084+0000 7f871f617640 1 -- 172.21.15.94:0/1209756397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87181fd5a0 con 0x7f871815c7e0 2024-04-25T16:51:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.084+0000 7f870dffb640 1 -- 172.21.15.94:0/1209756397 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708011c40 con 0x7f871815c7e0 2024-04-25T16:51:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.084+0000 7f871f617640 1 -- 172.21.15.94:0/1209756397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8718165db0 con 0x7f871815c7e0 2024-04-25T16:51:26.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.084+0000 7f871f617640 1 -- 172.21.15.94:0/1209756397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8718076830 con 0x7f871815c7e0 2024-04-25T16:51:26.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.084+0000 7f870dffb640 1 -- 172.21.15.94:0/1209756397 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8708018020 con 0x7f871815c7e0 2024-04-25T16:51:26.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.084+0000 7f870dffb640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f86fc06f430 0x7f86fc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.084+0000 7f870dffb640 1 -- 172.21.15.94:0/1209756397 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f870808ef00 con 0x7f871815c7e0 2024-04-25T16:51:26.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.084+0000 7f871d3b3640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f86fc06f430 0x7f86fc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.084+0000 7f871d3b3640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f86fc06f430 0x7f86fc071940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f87140ad000 tx=0x7f8714008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:26.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.088+0000 7f870dffb640 1 -- 172.21.15.94:0/1209756397 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f870801f070 con 0x7f871815c7e0 2024-04-25T16:51:26.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f871f617640 1 -- 172.21.15.94:0/1209756397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f87181fd010 con 0x7f871815c7e0 2024-04-25T16:51:26.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f870dffb640 1 -- 172.21.15.94:0/1209756397 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f870805bb40 con 0x7f871815c7e0 2024-04-25T16:51:26.237 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:26.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f871f617640 1 -- 172.21.15.94:0/1209756397 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f86fc06f430 msgr2=0x7f86fc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f871f617640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f86fc06f430 0x7f86fc071940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f87140ad000 tx=0x7f8714008040 comp rx=0 tx=0).stop 2024-04-25T16:51:26.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f871f617640 1 -- 172.21.15.94:0/1209756397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f871815c7e0 msgr2=0x7f87181fc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f871f617640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f871815c7e0 0x7f87181fc900 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f870800b410 tx=0x7f870800b8d0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f871f617640 1 -- 172.21.15.94:0/1209756397 shutdown_connections 2024-04-25T16:51:26.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f871f617640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f86fc06f430 0x7f86fc071940 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f871f617640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87181f94a0 0x7f871814df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f871f617640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f871815c7e0 0x7f87181fc900 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f871f617640 1 --2- 172.21.15.94:0/1209756397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87180b0330 0x7f8718151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f871f617640 1 -- 172.21.15.94:0/1209756397 >> 172.21.15.94:0/1209756397 conn(0x7f87180b47f0 msgr2=0x7f8718159250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:26.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f871f617640 1 -- 172.21.15.94:0/1209756397 shutdown_connections 2024-04-25T16:51:26.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.240+0000 7f871f617640 1 -- 172.21.15.94:0/1209756397 wait complete. 2024-04-25T16:51:26.257 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:51:26.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1addef640 1 Processor -- start 2024-04-25T16:51:26.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1addef640 1 -- start start 2024-04-25T16:51:26.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1addef640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd1a80b0420 0x7fd1a806ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1addef640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1a806f2a0 0x7fd1a806c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1addef640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1a8070630 con 0x7fd1a806cb40 2024-04-25T16:51:26.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1addef640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd1a80aeaa0 con 0x7fd1a80b0420 2024-04-25T16:51:26.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1addef640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd1a80703e0 con 0x7fd1a806f2a0 2024-04-25T16:51:26.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1a6ffd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1a806f2a0 0x7fd1a806c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1a77fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd1a80b0420 0x7fd1a806ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1a6ffd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1a806f2a0 0x7fd1a806c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:52146/0 (socket says 172.21.15.94:52146) 2024-04-25T16:51:26.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1a6ffd640 1 -- 172.21.15.94:0/1821316722 learned_addr learned my addr 172.21.15.94:0/1821316722 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:26.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1a7fff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd1a806cb40 0x7fd1a815dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:39830/0 (socket says 172.21.15.94:39830) 2024-04-25T16:51:26.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1a77fe640 1 -- 172.21.15.94:0/1821316722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1a806f2a0 msgr2=0x7fd1a806c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:26.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1a77fe640 1 --2- 172.21.15.94:0/1821316722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1a806f2a0 0x7fd1a806c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1a77fe640 1 -- 172.21.15.94:0/1821316722 >> v1:172.21.15.138:6789/0 conn(0x7fd1a806cb40 legacy=0x7fd1a815dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1a77fe640 1 -- 172.21.15.94:0/1821316722 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a806f6b0 con 0x7fd1a80b0420 2024-04-25T16:51:26.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.400+0000 7fd1a6ffd640 1 --2- 172.21.15.94:0/1821316722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1a806f2a0 0x7fd1a806c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:26.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.404+0000 7fd1a77fe640 1 --2- 172.21.15.94:0/1821316722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a80b0420 0x7fd1a806ed60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00097c0 tx=0x7fd1a001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fce6e5b8c24fe807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:26.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.404+0000 7fd1a67fc640 1 -- 172.21.15.94:0/1821316722 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00b2070 con 0x7fd1a80b0420 2024-04-25T16:51:26.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.404+0000 7fd1a67fc640 1 -- 172.21.15.94:0/1821316722 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a00ad640 con 0x7fd1a80b0420 2024-04-25T16:51:26.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.404+0000 7fd1a67fc640 1 -- 172.21.15.94:0/1821316722 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00ada10 con 0x7fd1a80b0420 2024-04-25T16:51:26.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.404+0000 7fd1addef640 1 -- 172.21.15.94:0/1821316722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a80b0420 msgr2=0x7fd1a806ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.404+0000 7fd1addef640 1 --2- 172.21.15.94:0/1821316722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a80b0420 0x7fd1a806ed60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00097c0 tx=0x7fd1a001c4a0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.404+0000 7fd1addef640 1 -- 172.21.15.94:0/1821316722 shutdown_connections 2024-04-25T16:51:26.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.404+0000 7fd1addef640 1 --2- 172.21.15.94:0/1821316722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1a806f2a0 0x7fd1a806c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.404+0000 7fd1addef640 1 --2- 172.21.15.94:0/1821316722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a80b0420 0x7fd1a806ed60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.404+0000 7fd1addef640 1 -- 172.21.15.94:0/1821316722 >> 172.21.15.94:0/1821316722 conn(0x7fd1a80b47f0 msgr2=0x7fd1a80b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:26.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.404+0000 7fd1addef640 1 -- 172.21.15.94:0/1821316722 shutdown_connections 2024-04-25T16:51:26.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.404+0000 7fd1addef640 1 -- 172.21.15.94:0/1821316722 wait complete. 2024-04-25T16:51:26.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1addef640 1 Processor -- start 2024-04-25T16:51:26.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1addef640 1 -- start start 2024-04-25T16:51:26.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1addef640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1a806cb40 0x7fd1a81491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1addef640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a806f2a0 0x7fd1a815bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1addef640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1a80b0420 0x7fd1a815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1addef640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a806d360 con 0x7fd1a80b0420 2024-04-25T16:51:26.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1addef640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a80aef30 con 0x7fd1a806f2a0 2024-04-25T16:51:26.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1addef640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd1a80b0830 con 0x7fd1a806cb40 2024-04-25T16:51:26.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1a7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1a80b0420 0x7fd1a815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1a7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1a80b0420 0x7fd1a815c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42544/0 (socket says 172.21.15.94:42544) 2024-04-25T16:51:26.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1a7fff640 1 -- 172.21.15.94:0/2747052054 learned_addr learned my addr 172.21.15.94:0/2747052054 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1a77fe640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1a806cb40 0x7fd1a81491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1a6ffd640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a806f2a0 0x7fd1a815bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1a7fff640 1 -- 172.21.15.94:0/2747052054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1a806cb40 msgr2=0x7fd1a81491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1a7fff640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1a806cb40 0x7fd1a81491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1a7fff640 1 -- 172.21.15.94:0/2747052054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a806f2a0 msgr2=0x7fd1a815bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1a7fff640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a806f2a0 0x7fd1a815bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1a7fff640 1 -- 172.21.15.94:0/2747052054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a8156230 con 0x7fd1a80b0420 2024-04-25T16:51:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1a6ffd640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a806f2a0 0x7fd1a815bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1a7fff640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1a80b0420 0x7fd1a815c0c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd19000c3c0 tx=0x7fd19000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd19ffff640 1 -- 172.21.15.94:0/2747052054 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900074e0 con 0x7fd1a80b0420 2024-04-25T16:51:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd19ffff640 1 -- 172.21.15.94:0/2747052054 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd190007680 con 0x7fd1a80b0420 2024-04-25T16:51:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1addef640 1 -- 172.21.15.94:0/2747052054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a8156510 con 0x7fd1a80b0420 2024-04-25T16:51:26.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd19ffff640 1 -- 172.21.15.94:0/2747052054 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900049f0 con 0x7fd1a80b0420 2024-04-25T16:51:26.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.408+0000 7fd1addef640 1 -- 172.21.15.94:0/2747052054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a815ae20 con 0x7fd1a80b0420 2024-04-25T16:51:26.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.412+0000 7fd1addef640 1 -- 172.21.15.94:0/2747052054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a8076830 con 0x7fd1a80b0420 2024-04-25T16:51:26.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.412+0000 7fd19ffff640 1 -- 172.21.15.94:0/2747052054 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd19001a020 con 0x7fd1a80b0420 2024-04-25T16:51:26.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.412+0000 7fd19ffff640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd18806f400 0x7fd188071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.412+0000 7fd1a77fe640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd18806f400 0x7fd188071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.412+0000 7fd19ffff640 1 -- 172.21.15.94:0/2747052054 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd19008ed80 con 0x7fd1a80b0420 2024-04-25T16:51:26.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.412+0000 7fd1a77fe640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd18806f400 0x7fd188071910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00af000 tx=0x7fd1a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:26.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.416+0000 7fd19ffff640 1 -- 172.21.15.94:0/2747052054 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd19005b930 con 0x7fd1a80b0420 2024-04-25T16:51:26.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.576+0000 7fd1addef640 1 -- 172.21.15.94:0/2747052054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd1a8149cd0 con 0x7fd1a80b0420 2024-04-25T16:51:26.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.576+0000 7fd19ffff640 1 -- 172.21.15.94:0/2747052054 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fd19005b230 con 0x7fd1a80b0420 2024-04-25T16:51:26.575 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:51:26.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.580+0000 7fd1addef640 1 -- 172.21.15.94:0/2747052054 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd18806f400 msgr2=0x7fd188071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.580+0000 7fd1addef640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd18806f400 0x7fd188071910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00af000 tx=0x7fd1a0008040 comp rx=0 tx=0).stop 2024-04-25T16:51:26.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.580+0000 7fd1addef640 1 -- 172.21.15.94:0/2747052054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1a80b0420 msgr2=0x7fd1a815c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.580+0000 7fd1addef640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1a80b0420 0x7fd1a815c0c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd19000c3c0 tx=0x7fd19000c880 comp rx=0 tx=0).stop 2024-04-25T16:51:26.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.580+0000 7fd1addef640 1 -- 172.21.15.94:0/2747052054 shutdown_connections 2024-04-25T16:51:26.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.580+0000 7fd1addef640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1a80b0420 0x7fd1a815c0c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.580+0000 7fd1addef640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1a806f2a0 0x7fd1a815bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.580+0000 7fd1addef640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd18806f400 0x7fd188071910 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.580+0000 7fd1addef640 1 --2- 172.21.15.94:0/2747052054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1a806cb40 0x7fd1a81491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.580+0000 7fd1addef640 1 -- 172.21.15.94:0/2747052054 >> 172.21.15.94:0/2747052054 conn(0x7fd1a80b47f0 msgr2=0x7fd1a80d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:26.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.580+0000 7fd1addef640 1 -- 172.21.15.94:0/2747052054 shutdown_connections 2024-04-25T16:51:26.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.580+0000 7fd1addef640 1 -- 172.21.15.94:0/2747052054 wait complete. 2024-04-25T16:51:26.596 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:51:26.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.740+0000 7f1e94e40640 1 Processor -- start 2024-04-25T16:51:26.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.740+0000 7f1e94e40640 1 -- start start 2024-04-25T16:51:26.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.740+0000 7f1e94e40640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1e9015c7e0 0x7f1e900afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.740+0000 7f1e94e40640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e90074380 con 0x7f1e9006de20 2024-04-25T16:51:26.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.740+0000 7f1e94e40640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e900727e0 con 0x7f1e900b0330 2024-04-25T16:51:26.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.740+0000 7f1e94e40640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f1e90072ac0 con 0x7f1e9015c7e0 2024-04-25T16:51:26.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.740+0000 7f1e8e59c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1e9015c7e0 0x7f1e900afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.740+0000 7f1e8e59c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1e9015c7e0 0x7f1e900afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42548/0 (socket says 172.21.15.94:42548) 2024-04-25T16:51:26.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.740+0000 7f1e8e59c640 1 -- 172.21.15.94:0/3285703413 learned_addr learned my addr 172.21.15.94:0/3285703413 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:26.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.740+0000 7f1e8e59c640 1 -- 172.21.15.94:0/3285703413 >> v1:172.21.15.138:6789/0 conn(0x7f1e900b0330 legacy=0x7f1e9006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:26.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.740+0000 7f1e8e59c640 1 -- 172.21.15.94:0/3285703413 >> v1:172.21.15.94:6789/0 conn(0x7f1e9006de20 legacy=0x7f1e9006e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.740+0000 7f1e8e59c640 1 -- 172.21.15.94:0/3285703413 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e90072da0 con 0x7f1e9015c7e0 2024-04-25T16:51:26.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.744+0000 7f1e8e59c640 1 --2- 172.21.15.94:0/3285703413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e9015c7e0 0x7f1e900afdf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000bb10 tx=0x7f1e8001eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a20dfb82018028 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:26.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.744+0000 7f1e8dd9b640 1 -- 172.21.15.94:0/3285703413 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e800b4070 con 0x7f1e9015c7e0 2024-04-25T16:51:26.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.744+0000 7f1e8dd9b640 1 -- 172.21.15.94:0/3285703413 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e800a7b00 con 0x7f1e9015c7e0 2024-04-25T16:51:26.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.744+0000 7f1e8dd9b640 1 -- 172.21.15.94:0/3285703413 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e800af440 con 0x7f1e9015c7e0 2024-04-25T16:51:26.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.744+0000 7f1e94e40640 1 -- 172.21.15.94:0/3285703413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e9015c7e0 msgr2=0x7f1e900afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.744+0000 7f1e94e40640 1 --2- 172.21.15.94:0/3285703413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e9015c7e0 0x7f1e900afdf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000bb10 tx=0x7f1e8001eb30 comp rx=0 tx=0).stop 2024-04-25T16:51:26.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.744+0000 7f1e94e40640 1 -- 172.21.15.94:0/3285703413 shutdown_connections 2024-04-25T16:51:26.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.744+0000 7f1e94e40640 1 --2- 172.21.15.94:0/3285703413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e9015c7e0 0x7f1e900afdf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.744+0000 7f1e94e40640 1 -- 172.21.15.94:0/3285703413 >> 172.21.15.94:0/3285703413 conn(0x7f1e900b47f0 msgr2=0x7f1e900b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:26.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.744+0000 7f1e94e40640 1 -- 172.21.15.94:0/3285703413 shutdown_connections 2024-04-25T16:51:26.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.744+0000 7f1e94e40640 1 -- 172.21.15.94:0/3285703413 wait complete. 2024-04-25T16:51:26.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.744+0000 7f1e94e40640 1 Processor -- start 2024-04-25T16:51:26.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.744+0000 7f1e94e40640 1 -- start start 2024-04-25T16:51:26.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e94e40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e900b0330 0x7f1e901568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e94e40640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9015c7e0 0x7f1e9014a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e94e40640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e9015ef40 0x7f1e9015f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e94e40640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9015cbf0 con 0x7f1e900b0330 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e94e40640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9006f2c0 con 0x7f1e9015ef40 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e94e40640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9006ce10 con 0x7f1e9015c7e0 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e8e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e900b0330 0x7f1e901568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e8e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e900b0330 0x7f1e901568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42558/0 (socket says 172.21.15.94:42558) 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e8e59c640 1 -- 172.21.15.94:0/3662590882 learned_addr learned my addr 172.21.15.94:0/3662590882 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e8e59c640 1 -- 172.21.15.94:0/3662590882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9015c7e0 msgr2=0x7f1e9014a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e8e59c640 1 --2- 172.21.15.94:0/3662590882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9015c7e0 0x7f1e9014a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e8e59c640 1 -- 172.21.15.94:0/3662590882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e9015ef40 msgr2=0x7f1e9015f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e8ed9d640 1 --2- 172.21.15.94:0/3662590882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e9015ef40 0x7f1e9015f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e8e59c640 1 --2- 172.21.15.94:0/3662590882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e9015ef40 0x7f1e9015f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e8e59c640 1 -- 172.21.15.94:0/3662590882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e9015f890 con 0x7f1e900b0330 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e8e59c640 1 --2- 172.21.15.94:0/3662590882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e900b0330 0x7f1e901568f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000b2c0 tx=0x7f1e8001b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e86ffd640 1 -- 172.21.15.94:0/3662590882 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e800b4040 con 0x7f1e900b0330 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e86ffd640 1 -- 172.21.15.94:0/3662590882 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e800c1070 con 0x7f1e900b0330 2024-04-25T16:51:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e94e40640 1 -- 172.21.15.94:0/3662590882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e9009ef80 con 0x7f1e900b0330 2024-04-25T16:51:26.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e86ffd640 1 -- 172.21.15.94:0/3662590882 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e800b9460 con 0x7f1e900b0330 2024-04-25T16:51:26.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e94e40640 1 -- 172.21.15.94:0/3662590882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e90159b00 con 0x7f1e900b0330 2024-04-25T16:51:26.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e94e40640 1 -- 172.21.15.94:0/3662590882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e90159650 con 0x7f1e900b0330 2024-04-25T16:51:26.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.748+0000 7f1e86ffd640 1 -- 172.21.15.94:0/3662590882 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1e800b7070 con 0x7f1e900b0330 2024-04-25T16:51:26.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.752+0000 7f1e86ffd640 1 --2- 172.21.15.94:0/3662590882 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1e7006f430 0x7f1e70071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:26.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.752+0000 7f1e87fff640 1 --2- 172.21.15.94:0/3662590882 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1e7006f430 0x7f1e70071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:26.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.752+0000 7f1e86ffd640 1 -- 172.21.15.94:0/3662590882 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1e8012d0f0 con 0x7f1e900b0330 2024-04-25T16:51:26.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.752+0000 7f1e87fff640 1 --2- 172.21.15.94:0/3662590882 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1e7006f430 0x7f1e70071940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1e900b13c0 tx=0x7f1e78006bd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:26.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.756+0000 7f1e86ffd640 1 -- 172.21.15.94:0/3662590882 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e9015f890 con 0x7f1e900b0330 2024-04-25T16:51:26.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.900+0000 7f1e94e40640 1 -- 172.21.15.94:0/3662590882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1e90076830 con 0x7f1e900b0330 2024-04-25T16:51:26.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.904+0000 7f1e86ffd640 1 -- 172.21.15.94:0/3662590882 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f1e800187e0 con 0x7f1e900b0330 2024-04-25T16:51:26.900 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:26.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.904+0000 7f1e94e40640 1 -- 172.21.15.94:0/3662590882 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1e7006f430 msgr2=0x7f1e70071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.904+0000 7f1e94e40640 1 --2- 172.21.15.94:0/3662590882 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1e7006f430 0x7f1e70071940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1e900b13c0 tx=0x7f1e78006bd0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.904+0000 7f1e94e40640 1 -- 172.21.15.94:0/3662590882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e900b0330 msgr2=0x7f1e901568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:26.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.904+0000 7f1e94e40640 1 --2- 172.21.15.94:0/3662590882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e900b0330 0x7f1e901568f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000b2c0 tx=0x7f1e8001b8d0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.904+0000 7f1e94e40640 1 -- 172.21.15.94:0/3662590882 shutdown_connections 2024-04-25T16:51:26.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.904+0000 7f1e94e40640 1 --2- 172.21.15.94:0/3662590882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e9015ef40 0x7f1e9015f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.904+0000 7f1e94e40640 1 --2- 172.21.15.94:0/3662590882 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1e7006f430 0x7f1e70071940 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.904+0000 7f1e94e40640 1 --2- 172.21.15.94:0/3662590882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9015c7e0 0x7f1e9014a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.904+0000 7f1e94e40640 1 --2- 172.21.15.94:0/3662590882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e900b0330 0x7f1e901568f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:26.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.904+0000 7f1e94e40640 1 -- 172.21.15.94:0/3662590882 >> 172.21.15.94:0/3662590882 conn(0x7f1e900b47f0 msgr2=0x7f1e900b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:26.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.904+0000 7f1e94e40640 1 -- 172.21.15.94:0/3662590882 shutdown_connections 2024-04-25T16:51:26.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:26.904+0000 7f1e94e40640 1 -- 172.21.15.94:0/3662590882 wait complete. 2024-04-25T16:51:26.921 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:51:27.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f176f70e640 1 Processor -- start 2024-04-25T16:51:27.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f176f70e640 1 -- start start 2024-04-25T16:51:27.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f176f70e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f17680d1390 0x7f17680b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f176f70e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17680b4b50 con 0x7f176806e770 2024-04-25T16:51:27.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f176f70e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f176815e440 con 0x7f1768161530 2024-04-25T16:51:27.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f176f70e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f17680b1230 con 0x7f17680d1390 2024-04-25T16:51:27.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f176d4aa640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f176806e770 0x7f17680d0c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39176/0 (socket says 172.21.15.94:39176) 2024-04-25T16:51:27.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f176d4aa640 1 -- 172.21.15.94:0/165986609 learned_addr learned my addr 172.21.15.94:0/165986609 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:27.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f176dcab640 1 --2- 172.21.15.94:0/165986609 >> v2:172.21.15.138:3300/0 conn(0x7f17680d1390 0x7f17680b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f175ffff640 1 -- 172.21.15.94:0/165986609 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3555326209 0 0) 0x7f17680b4b50 con 0x7f176806e770 2024-04-25T16:51:27.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f175ffff640 1 -- 172.21.15.94:0/165986609 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1754003530 con 0x7f176806e770 2024-04-25T16:51:27.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f176dcab640 1 -- 172.21.15.94:0/165986609 >> v1:172.21.15.138:6789/0 conn(0x7f1768161530 legacy=0x7f17680ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f176dcab640 1 -- 172.21.15.94:0/165986609 >> v1:172.21.15.94:6789/0 conn(0x7f176806e770 legacy=0x7f17680d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.064+0000 7f176dcab640 1 -- 172.21.15.94:0/165986609 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176815e720 con 0x7f17680d1390 2024-04-25T16:51:27.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.068+0000 7f176dcab640 1 --2- 172.21.15.94:0/165986609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17680d1390 0x7f17680b0cf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17600097f0 tx=0x7f176001c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3873ebd3a331ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:27.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.068+0000 7f175ffff640 1 -- 172.21.15.94:0/165986609 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17600b0070 con 0x7f17680d1390 2024-04-25T16:51:27.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.068+0000 7f175ffff640 1 -- 172.21.15.94:0/165986609 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17600ab600 con 0x7f17680d1390 2024-04-25T16:51:27.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.068+0000 7f175ffff640 1 -- 172.21.15.94:0/165986609 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17600ab9d0 con 0x7f17680d1390 2024-04-25T16:51:27.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.068+0000 7f176f70e640 1 -- 172.21.15.94:0/165986609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17680d1390 msgr2=0x7f17680b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.068+0000 7f176f70e640 1 --2- 172.21.15.94:0/165986609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17680d1390 0x7f17680b0cf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17600097f0 tx=0x7f176001c380 comp rx=0 tx=0).stop 2024-04-25T16:51:27.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.068+0000 7f176f70e640 1 -- 172.21.15.94:0/165986609 shutdown_connections 2024-04-25T16:51:27.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.068+0000 7f176f70e640 1 --2- 172.21.15.94:0/165986609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17680d1390 0x7f17680b0cf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.068+0000 7f176f70e640 1 -- 172.21.15.94:0/165986609 >> 172.21.15.94:0/165986609 conn(0x7f17680d1d30 msgr2=0x7f17680d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:27.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.068+0000 7f176f70e640 1 -- 172.21.15.94:0/165986609 shutdown_connections 2024-04-25T16:51:27.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.068+0000 7f176f70e640 1 -- 172.21.15.94:0/165986609 wait complete. 2024-04-25T16:51:27.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176f70e640 1 Processor -- start 2024-04-25T16:51:27.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176f70e640 1 -- start start 2024-04-25T16:51:27.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176f70e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f176806e770 0x7f176815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176f70e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17680d1390 0x7f1768073010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176f70e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1768161530 0x7f1768073550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176f70e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f176816b8e0 con 0x7f1768161530 2024-04-25T16:51:27.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176f70e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17680cf820 con 0x7f17680d1390 2024-04-25T16:51:27.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176f70e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f176816ba60 con 0x7f176806e770 2024-04-25T16:51:27.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176dcab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1768161530 0x7f1768073550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176dcab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1768161530 0x7f1768073550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42574/0 (socket says 172.21.15.94:42574) 2024-04-25T16:51:27.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176dcab640 1 -- 172.21.15.94:0/1422268459 learned_addr learned my addr 172.21.15.94:0/1422268459 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:27.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176d4aa640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f176806e770 0x7f176815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176cca9640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17680d1390 0x7f1768073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176dcab640 1 -- 172.21.15.94:0/1422268459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f176806e770 msgr2=0x7f176815b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176dcab640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f176806e770 0x7f176815b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176dcab640 1 -- 172.21.15.94:0/1422268459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17680d1390 msgr2=0x7f1768073010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176dcab640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17680d1390 0x7f1768073010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176dcab640 1 -- 172.21.15.94:0/1422268459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1768153250 con 0x7f1768161530 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176dcab640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1768161530 0x7f1768073550 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f17600ae040 tx=0x7f1760019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f175dffb640 1 -- 172.21.15.94:0/1422268459 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17600b8020 con 0x7f1768161530 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f175dffb640 1 -- 172.21.15.94:0/1422268459 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17600b4410 con 0x7f1768161530 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176f70e640 1 -- 172.21.15.94:0/1422268459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176815c420 con 0x7f1768161530 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f175dffb640 1 -- 172.21.15.94:0/1422268459 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17600b0040 con 0x7f1768161530 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.072+0000 7f176f70e640 1 -- 172.21.15.94:0/1422268459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176815c9e0 con 0x7f1768161530 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.076+0000 7f176f70e640 1 -- 172.21.15.94:0/1422268459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f176806a730 con 0x7f1768161530 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.076+0000 7f175dffb640 1 -- 172.21.15.94:0/1422268459 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f17600bb020 con 0x7f1768161530 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.076+0000 7f175dffb640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f174c06f430 0x7f174c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.076+0000 7f175dffb640 1 -- 172.21.15.94:0/1422268459 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f17601296c0 con 0x7f1768161530 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.076+0000 7f176d4aa640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f174c06f430 0x7f174c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.076+0000 7f176d4aa640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f174c06f430 0x7f174c071940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1764001570 tx=0x7f1764009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:27.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.080+0000 7f175dffb640 1 -- 172.21.15.94:0/1422268459 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f17600c2030 con 0x7f1768161530 2024-04-25T16:51:27.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.264+0000 7f176f70e640 1 -- 172.21.15.94:0/1422268459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f176806cd50 con 0x7f1768161530 2024-04-25T16:51:27.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.264+0000 7f175dffb640 1 -- 172.21.15.94:0/1422268459 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f1760016090 con 0x7f1768161530 2024-04-25T16:51:27.262 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:27.262 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2352186041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2352186041},{"type":"v1","addr":"172.21.15.138:6833","nonce":2352186041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3210927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3210927873},{"type":"v1","addr":"172.21.15.94:6835","nonce":3210927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2237413225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2237413225},{"type":"v1","addr":"172.21.15.94:6837","nonce":2237413225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:50:54.403299+0000","modified":"2024-04-25T16:51:18.139471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/3652545132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3652545132},{"type":"v1","addr":"172.21.15.138:6835","nonce":3652545132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T16:51:27.262 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:51:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.264+0000 7f176f70e640 1 -- 172.21.15.94:0/1422268459 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f174c06f430 msgr2=0x7f174c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.264+0000 7f176f70e640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f174c06f430 0x7f174c071940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1764001570 tx=0x7f1764009040 comp rx=0 tx=0).stop 2024-04-25T16:51:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.264+0000 7f176f70e640 1 -- 172.21.15.94:0/1422268459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1768161530 msgr2=0x7f1768073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.264+0000 7f176f70e640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1768161530 0x7f1768073550 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f17600ae040 tx=0x7f1760019120 comp rx=0 tx=0).stop 2024-04-25T16:51:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.268+0000 7f176f70e640 1 -- 172.21.15.94:0/1422268459 shutdown_connections 2024-04-25T16:51:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.268+0000 7f176f70e640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1768161530 0x7f1768073550 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.268+0000 7f176f70e640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17680d1390 0x7f1768073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.268+0000 7f176f70e640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f174c06f430 0x7f174c071940 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.268+0000 7f176f70e640 1 --2- 172.21.15.94:0/1422268459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f176806e770 0x7f176815b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.268+0000 7f176f70e640 1 -- 172.21.15.94:0/1422268459 >> 172.21.15.94:0/1422268459 conn(0x7f17680d1d30 msgr2=0x7f17680d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.268+0000 7f176f70e640 1 -- 172.21.15.94:0/1422268459 shutdown_connections 2024-04-25T16:51:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.268+0000 7f176f70e640 1 -- 172.21.15.94:0/1422268459 wait complete. 2024-04-25T16:51:27.282 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f611e0d6640 1 Processor -- start 2024-04-25T16:51:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f611e0d6640 1 -- start start 2024-04-25T16:51:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f611e0d6640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6118070080 0x7f611815e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f611e0d6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f611815e7c0 0x7f611806c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f611e0d6640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f611806fc60 con 0x7f611815ebd0 2024-04-25T16:51:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f611e0d6640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f6118158550 con 0x7f6118070080 2024-04-25T16:51:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f611e0d6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f611806f7d0 con 0x7f611815e7c0 2024-04-25T16:51:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f61177fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6118070080 0x7f611815e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f61177fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6118070080 0x7f611815e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42586/0 (socket says 172.21.15.94:42586) 2024-04-25T16:51:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f61177fe640 1 -- 172.21.15.94:0/192454792 learned_addr learned my addr 172.21.15.94:0/192454792 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f6117fff640 1 --1- 172.21.15.94:0/192454792 >> v1:172.21.15.94:6789/0 conn(0x7f611815ebd0 0x7f611806cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:39188/0 (socket says 172.21.15.94:39188) 2024-04-25T16:51:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f610effd640 1 --2- 172.21.15.94:0/192454792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f611815e7c0 0x7f611806c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f61177fe640 1 -- 172.21.15.94:0/192454792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f611815e7c0 msgr2=0x7f611806c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.424+0000 7f61177fe640 1 --2- 172.21.15.94:0/192454792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f611815e7c0 0x7f611806c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f61177fe640 1 -- 172.21.15.94:0/192454792 >> v1:172.21.15.94:6789/0 conn(0x7f611815ebd0 legacy=0x7f611806cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f61177fe640 1 -- 172.21.15.94:0/192454792 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61180aef70 con 0x7f6118070080 2024-04-25T16:51:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f61177fe640 1 --2- 172.21.15.94:0/192454792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6118070080 0x7f611815e280 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6108009870 tx=0x7f610801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dea4c1d8aaec19b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f6116ffd640 1 -- 172.21.15.94:0/192454792 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61080ab640 con 0x7f6118070080 2024-04-25T16:51:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f6116ffd640 1 -- 172.21.15.94:0/192454792 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61080b0070 con 0x7f6118070080 2024-04-25T16:51:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f6116ffd640 1 -- 172.21.15.94:0/192454792 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61080aba00 con 0x7f6118070080 2024-04-25T16:51:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f611e0d6640 1 -- 172.21.15.94:0/192454792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6118070080 msgr2=0x7f611815e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f611e0d6640 1 --2- 172.21.15.94:0/192454792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6118070080 0x7f611815e280 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6108009870 tx=0x7f610801c530 comp rx=0 tx=0).stop 2024-04-25T16:51:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f611e0d6640 1 -- 172.21.15.94:0/192454792 shutdown_connections 2024-04-25T16:51:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f611e0d6640 1 --2- 172.21.15.94:0/192454792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f611815e7c0 0x7f611806c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f611e0d6640 1 --2- 172.21.15.94:0/192454792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6118070080 0x7f611815e280 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f611e0d6640 1 -- 172.21.15.94:0/192454792 >> 172.21.15.94:0/192454792 conn(0x7f61180b47f0 msgr2=0x7f61180b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f611e0d6640 1 -- 172.21.15.94:0/192454792 shutdown_connections 2024-04-25T16:51:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.428+0000 7f611e0d6640 1 -- 172.21.15.94:0/192454792 wait complete. 2024-04-25T16:51:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f611e0d6640 1 Processor -- start 2024-04-25T16:51:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f611e0d6640 1 -- start start 2024-04-25T16:51:27.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f611e0d6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6118070080 0x7f611809bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f611e0d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611815e7c0 0x7f6118141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f611e0d6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f611815ebd0 0x7f61180b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f611e0d6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6118070490 con 0x7f611815e7c0 2024-04-25T16:51:27.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f611e0d6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61180ae6c0 con 0x7f611815ebd0 2024-04-25T16:51:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f611e0d6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f611816ae90 con 0x7f6118070080 2024-04-25T16:51:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f610effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611815e7c0 0x7f6118141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f610effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611815e7c0 0x7f6118141380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42588/0 (socket says 172.21.15.94:42588) 2024-04-25T16:51:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f610effd640 1 -- 172.21.15.94:0/3448702972 learned_addr learned my addr 172.21.15.94:0/3448702972 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:27.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f61177fe640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6118070080 0x7f611809bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f610effd640 1 -- 172.21.15.94:0/3448702972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6118070080 msgr2=0x7f611809bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f6117fff640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f611815ebd0 0x7f61180b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f610effd640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6118070080 0x7f611809bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f610effd640 1 -- 172.21.15.94:0/3448702972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f611815ebd0 msgr2=0x7f61180b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f610effd640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f611815ebd0 0x7f61180b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f610effd640 1 -- 172.21.15.94:0/3448702972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611809e700 con 0x7f611815e7c0 2024-04-25T16:51:27.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f61177fe640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6118070080 0x7f611809bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f610effd640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611815e7c0 0x7f6118141380 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f610000d620 tx=0x7f610000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f6114ff9640 1 -- 172.21.15.94:0/3448702972 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610000b620 con 0x7f611815e7c0 2024-04-25T16:51:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f6114ff9640 1 -- 172.21.15.94:0/3448702972 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6100019070 con 0x7f611815e7c0 2024-04-25T16:51:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f611e0d6640 1 -- 172.21.15.94:0/3448702972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611809e9e0 con 0x7f611815e7c0 2024-04-25T16:51:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f6114ff9640 1 -- 172.21.15.94:0/3448702972 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610000b9e0 con 0x7f611815e7c0 2024-04-25T16:51:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.432+0000 7f611e0d6640 1 -- 172.21.15.94:0/3448702972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f611809efa0 con 0x7f611815e7c0 2024-04-25T16:51:27.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.436+0000 7f611e0d6640 1 -- 172.21.15.94:0/3448702972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6118076830 con 0x7f611815e7c0 2024-04-25T16:51:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.436+0000 7f6114ff9640 1 -- 172.21.15.94:0/3448702972 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f610000bb80 con 0x7f611815e7c0 2024-04-25T16:51:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.436+0000 7f6114ff9640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f60f806f430 0x7f60f8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.436+0000 7f6114ff9640 1 -- 172.21.15.94:0/3448702972 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6100090480 con 0x7f611815e7c0 2024-04-25T16:51:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.436+0000 7f61177fe640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f60f806f430 0x7f60f8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.436+0000 7f61177fe640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f60f806f430 0x7f60f8071940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61080af000 tx=0x7f6108008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.440+0000 7f6114ff9640 1 -- 172.21.15.94:0/3448702972 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f610005d110 con 0x7f611815e7c0 2024-04-25T16:51:27.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.600+0000 7f611e0d6640 1 -- 172.21.15.94:0/3448702972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f611809e490 con 0x7f611815e7c0 2024-04-25T16:51:27.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.604+0000 7f6114ff9640 1 -- 172.21.15.94:0/3448702972 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f610005ca10 con 0x7f611815e7c0 2024-04-25T16:51:27.600 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:27.600 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:59.528743+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:27.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.604+0000 7f611e0d6640 1 -- 172.21.15.94:0/3448702972 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f60f806f430 msgr2=0x7f60f8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.604+0000 7f611e0d6640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f60f806f430 0x7f60f8071940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61080af000 tx=0x7f6108008040 comp rx=0 tx=0).stop 2024-04-25T16:51:27.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.604+0000 7f611e0d6640 1 -- 172.21.15.94:0/3448702972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611815e7c0 msgr2=0x7f6118141380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.604+0000 7f611e0d6640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611815e7c0 0x7f6118141380 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f610000d620 tx=0x7f610000dae0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.604+0000 7f611e0d6640 1 -- 172.21.15.94:0/3448702972 shutdown_connections 2024-04-25T16:51:27.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.604+0000 7f611e0d6640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f611815ebd0 0x7f61180b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.604+0000 7f611e0d6640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f611815e7c0 0x7f6118141380 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.604+0000 7f611e0d6640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f60f806f430 0x7f60f8071940 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.604+0000 7f611e0d6640 1 --2- 172.21.15.94:0/3448702972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6118070080 0x7f611809bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.604+0000 7f611e0d6640 1 -- 172.21.15.94:0/3448702972 >> 172.21.15.94:0/3448702972 conn(0x7f61180b47f0 msgr2=0x7f6118160b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:27.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.604+0000 7f611e0d6640 1 -- 172.21.15.94:0/3448702972 shutdown_connections 2024-04-25T16:51:27.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.604+0000 7f611e0d6640 1 -- 172.21.15.94:0/3448702972 wait complete. 2024-04-25T16:51:27.622 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:51:27.623 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:51:27.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.764+0000 7f349f231640 1 Processor -- start 2024-04-25T16:51:27.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.764+0000 7f349f231640 1 -- start start 2024-04-25T16:51:27.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.764+0000 7f349f231640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f349815c7e0 0x7f34980afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.764+0000 7f349f231640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f34980b0330 0x7f349806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.764+0000 7f349f231640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f349806dc50 0x7f349806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.764+0000 7f349f231640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f3498074380 con 0x7f349815c7e0 2024-04-25T16:51:27.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.764+0000 7f349f231640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f349806e060 con 0x7f34980b0330 2024-04-25T16:51:27.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.764+0000 7f349f231640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34980727e0 con 0x7f349806dc50 2024-04-25T16:51:27.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.764+0000 7f349cfcd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f349815c7e0 0x7f34980afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.764+0000 7f349cfcd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f349815c7e0 0x7f34980afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42602/0 (socket says 172.21.15.94:42602) 2024-04-25T16:51:27.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.764+0000 7f349cfcd640 1 -- 172.21.15.94:0/1760416869 learned_addr learned my addr 172.21.15.94:0/1760416869 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:27.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.764+0000 7f349d7ce640 1 --2- 172.21.15.94:0/1760416869 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f349806dc50 0x7f349806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.764+0000 7f3497fff640 1 --2- 172.21.15.94:0/1760416869 >> v2:172.21.15.138:3300/0 conn(0x7f34980b0330 0x7f349806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349cfcd640 1 -- 172.21.15.94:0/1760416869 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f349806dc50 msgr2=0x7f349806e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:27.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349cfcd640 1 --2- 172.21.15.94:0/1760416869 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f349806dc50 0x7f349806e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349cfcd640 1 -- 172.21.15.94:0/1760416869 >> v2:172.21.15.138:3300/0 conn(0x7f34980b0330 msgr2=0x7f349806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:27.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349cfcd640 1 --2- 172.21.15.94:0/1760416869 >> v2:172.21.15.138:3300/0 conn(0x7f34980b0330 0x7f349806d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349cfcd640 1 -- 172.21.15.94:0/1760416869 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3498072ac0 con 0x7f349815c7e0 2024-04-25T16:51:27.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f3497fff640 1 --2- 172.21.15.94:0/1760416869 >> v2:172.21.15.138:3300/0 conn(0x7f34980b0330 0x7f349806d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:27.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349cfcd640 1 --2- 172.21.15.94:0/1760416869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f349815c7e0 0x7f34980afdf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f34980b5020 tx=0x7f348801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fe583b9aafbe0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:27.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f34977fe640 1 -- 172.21.15.94:0/1760416869 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34880ab620 con 0x7f349815c7e0 2024-04-25T16:51:27.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f34977fe640 1 -- 172.21.15.94:0/1760416869 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34880b0070 con 0x7f349815c7e0 2024-04-25T16:51:27.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f34977fe640 1 -- 172.21.15.94:0/1760416869 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34880ab9e0 con 0x7f349815c7e0 2024-04-25T16:51:27.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349f231640 1 -- 172.21.15.94:0/1760416869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f349815c7e0 msgr2=0x7f34980afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349f231640 1 --2- 172.21.15.94:0/1760416869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f349815c7e0 0x7f34980afdf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f34980b5020 tx=0x7f348801c460 comp rx=0 tx=0).stop 2024-04-25T16:51:27.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349f231640 1 -- 172.21.15.94:0/1760416869 shutdown_connections 2024-04-25T16:51:27.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349f231640 1 --2- 172.21.15.94:0/1760416869 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f349806dc50 0x7f349806e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349f231640 1 --2- 172.21.15.94:0/1760416869 >> v2:172.21.15.138:3300/0 conn(0x7f34980b0330 0x7f349806d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349f231640 1 --2- 172.21.15.94:0/1760416869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f349815c7e0 0x7f34980afdf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349f231640 1 -- 172.21.15.94:0/1760416869 >> 172.21.15.94:0/1760416869 conn(0x7f34980b47f0 msgr2=0x7f34980b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:27.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349f231640 1 -- 172.21.15.94:0/1760416869 shutdown_connections 2024-04-25T16:51:27.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.768+0000 7f349f231640 1 -- 172.21.15.94:0/1760416869 wait complete. 2024-04-25T16:51:27.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349f231640 1 Processor -- start 2024-04-25T16:51:27.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349f231640 1 -- start start 2024-04-25T16:51:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349f231640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f349806dc50 0x7f349809a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349f231640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34980b0330 0x7f34981568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349f231640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f349815c7e0 0x7f3498156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349f231640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f349815cbf0 con 0x7f349806dc50 2024-04-25T16:51:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349f231640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3498073ca0 con 0x7f34980b0330 2024-04-25T16:51:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349f231640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3498160460 con 0x7f349815c7e0 2024-04-25T16:51:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349cfcd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f349806dc50 0x7f349809a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349cfcd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f349806dc50 0x7f349809a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42608/0 (socket says 172.21.15.94:42608) 2024-04-25T16:51:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349cfcd640 1 -- 172.21.15.94:0/4122829505 learned_addr learned my addr 172.21.15.94:0/4122829505 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349d7ce640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f349815c7e0 0x7f3498156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349cfcd640 1 -- 172.21.15.94:0/4122829505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f349815c7e0 msgr2=0x7f3498156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349cfcd640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f349815c7e0 0x7f3498156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349cfcd640 1 -- 172.21.15.94:0/4122829505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34980b0330 msgr2=0x7f34981568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f3497fff640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34980b0330 0x7f34981568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349cfcd640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34980b0330 0x7f34981568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349cfcd640 1 -- 172.21.15.94:0/4122829505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34981523c0 con 0x7f349806dc50 2024-04-25T16:51:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349cfcd640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f349806dc50 0x7f349809a8b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f34880ad040 tx=0x7f348801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f34957fa640 1 -- 172.21.15.94:0/4122829505 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34880abda0 con 0x7f349806dc50 2024-04-25T16:51:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f34957fa640 1 -- 172.21.15.94:0/4122829505 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34880be3f0 con 0x7f349806dc50 2024-04-25T16:51:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349f231640 1 -- 172.21.15.94:0/4122829505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3498151320 con 0x7f349806dc50 2024-04-25T16:51:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f34957fa640 1 -- 172.21.15.94:0/4122829505 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34880be7e0 con 0x7f349806dc50 2024-04-25T16:51:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.772+0000 7f349f231640 1 -- 172.21.15.94:0/4122829505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34981518e0 con 0x7f349806dc50 2024-04-25T16:51:27.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.776+0000 7f349f231640 1 -- 172.21.15.94:0/4122829505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3498076830 con 0x7f349806dc50 2024-04-25T16:51:27.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.776+0000 7f34957fa640 1 -- 172.21.15.94:0/4122829505 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f34880b43f0 con 0x7f349806dc50 2024-04-25T16:51:27.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.776+0000 7f34957fa640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f348406f400 0x7f3484071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:27.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.776+0000 7f34957fa640 1 -- 172.21.15.94:0/4122829505 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f34881294c0 con 0x7f349806dc50 2024-04-25T16:51:27.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.776+0000 7f3497fff640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f348406f400 0x7f3484071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:27.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.776+0000 7f3497fff640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f348406f400 0x7f3484071910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f348c006fd0 tx=0x7f348c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:27.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.780+0000 7f34957fa640 1 -- 172.21.15.94:0/4122829505 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34880c4060 con 0x7f349806dc50 2024-04-25T16:51:27.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.940+0000 7f349f231640 1 -- 172.21.15.94:0/4122829505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f34980d0760 con 0x7f349806dc50 2024-04-25T16:51:27.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.940+0000 7f34957fa640 1 -- 172.21.15.94:0/4122829505 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f3488016110 con 0x7f349806dc50 2024-04-25T16:51:27.938 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:51:27.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.944+0000 7f349f231640 1 -- 172.21.15.94:0/4122829505 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f348406f400 msgr2=0x7f3484071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.944+0000 7f349f231640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f348406f400 0x7f3484071910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f348c006fd0 tx=0x7f348c008040 comp rx=0 tx=0).stop 2024-04-25T16:51:27.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.944+0000 7f349f231640 1 -- 172.21.15.94:0/4122829505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f349806dc50 msgr2=0x7f349809a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:27.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.944+0000 7f349f231640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f349806dc50 0x7f349809a8b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f34880ad040 tx=0x7f348801aca0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.944+0000 7f349f231640 1 -- 172.21.15.94:0/4122829505 shutdown_connections 2024-04-25T16:51:27.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.944+0000 7f349f231640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f348406f400 0x7f3484071910 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.944+0000 7f349f231640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f349815c7e0 0x7f3498156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.944+0000 7f349f231640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34980b0330 0x7f34981568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.944+0000 7f349f231640 1 --2- 172.21.15.94:0/4122829505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f349806dc50 0x7f349809a8b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:27.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.944+0000 7f349f231640 1 -- 172.21.15.94:0/4122829505 >> 172.21.15.94:0/4122829505 conn(0x7f34980b47f0 msgr2=0x7f3498156160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:27.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.944+0000 7f349f231640 1 -- 172.21.15.94:0/4122829505 shutdown_connections 2024-04-25T16:51:27.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:27.944+0000 7f349f231640 1 -- 172.21.15.94:0/4122829505 wait complete. 2024-04-25T16:51:27.960 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:51:28.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.116+0000 7faa13c41640 1 Processor -- start 2024-04-25T16:51:28.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.116+0000 7faa13c41640 1 -- start start 2024-04-25T16:51:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa13c41640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c15cf30 0x7faa0c0aea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa13c41640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faa0c0aef80 0x7faa0c0af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa13c41640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa0c15d340 con 0x7faa0c072b60 2024-04-25T16:51:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa13c41640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7faa0c0703e0 con 0x7faa0c0aef80 2024-04-25T16:51:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa13c41640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa0c06f360 con 0x7faa0c15cf30 2024-04-25T16:51:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa119dd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7faa0c072b60 0x7faa0c15c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50378/0 (socket says 172.21.15.94:50378) 2024-04-25T16:51:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa119dd640 1 -- 172.21.15.94:0/4027504625 learned_addr learned my addr 172.21.15.94:0/4027504625 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa121de640 1 --2- 172.21.15.94:0/4027504625 >> v2:172.21.15.138:3300/0 conn(0x7faa0c0aef80 0x7faa0c0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa111dc640 1 --2- 172.21.15.94:0/4027504625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c15cf30 0x7faa0c0aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1845980448 0 0) 0x7faa0c15d340 con 0x7faa0c072b60 2024-04-25T16:51:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9f8003450 con 0x7faa0c072b60 2024-04-25T16:51:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2492993555 0 0) 0x7fa9f8003450 con 0x7faa0c072b60 2024-04-25T16:51:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c15cf30 msgr2=0x7faa0c0aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 --2- 172.21.15.94:0/4027504625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c15cf30 0x7faa0c0aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 >> v2:172.21.15.138:3300/0 conn(0x7faa0c0aef80 msgr2=0x7faa0c0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 --2- 172.21.15.94:0/4027504625 >> v2:172.21.15.138:3300/0 conn(0x7faa0c0aef80 0x7faa0c0af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c06f620 con 0x7faa0c072b60 2024-04-25T16:51:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7faa08001680 con 0x7faa0c072b60 2024-04-25T16:51:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 >> v1:172.21.15.94:6789/0 conn(0x7faa0c072b60 legacy=0x7faa0c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 --2- 172.21.15.94:0/4027504625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9f8003ab0 0x7fa9f8003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 --2- 172.21.15.94:0/4027504625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f8008860 0x7fa9f800ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 --2- 172.21.15.94:0/4027504625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c15cf30 0x7fa9f800d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c0703e0 con 0x7fa9f8008860 2024-04-25T16:51:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c06f360 con 0x7fa9f8003ab0 2024-04-25T16:51:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f800dc60 con 0x7faa0c15cf30 2024-04-25T16:51:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa111dc640 1 --2- 172.21.15.94:0/4027504625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f8008860 0x7fa9f800ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa119dd640 1 --2- 172.21.15.94:0/4027504625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9f8003ab0 0x7fa9f8003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa121de640 1 --2- 172.21.15.94:0/4027504625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c15cf30 0x7fa9f800d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa111dc640 1 -- 172.21.15.94:0/4027504625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c15cf30 msgr2=0x7fa9f800d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa111dc640 1 --2- 172.21.15.94:0/4027504625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c15cf30 0x7fa9f800d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa111dc640 1 -- 172.21.15.94:0/4027504625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9f8003ab0 msgr2=0x7fa9f8003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa111dc640 1 --2- 172.21.15.94:0/4027504625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9f8003ab0 0x7fa9f8003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.120+0000 7faa111dc640 1 -- 172.21.15.94:0/4027504625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f8003450 con 0x7fa9f8008860 2024-04-25T16:51:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.124+0000 7faa111dc640 1 --2- 172.21.15.94:0/4027504625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f8008860 0x7fa9f800ad20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009ae0 tx=0x7fa9fc01ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d8179a848c92df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.124+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc0b2070 con 0x7fa9f8008860 2024-04-25T16:51:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.124+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9fc0a5540 con 0x7fa9f8008860 2024-04-25T16:51:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.124+0000 7faa109db640 1 -- 172.21.15.94:0/4027504625 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc0096b0 con 0x7fa9f8008860 2024-04-25T16:51:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.124+0000 7faa13c41640 1 -- 172.21.15.94:0/4027504625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f8008860 msgr2=0x7fa9f800ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.124+0000 7faa13c41640 1 --2- 172.21.15.94:0/4027504625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f8008860 0x7fa9f800ad20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009ae0 tx=0x7fa9fc01ca60 comp rx=0 tx=0).stop 2024-04-25T16:51:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.124+0000 7faa121de640 1 -- 172.21.15.94:0/4027504625 reap_dead start 2024-04-25T16:51:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.124+0000 7faa13c41640 1 -- 172.21.15.94:0/4027504625 shutdown_connections 2024-04-25T16:51:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.124+0000 7faa13c41640 1 -- 172.21.15.94:0/4027504625 >> 172.21.15.94:0/4027504625 conn(0x7faa0c0b47f0 msgr2=0x7faa0c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.124+0000 7faa13c41640 1 -- 172.21.15.94:0/4027504625 shutdown_connections 2024-04-25T16:51:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.124+0000 7faa13c41640 1 -- 172.21.15.94:0/4027504625 wait complete. 2024-04-25T16:51:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa13c41640 1 Processor -- start 2024-04-25T16:51:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa13c41640 1 -- start start 2024-04-25T16:51:28.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa13c41640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f8008860 0x7faa0c090880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa13c41640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c0af390 0x7faa0c0afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa13c41640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa0c0b03b0 0x7faa0c161c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa13c41640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c072f70 con 0x7fa9f8008860 2024-04-25T16:51:28.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa13c41640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c0730f0 con 0x7faa0c0b03b0 2024-04-25T16:51:28.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa13c41640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa0c159cc0 con 0x7faa0c0af390 2024-04-25T16:51:28.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa119dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f8008860 0x7faa0c090880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa119dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f8008860 0x7faa0c090880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46848/0 (socket says 172.21.15.94:46848) 2024-04-25T16:51:28.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa119dd640 1 -- 172.21.15.94:0/556323374 learned_addr learned my addr 172.21.15.94:0/556323374 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa111dc640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c0af390 0x7faa0c0afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa121de640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa0c0b03b0 0x7faa0c161c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa119dd640 1 -- 172.21.15.94:0/556323374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c0af390 msgr2=0x7faa0c0afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa119dd640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c0af390 0x7faa0c0afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa119dd640 1 -- 172.21.15.94:0/556323374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa0c0b03b0 msgr2=0x7faa0c161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa119dd640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa0c0b03b0 0x7faa0c161c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa119dd640 1 -- 172.21.15.94:0/556323374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c0b07c0 con 0x7fa9f8008860 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa111dc640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c0af390 0x7faa0c0afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa121de640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa0c0b03b0 0x7faa0c161c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa119dd640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f8008860 0x7faa0c090880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faa08001210 tx=0x7faa0800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa027fc640 1 -- 172.21.15.94:0/556323374 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0800ee00 con 0x7fa9f8008860 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa027fc640 1 -- 172.21.15.94:0/556323374 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa08018070 con 0x7fa9f8008860 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa13c41640 1 -- 172.21.15.94:0/556323374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0c157480 con 0x7fa9f8008860 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa027fc640 1 -- 172.21.15.94:0/556323374 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0800cab0 con 0x7fa9f8008860 2024-04-25T16:51:28.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa13c41640 1 -- 172.21.15.94:0/556323374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0c157a40 con 0x7fa9f8008860 2024-04-25T16:51:28.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.128+0000 7faa13c41640 1 -- 172.21.15.94:0/556323374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa0c076830 con 0x7fa9f8008860 2024-04-25T16:51:28.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.132+0000 7faa027fc640 1 -- 172.21.15.94:0/556323374 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faa0800cc50 con 0x7fa9f8008860 2024-04-25T16:51:28.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.132+0000 7faa027fc640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa9f006f430 0x7fa9f0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.132+0000 7faa111dc640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa9f006f430 0x7fa9f0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.132+0000 7faa027fc640 1 -- 172.21.15.94:0/556323374 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faa0808fdb0 con 0x7fa9f8008860 2024-04-25T16:51:28.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.132+0000 7faa111dc640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa9f006f430 0x7fa9f0071940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0af000 tx=0x7fa9fc0046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.136+0000 7faa027fc640 1 -- 172.21.15.94:0/556323374 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faa0c0b07c0 con 0x7fa9f8008860 2024-04-25T16:51:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.284+0000 7faa13c41640 1 -- 172.21.15.94:0/556323374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faa0c15b3a0 con 0x7fa9f8008860 2024-04-25T16:51:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.284+0000 7faa027fc640 1 -- 172.21.15.94:0/556323374 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7faa0805ca40 con 0x7fa9f8008860 2024-04-25T16:51:28.282 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:28.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.288+0000 7faa13c41640 1 -- 172.21.15.94:0/556323374 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa9f006f430 msgr2=0x7fa9f0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.288+0000 7faa13c41640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa9f006f430 0x7fa9f0071940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0af000 tx=0x7fa9fc0046d0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.288+0000 7faa13c41640 1 -- 172.21.15.94:0/556323374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f8008860 msgr2=0x7faa0c090880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.288+0000 7faa13c41640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f8008860 0x7faa0c090880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faa08001210 tx=0x7faa0800ec10 comp rx=0 tx=0).stop 2024-04-25T16:51:28.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.288+0000 7faa13c41640 1 -- 172.21.15.94:0/556323374 shutdown_connections 2024-04-25T16:51:28.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.288+0000 7faa13c41640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa0c0b03b0 0x7faa0c161c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.288+0000 7faa13c41640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa9f006f430 0x7fa9f0071940 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.288+0000 7faa13c41640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa0c0af390 0x7faa0c0afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.288+0000 7faa13c41640 1 --2- 172.21.15.94:0/556323374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f8008860 0x7faa0c090880 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.288+0000 7faa13c41640 1 -- 172.21.15.94:0/556323374 >> 172.21.15.94:0/556323374 conn(0x7faa0c0b47f0 msgr2=0x7faa0c073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:28.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.288+0000 7faa13c41640 1 -- 172.21.15.94:0/556323374 shutdown_connections 2024-04-25T16:51:28.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.288+0000 7faa13c41640 1 -- 172.21.15.94:0/556323374 wait complete. 2024-04-25T16:51:28.303 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:51:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f30680e5640 1 Processor -- start 2024-04-25T16:51:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f30680e5640 1 -- start start 2024-04-25T16:51:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f30680e5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f306006f470 0x7f306015dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f30680e5640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f306015e160 0x7f306015e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f30680e5640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3060070630 con 0x7f30600b0420 2024-04-25T16:51:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f30680e5640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f30600aeaa0 con 0x7f306015e160 2024-04-25T16:51:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f30680e5640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f30600703e0 con 0x7f306006f470 2024-04-25T16:51:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f3066682640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f306015e160 0x7f306015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f3066682640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f306015e160 0x7f306015e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46858/0 (socket says 172.21.15.94:46858) 2024-04-25T16:51:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f3066682640 1 -- 172.21.15.94:0/643433348 learned_addr learned my addr 172.21.15.94:0/643433348 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f3065680640 1 --2- 172.21.15.94:0/643433348 >> v2:172.21.15.138:3300/0 conn(0x7f306006f470 0x7f306015dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f3066682640 1 -- 172.21.15.94:0/643433348 >> v2:172.21.15.138:3300/0 conn(0x7f306006f470 msgr2=0x7f306015dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f3066682640 1 --2- 172.21.15.94:0/643433348 >> v2:172.21.15.138:3300/0 conn(0x7f306006f470 0x7f306015dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f3066682640 1 -- 172.21.15.94:0/643433348 >> v1:172.21.15.94:6789/0 conn(0x7f30600b0420 legacy=0x7f306006ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.456+0000 7f3066682640 1 -- 172.21.15.94:0/643433348 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30600ae170 con 0x7f306015e160 2024-04-25T16:51:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f3066682640 1 --2- 172.21.15.94:0/643433348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f306015e160 0x7f306015e570 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f30600d1190 tx=0x7f305801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c90d3685e31656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f3064e7f640 1 -- 172.21.15.94:0/643433348 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30580aa650 con 0x7f306015e160 2024-04-25T16:51:28.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f3064e7f640 1 -- 172.21.15.94:0/643433348 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30580af070 con 0x7f306015e160 2024-04-25T16:51:28.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f3064e7f640 1 -- 172.21.15.94:0/643433348 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30580aaa10 con 0x7f306015e160 2024-04-25T16:51:28.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f30680e5640 1 -- 172.21.15.94:0/643433348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f306015e160 msgr2=0x7f306015e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f30680e5640 1 --2- 172.21.15.94:0/643433348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f306015e160 0x7f306015e570 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f30600d1190 tx=0x7f305801c4f0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f30680e5640 1 -- 172.21.15.94:0/643433348 shutdown_connections 2024-04-25T16:51:28.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f30680e5640 1 --2- 172.21.15.94:0/643433348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f306015e160 0x7f306015e570 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f30680e5640 1 --2- 172.21.15.94:0/643433348 >> v2:172.21.15.138:3300/0 conn(0x7f306006f470 0x7f306015dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f30680e5640 1 -- 172.21.15.94:0/643433348 >> 172.21.15.94:0/643433348 conn(0x7f30600b47f0 msgr2=0x7f30600b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:28.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f30680e5640 1 -- 172.21.15.94:0/643433348 shutdown_connections 2024-04-25T16:51:28.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f30680e5640 1 -- 172.21.15.94:0/643433348 wait complete. 2024-04-25T16:51:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f30680e5640 1 Processor -- start 2024-04-25T16:51:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.460+0000 7f30680e5640 1 -- start start 2024-04-25T16:51:28.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f30680e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30600b0420 0x7f306009b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f30680e5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f306015e160 0x7f3060149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f30680e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f306006ed60 0x7f306006f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f30680e5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f30600b09b0 con 0x7f30600b0420 2024-04-25T16:51:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f30680e5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30600b0830 con 0x7f306006ed60 2024-04-25T16:51:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f30680e5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f306016a6f0 con 0x7f306015e160 2024-04-25T16:51:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f3065e81640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30600b0420 0x7f306009b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f3065e81640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30600b0420 0x7f306009b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46874/0 (socket says 172.21.15.94:46874) 2024-04-25T16:51:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f3065e81640 1 -- 172.21.15.94:0/1536220245 learned_addr learned my addr 172.21.15.94:0/1536220245 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f3066682640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f306006ed60 0x7f306006f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f3065680640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f306015e160 0x7f3060149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f3065e81640 1 -- 172.21.15.94:0/1536220245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f306015e160 msgr2=0x7f3060149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f3065e81640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f306015e160 0x7f3060149b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f3065e81640 1 -- 172.21.15.94:0/1536220245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f306006ed60 msgr2=0x7f306006f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f3065e81640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f306006ed60 0x7f306006f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f3065e81640 1 -- 172.21.15.94:0/1536220245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30600d0190 con 0x7f30600b0420 2024-04-25T16:51:28.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f3066682640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f306006ed60 0x7f306006f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:28.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f3065e81640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30600b0420 0x7f306009b7f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f305c00c760 tx=0x7f305c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f30567fc640 1 -- 172.21.15.94:0/1536220245 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c016020 con 0x7f30600b0420 2024-04-25T16:51:28.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f30567fc640 1 -- 172.21.15.94:0/1536220245 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f305c006d80 con 0x7f30600b0420 2024-04-25T16:51:28.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f30680e5640 1 -- 172.21.15.94:0/1536220245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306006f6b0 con 0x7f30600b0420 2024-04-25T16:51:28.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f30567fc640 1 -- 172.21.15.94:0/1536220245 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c003bc0 con 0x7f30600b0420 2024-04-25T16:51:28.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f30680e5640 1 -- 172.21.15.94:0/1536220245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f306014a9a0 con 0x7f30600b0420 2024-04-25T16:51:28.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.464+0000 7f30680e5640 1 -- 172.21.15.94:0/1536220245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3060076830 con 0x7f30600b0420 2024-04-25T16:51:28.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.468+0000 7f30567fc640 1 -- 172.21.15.94:0/1536220245 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f305c00a7f0 con 0x7f30600b0420 2024-04-25T16:51:28.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.468+0000 7f30567fc640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f304406f400 0x7f3044071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.468+0000 7f30567fc640 1 -- 172.21.15.94:0/1536220245 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f305c08ffc0 con 0x7f30600b0420 2024-04-25T16:51:28.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.468+0000 7f3065680640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f304406f400 0x7f3044071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.468+0000 7f3065680640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f304406f400 0x7f3044071910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f30600b2500 tx=0x7f3050008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.472+0000 7f30567fc640 1 -- 172.21.15.94:0/1536220245 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f305c020020 con 0x7f30600b0420 2024-04-25T16:51:28.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.628+0000 7f30680e5640 1 -- 172.21.15.94:0/1536220245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f306006fb20 con 0x7f30600b0420 2024-04-25T16:51:28.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.628+0000 7f30567fc640 1 -- 172.21.15.94:0/1536220245 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f305c05cce0 con 0x7f30600b0420 2024-04-25T16:51:28.628 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:51:28.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.632+0000 7f30680e5640 1 -- 172.21.15.94:0/1536220245 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f304406f400 msgr2=0x7f3044071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.632+0000 7f30680e5640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f304406f400 0x7f3044071910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f30600b2500 tx=0x7f3050008040 comp rx=0 tx=0).stop 2024-04-25T16:51:28.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.632+0000 7f30680e5640 1 -- 172.21.15.94:0/1536220245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30600b0420 msgr2=0x7f306009b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.632+0000 7f30680e5640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30600b0420 0x7f306009b7f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f305c00c760 tx=0x7f305c00cc20 comp rx=0 tx=0).stop 2024-04-25T16:51:28.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.632+0000 7f30680e5640 1 -- 172.21.15.94:0/1536220245 shutdown_connections 2024-04-25T16:51:28.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.632+0000 7f30680e5640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f306006ed60 0x7f306006f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.632+0000 7f30680e5640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f304406f400 0x7f3044071910 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.632+0000 7f30680e5640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f306015e160 0x7f3060149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.632+0000 7f30680e5640 1 --2- 172.21.15.94:0/1536220245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30600b0420 0x7f306009b7f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.632+0000 7f30680e5640 1 -- 172.21.15.94:0/1536220245 >> 172.21.15.94:0/1536220245 conn(0x7f30600b47f0 msgr2=0x7f30601547e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:28.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.632+0000 7f30680e5640 1 -- 172.21.15.94:0/1536220245 shutdown_connections 2024-04-25T16:51:28.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.632+0000 7f30680e5640 1 -- 172.21.15.94:0/1536220245 wait complete. 2024-04-25T16:51:28.648 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:51:28.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6caaa58640 1 Processor -- start 2024-04-25T16:51:28.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6caaa58640 1 -- start start 2024-04-25T16:51:28.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6caaa58640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40b78c0 0x7f6ca40b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6caaa58640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ca40cdc10 con 0x7f6ca40d2000 2024-04-25T16:51:28.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6caaa58640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ca40cdef0 con 0x7f6ca40b6270 2024-04-25T16:51:28.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6caaa58640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca40cfce0 con 0x7f6ca40b78c0 2024-04-25T16:51:28.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca37fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f6ca40d2000 0x7f6ca40d2410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50400/0 (socket says 172.21.15.94:50400) 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca37fe640 1 -- 172.21.15.94:0/482506105 learned_addr learned my addr 172.21.15.94:0/482506105 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca3fff640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40b78c0 0x7f6ca40b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 550685917 0 0) 0x7f6ca40cdc10 con 0x7f6ca40d2000 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c90003530 con 0x7f6ca40d2000 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1934842459 0 0) 0x7f6c90003530 con 0x7f6ca40d2000 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40b78c0 msgr2=0x7f6ca40b7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40b78c0 0x7f6ca40b7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 >> v1:172.21.15.138:6789/0 conn(0x7f6ca40b6270 legacy=0x7f6ca40b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca40ce1d0 con 0x7f6ca40d2000 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f6c8c001a00 con 0x7f6ca40d2000 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 >> v1:172.21.15.94:6789/0 conn(0x7f6ca40d2000 legacy=0x7f6ca40d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40b78c0 0x7f6c900039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c90003f70 0x7f6c90004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c90004d50 0x7f6c90005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9000bbf0 con 0x7f6c90003f70 2024-04-25T16:51:28.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9000bd70 con 0x7f6c90004d50 2024-04-25T16:51:28.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6c9000bf70 con 0x7f6ca40b78c0 2024-04-25T16:51:28.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.804+0000 7f6ca37fe640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c90003f70 0x7f6c90004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca3fff640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40b78c0 0x7f6c900039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca8ff5640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c90004d50 0x7f6c90005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca37fe640 1 -- 172.21.15.94:0/482506105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40b78c0 msgr2=0x7f6c900039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca37fe640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40b78c0 0x7f6c900039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca37fe640 1 -- 172.21.15.94:0/482506105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c90004d50 msgr2=0x7f6c90005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca37fe640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c90004d50 0x7f6c90005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca37fe640 1 -- 172.21.15.94:0/482506105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c90003530 con 0x7f6c90003f70 2024-04-25T16:51:28.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca3fff640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40b78c0 0x7f6c900039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:28.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca8ff5640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c90004d50 0x7f6c90005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:28.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca37fe640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c90003f70 0x7f6c90004810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00a850 tx=0x7f6c8c01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f10437acd53ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c00a4a0 con 0x7f6c90003f70 2024-04-25T16:51:28.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8c0ad650 con 0x7f6c90003f70 2024-04-25T16:51:28.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca2ffd640 1 -- 172.21.15.94:0/482506105 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0ada40 con 0x7f6c90003f70 2024-04-25T16:51:28.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6caaa58640 1 -- 172.21.15.94:0/482506105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c90003f70 msgr2=0x7f6c90004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6caaa58640 1 --2- 172.21.15.94:0/482506105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c90003f70 0x7f6c90004810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00a850 tx=0x7f6c8c01d3a0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6ca8ff5640 1 -- 172.21.15.94:0/482506105 reap_dead start 2024-04-25T16:51:28.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6caaa58640 1 -- 172.21.15.94:0/482506105 shutdown_connections 2024-04-25T16:51:28.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6caaa58640 1 -- 172.21.15.94:0/482506105 >> 172.21.15.94:0/482506105 conn(0x7f6ca40cfea0 msgr2=0x7f6ca40d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:28.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6caaa58640 1 -- 172.21.15.94:0/482506105 shutdown_connections 2024-04-25T16:51:28.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.808+0000 7f6caaa58640 1 -- 172.21.15.94:0/482506105 wait complete. 2024-04-25T16:51:28.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6caaa58640 1 Processor -- start 2024-04-25T16:51:28.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6caaa58640 1 -- start start 2024-04-25T16:51:28.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6caaa58640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c90003f70 0x7f6ca4148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6caaa58640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca41605c0 0x7f6ca41627e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6caaa58640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40af9d0 0x7f6ca40afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6caaa58640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca40cecb0 con 0x7f6ca41605c0 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6caaa58640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca40ce820 con 0x7f6c90003f70 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6caaa58640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca40b7cd0 con 0x7f6ca40af9d0 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca41605c0 0x7f6ca41627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca41605c0 0x7f6ca41627e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46882/0 (socket says 172.21.15.94:46882) 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca37fe640 1 -- 172.21.15.94:0/3474493236 learned_addr learned my addr 172.21.15.94:0/3474493236 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca3fff640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c90003f70 0x7f6ca4148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca37fe640 1 -- 172.21.15.94:0/3474493236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40af9d0 msgr2=0x7f6ca40afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca8ff5640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40af9d0 0x7f6ca40afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca37fe640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40af9d0 0x7f6ca40afde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca37fe640 1 -- 172.21.15.94:0/3474493236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c90003f70 msgr2=0x7f6ca4148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca37fe640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c90003f70 0x7f6ca4148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca37fe640 1 -- 172.21.15.94:0/3474493236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca4148d40 con 0x7f6ca41605c0 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca3fff640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c90003f70 0x7f6ca4148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:28.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca37fe640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca41605c0 0x7f6ca41627e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c01ce10 tx=0x7f6c90005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca0ff9640 1 -- 172.21.15.94:0/3474493236 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0bd070 con 0x7f6ca41605c0 2024-04-25T16:51:28.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca0ff9640 1 -- 172.21.15.94:0/3474493236 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8c0017c0 con 0x7f6ca41605c0 2024-04-25T16:51:28.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6caaa58640 1 -- 172.21.15.94:0/3474493236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca4162d20 con 0x7f6ca41605c0 2024-04-25T16:51:28.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6ca0ff9640 1 -- 172.21.15.94:0/3474493236 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0b2040 con 0x7f6ca41605c0 2024-04-25T16:51:28.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6caaa58640 1 -- 172.21.15.94:0/3474493236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca4163480 con 0x7f6ca41605c0 2024-04-25T16:51:28.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.812+0000 7f6caaa58640 1 -- 172.21.15.94:0/3474493236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca4162ef0 con 0x7f6ca41605c0 2024-04-25T16:51:28.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.816+0000 7f6ca0ff9640 1 -- 172.21.15.94:0/3474493236 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6c8c0c0070 con 0x7f6ca41605c0 2024-04-25T16:51:28.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.816+0000 7f6ca0ff9640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6c8406f400 0x7f6c84071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.816+0000 7f6ca0ff9640 1 -- 172.21.15.94:0/3474493236 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6c8c129620 con 0x7f6ca41605c0 2024-04-25T16:51:28.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.816+0000 7f6ca3fff640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6c8406f400 0x7f6c84071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.816+0000 7f6ca3fff640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6c8406f400 0x7f6c84071910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009ac0 tx=0x7f6c9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.820+0000 7f6ca0ff9640 1 -- 172.21.15.94:0/3474493236 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6c8c0170b0 con 0x7f6ca41605c0 2024-04-25T16:51:28.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.979+0000 7f6caaa58640 1 -- 172.21.15.94:0/3474493236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6ca406a730 con 0x7f6ca41605c0 2024-04-25T16:51:28.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.979+0000 7f6ca0ff9640 1 -- 172.21.15.94:0/3474493236 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f6c8c0f61d0 con 0x7f6ca41605c0 2024-04-25T16:51:28.979 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:28.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.983+0000 7f6caaa58640 1 -- 172.21.15.94:0/3474493236 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6c8406f400 msgr2=0x7f6c84071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.983+0000 7f6caaa58640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6c8406f400 0x7f6c84071910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009ac0 tx=0x7f6c9c008040 comp rx=0 tx=0).stop 2024-04-25T16:51:28.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.983+0000 7f6caaa58640 1 -- 172.21.15.94:0/3474493236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca41605c0 msgr2=0x7f6ca41627e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.983+0000 7f6caaa58640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca41605c0 0x7f6ca41627e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c01ce10 tx=0x7f6c90005840 comp rx=0 tx=0).stop 2024-04-25T16:51:28.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.983+0000 7f6caaa58640 1 -- 172.21.15.94:0/3474493236 shutdown_connections 2024-04-25T16:51:28.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.983+0000 7f6caaa58640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6c8406f400 0x7f6c84071910 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.983+0000 7f6caaa58640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca40af9d0 0x7f6ca40afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.983+0000 7f6caaa58640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca41605c0 0x7f6ca41627e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.983+0000 7f6caaa58640 1 --2- 172.21.15.94:0/3474493236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c90003f70 0x7f6ca4148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.983+0000 7f6caaa58640 1 -- 172.21.15.94:0/3474493236 >> 172.21.15.94:0/3474493236 conn(0x7f6ca40cfea0 msgr2=0x7f6ca40b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:28.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.983+0000 7f6caaa58640 1 -- 172.21.15.94:0/3474493236 shutdown_connections 2024-04-25T16:51:28.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:28.983+0000 7f6caaa58640 1 -- 172.21.15.94:0/3474493236 wait complete. 2024-04-25T16:51:29.000 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:51:29.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.143+0000 7f52979c9640 1 Processor -- start 2024-04-25T16:51:29.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.143+0000 7f52979c9640 1 -- start start 2024-04-25T16:51:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f52979c9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5290070180 0x7f529015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f52979c9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5290160bf0 0x7f529006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f52979c9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529015d620 con 0x7f5290161000 2024-04-25T16:51:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f52979c9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f529015dec0 con 0x7f5290070180 2024-04-25T16:51:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f52979c9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5290160760 con 0x7f5290160bf0 2024-04-25T16:51:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f5295765640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5290070180 0x7f529015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f5295765640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5290070180 0x7f529015cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46884/0 (socket says 172.21.15.94:46884) 2024-04-25T16:51:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f5295765640 1 -- 172.21.15.94:0/2440987972 learned_addr learned my addr 172.21.15.94:0/2440987972 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f5294f64640 1 --2- 172.21.15.94:0/2440987972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5290160bf0 0x7f529006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f5295765640 1 -- 172.21.15.94:0/2440987972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5290160bf0 msgr2=0x7f529006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f5295765640 1 --2- 172.21.15.94:0/2440987972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5290160bf0 0x7f529006d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f5295765640 1 -- 172.21.15.94:0/2440987972 >> v1:172.21.15.94:6789/0 conn(0x7f5290161000 legacy=0x7f52900af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f5295765640 1 -- 172.21.15.94:0/2440987972 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529015d900 con 0x7f5290070180 2024-04-25T16:51:29.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.147+0000 7f5295765640 1 --2- 172.21.15.94:0/2440987972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5290070180 0x7f529015cf40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5280009870 tx=0x7f528001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b05af8af8b416e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:29.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.151+0000 7f5287fff640 1 -- 172.21.15.94:0/2440987972 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52800a7070 con 0x7f5290070180 2024-04-25T16:51:29.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.151+0000 7f5287fff640 1 -- 172.21.15.94:0/2440987972 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52800b0070 con 0x7f5290070180 2024-04-25T16:51:29.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.151+0000 7f5287fff640 1 -- 172.21.15.94:0/2440987972 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52800ab890 con 0x7f5290070180 2024-04-25T16:51:29.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.151+0000 7f52979c9640 1 -- 172.21.15.94:0/2440987972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5290070180 msgr2=0x7f529015cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.151+0000 7f52979c9640 1 --2- 172.21.15.94:0/2440987972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5290070180 0x7f529015cf40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5280009870 tx=0x7f528001c560 comp rx=0 tx=0).stop 2024-04-25T16:51:29.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.151+0000 7f52979c9640 1 -- 172.21.15.94:0/2440987972 shutdown_connections 2024-04-25T16:51:29.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.151+0000 7f52979c9640 1 --2- 172.21.15.94:0/2440987972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5290160bf0 0x7f529006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.151+0000 7f52979c9640 1 --2- 172.21.15.94:0/2440987972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5290070180 0x7f529015cf40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.151+0000 7f52979c9640 1 -- 172.21.15.94:0/2440987972 >> 172.21.15.94:0/2440987972 conn(0x7f52900b47f0 msgr2=0x7f52900b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:29.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.151+0000 7f52979c9640 1 -- 172.21.15.94:0/2440987972 shutdown_connections 2024-04-25T16:51:29.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.151+0000 7f52979c9640 1 -- 172.21.15.94:0/2440987972 wait complete. 2024-04-25T16:51:29.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.151+0000 7f52979c9640 1 Processor -- start 2024-04-25T16:51:29.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.151+0000 7f52979c9640 1 -- start start 2024-04-25T16:51:29.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f52979c9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5290070180 0x7f529009c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f52979c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5290160bf0 0x7f529009a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f52979c9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5290161000 0x7f529009b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f52979c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f529016a2f0 con 0x7f5290160bf0 2024-04-25T16:51:29.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f52979c9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f529006e060 con 0x7f5290070180 2024-04-25T16:51:29.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f52979c9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f529016a470 con 0x7f5290161000 2024-04-25T16:51:29.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5294f64640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5290160bf0 0x7f529009a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5294f64640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5290160bf0 0x7f529009a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46894/0 (socket says 172.21.15.94:46894) 2024-04-25T16:51:29.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5294f64640 1 -- 172.21.15.94:0/1437531594 learned_addr learned my addr 172.21.15.94:0/1437531594 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:29.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5295765640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5290070180 0x7f529009c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5295f66640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5290161000 0x7f529009b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5294f64640 1 -- 172.21.15.94:0/1437531594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5290161000 msgr2=0x7f529009b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5294f64640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5290161000 0x7f529009b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5294f64640 1 -- 172.21.15.94:0/1437531594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5290070180 msgr2=0x7f529009c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5294f64640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5290070180 0x7f529009c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5294f64640 1 -- 172.21.15.94:0/1437531594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5290166b60 con 0x7f5290160bf0 2024-04-25T16:51:29.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5295765640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5290070180 0x7f529009c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:29.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5295f66640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5290161000 0x7f529009b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:29.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5294f64640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5290160bf0 0x7f529009a8b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f528c00d5f0 tx=0x7f528c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:29.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5285ffb640 1 -- 172.21.15.94:0/1437531594 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c0125f0 con 0x7f5290160bf0 2024-04-25T16:51:29.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5285ffb640 1 -- 172.21.15.94:0/1437531594 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f528c019070 con 0x7f5290160bf0 2024-04-25T16:51:29.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f52979c9640 1 -- 172.21.15.94:0/1437531594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5290166e40 con 0x7f5290160bf0 2024-04-25T16:51:29.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5285ffb640 1 -- 172.21.15.94:0/1437531594 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c0129b0 con 0x7f5290160bf0 2024-04-25T16:51:29.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f52979c9640 1 -- 172.21.15.94:0/1437531594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f529015fc50 con 0x7f5290160bf0 2024-04-25T16:51:29.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f52979c9640 1 -- 172.21.15.94:0/1437531594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5290076830 con 0x7f5290160bf0 2024-04-25T16:51:29.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.155+0000 7f5285ffb640 1 -- 172.21.15.94:0/1437531594 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f528c012b50 con 0x7f5290160bf0 2024-04-25T16:51:29.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.159+0000 7f5285ffb640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f527406f400 0x7f5274071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.159+0000 7f5285ffb640 1 -- 172.21.15.94:0/1437531594 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f528c08f9f0 con 0x7f5290160bf0 2024-04-25T16:51:29.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.159+0000 7f5295765640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f527406f400 0x7f5274071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.159+0000 7f5295765640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f527406f400 0x7f5274071910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f52800af000 tx=0x7f5280008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:29.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.163+0000 7f5285ffb640 1 -- 172.21.15.94:0/1437531594 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f528c05c630 con 0x7f5290160bf0 2024-04-25T16:51:29.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.351+0000 7f52979c9640 1 -- 172.21.15.94:0/1437531594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52900b2b60 con 0x7f5290160bf0 2024-04-25T16:51:29.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.351+0000 7f5285ffb640 1 -- 172.21.15.94:0/1437531594 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f528c05bf30 con 0x7f5290160bf0 2024-04-25T16:51:29.351 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:29.351 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2352186041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2352186041},{"type":"v1","addr":"172.21.15.138:6833","nonce":2352186041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3210927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3210927873},{"type":"v1","addr":"172.21.15.94:6835","nonce":3210927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2237413225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2237413225},{"type":"v1","addr":"172.21.15.94:6837","nonce":2237413225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:50:54.403299+0000","modified":"2024-04-25T16:51:18.139471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/3652545132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3652545132},{"type":"v1","addr":"172.21.15.138:6835","nonce":3652545132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T16:51:29.351 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:51:29.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.355+0000 7f52979c9640 1 -- 172.21.15.94:0/1437531594 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f527406f400 msgr2=0x7f5274071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.355+0000 7f52979c9640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f527406f400 0x7f5274071910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f52800af000 tx=0x7f5280008040 comp rx=0 tx=0).stop 2024-04-25T16:51:29.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.355+0000 7f52979c9640 1 -- 172.21.15.94:0/1437531594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5290160bf0 msgr2=0x7f529009a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.355+0000 7f52979c9640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5290160bf0 0x7f529009a8b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f528c00d5f0 tx=0x7f528c00dab0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.355+0000 7f52979c9640 1 -- 172.21.15.94:0/1437531594 shutdown_connections 2024-04-25T16:51:29.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.355+0000 7f52979c9640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f527406f400 0x7f5274071910 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.355+0000 7f52979c9640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5290161000 0x7f529009b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.355+0000 7f52979c9640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5290160bf0 0x7f529009a8b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.355+0000 7f52979c9640 1 --2- 172.21.15.94:0/1437531594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5290070180 0x7f529009c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.355+0000 7f52979c9640 1 -- 172.21.15.94:0/1437531594 >> 172.21.15.94:0/1437531594 conn(0x7f52900b47f0 msgr2=0x7f52900b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:29.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.355+0000 7f52979c9640 1 -- 172.21.15.94:0/1437531594 shutdown_connections 2024-04-25T16:51:29.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.355+0000 7f52979c9640 1 -- 172.21.15.94:0/1437531594 wait complete. 2024-04-25T16:51:29.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:29.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f9a2d640 1 Processor -- start 2024-04-25T16:51:29.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f9a2d640 1 -- start start 2024-04-25T16:51:29.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f9a2d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6f406f6c0 0x7fc6f40d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f9a2d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6f40d3e20 con 0x7fc6f40d3a10 2024-04-25T16:51:29.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f9a2d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6f40b7e70 con 0x7fc6f415ff50 2024-04-25T16:51:29.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f9a2d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f40d2fd0 con 0x7fc6f406f6c0 2024-04-25T16:51:29.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f37fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc6f40d3a10 0x7fc6f40b7760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50412/0 (socket says 172.21.15.94:50412) 2024-04-25T16:51:29.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f37fe640 1 -- 172.21.15.94:0/314234668 learned_addr learned my addr 172.21.15.94:0/314234668 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:29.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2077814511 0 0) 0x7fc6ec0014a0 con 0x7fc6f40d3a10 2024-04-25T16:51:29.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6e0003530 con 0x7fc6f40d3a10 2024-04-25T16:51:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f2ffd640 1 --2- 172.21.15.94:0/314234668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6f406f6c0 0x7fc6f40d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 523908174 0 0) 0x7fc6e0003530 con 0x7fc6f40d3a10 2024-04-25T16:51:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6f406f6c0 msgr2=0x7fc6f40d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 --2- 172.21.15.94:0/314234668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6f406f6c0 0x7fc6f40d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 >> v1:172.21.15.138:6789/0 conn(0x7fc6f415ff50 legacy=0x7fc6f40b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f40d1d40 con 0x7fc6f40d3a10 2024-04-25T16:51:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fc6ec001bd0 con 0x7fc6f40d3a10 2024-04-25T16:51:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 >> v1:172.21.15.94:6789/0 conn(0x7fc6f40d3a10 legacy=0x7fc6f40b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 --2- 172.21.15.94:0/314234668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e00039f0 0x7fc6e0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 --2- 172.21.15.94:0/314234668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6e0004810 0x7fc6e0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 --2- 172.21.15.94:0/314234668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6f406f6c0 0x7fc6e00052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f40d2fd0 con 0x7fc6e0004810 2024-04-25T16:51:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e000bbb0 con 0x7fc6e00039f0 2024-04-25T16:51:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e000bdb0 con 0x7fc6f406f6c0 2024-04-25T16:51:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6ea7fc640 1 --2- 172.21.15.94:0/314234668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6e0004810 0x7fc6e0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f37fe640 1 --2- 172.21.15.94:0/314234668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6f406f6c0 0x7fc6e00052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6f2ffd640 1 --2- 172.21.15.94:0/314234668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e00039f0 0x7fc6e0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6ea7fc640 1 -- 172.21.15.94:0/314234668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6f406f6c0 msgr2=0x7fc6e00052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6ea7fc640 1 --2- 172.21.15.94:0/314234668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6f406f6c0 0x7fc6e00052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6ea7fc640 1 -- 172.21.15.94:0/314234668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e00039f0 msgr2=0x7fc6e0003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6ea7fc640 1 --2- 172.21.15.94:0/314234668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e00039f0 0x7fc6e0003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.507+0000 7fc6ea7fc640 1 -- 172.21.15.94:0/314234668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e0003530 con 0x7fc6e0004810 2024-04-25T16:51:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.511+0000 7fc6ea7fc640 1 --2- 172.21.15.94:0/314234668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6e0004810 0x7fc6e0004c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc6f41638e0 tx=0x7fc6dc01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4781928e797498a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.511+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0013a0 con 0x7fc6e0004810 2024-04-25T16:51:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.511+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6dc0052f0 con 0x7fc6e0004810 2024-04-25T16:51:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.511+0000 7fc6f27fc640 1 -- 172.21.15.94:0/314234668 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc003840 con 0x7fc6e0004810 2024-04-25T16:51:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.511+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/314234668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6e0004810 msgr2=0x7fc6e0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.511+0000 7fc6f9a2d640 1 --2- 172.21.15.94:0/314234668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6e0004810 0x7fc6e0004c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc6f41638e0 tx=0x7fc6dc01c380 comp rx=0 tx=0).stop 2024-04-25T16:51:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.511+0000 7fc6f37fe640 1 -- 172.21.15.94:0/314234668 reap_dead start 2024-04-25T16:51:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.511+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/314234668 shutdown_connections 2024-04-25T16:51:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.511+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/314234668 >> 172.21.15.94:0/314234668 conn(0x7fc6f409e950 msgr2=0x7fc6f408d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.511+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/314234668 shutdown_connections 2024-04-25T16:51:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.511+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/314234668 wait complete. 2024-04-25T16:51:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f9a2d640 1 Processor -- start 2024-04-25T16:51:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f9a2d640 1 -- start start 2024-04-25T16:51:29.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f9a2d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e0004810 0x7fc6f4155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f9a2d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f406e390 0x7fc6f40b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f9a2d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6f406e7a0 0x7fc6f40b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f9a2d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f40d2aa0 con 0x7fc6f406e390 2024-04-25T16:51:29.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f9a2d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f40d2cf0 con 0x7fc6e0004810 2024-04-25T16:51:29.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f9a2d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f40d5e00 con 0x7fc6f406e7a0 2024-04-25T16:51:29.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6ea7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f406e390 0x7fc6f40b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6ea7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f406e390 0x7fc6f40b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46910/0 (socket says 172.21.15.94:46910) 2024-04-25T16:51:29.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6ea7fc640 1 -- 172.21.15.94:0/1659761663 learned_addr learned my addr 172.21.15.94:0/1659761663 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:29.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f2ffd640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e0004810 0x7fc6f4155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f37fe640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6f406e7a0 0x7fc6f40b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6ea7fc640 1 -- 172.21.15.94:0/1659761663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6f406e7a0 msgr2=0x7fc6f40b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6ea7fc640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6f406e7a0 0x7fc6f40b66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6ea7fc640 1 -- 172.21.15.94:0/1659761663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e0004810 msgr2=0x7fc6f4155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6ea7fc640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e0004810 0x7fc6f4155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6ea7fc640 1 -- 172.21.15.94:0/1659761663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f4156230 con 0x7fc6f406e390 2024-04-25T16:51:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6ea7fc640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f406e390 0x7fc6f40b61b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc01c860 tx=0x7fc6e0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f0ff9640 1 -- 172.21.15.94:0/1659761663 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc000ee0 con 0x7fc6f406e390 2024-04-25T16:51:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f0ff9640 1 -- 172.21.15.94:0/1659761663 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6dc0054a0 con 0x7fc6f406e390 2024-04-25T16:51:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/1659761663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f4160d70 con 0x7fc6f406e390 2024-04-25T16:51:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f0ff9640 1 -- 172.21.15.94:0/1659761663 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0067e0 con 0x7fc6f406e390 2024-04-25T16:51:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/1659761663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f41614f0 con 0x7fc6f406e390 2024-04-25T16:51:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.515+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/1659761663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f4076830 con 0x7fc6f406e390 2024-04-25T16:51:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.519+0000 7fc6f0ff9640 1 -- 172.21.15.94:0/1659761663 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc6dc0bd020 con 0x7fc6f406e390 2024-04-25T16:51:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.519+0000 7fc6f0ff9640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc6d406f5a0 0x7fc6d4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.519+0000 7fc6f0ff9640 1 -- 172.21.15.94:0/1659761663 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc6dc12a310 con 0x7fc6f406e390 2024-04-25T16:51:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.519+0000 7fc6f2ffd640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc6d406f5a0 0x7fc6d4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.519+0000 7fc6f2ffd640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc6d406f5a0 0x7fc6d4071ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4006fd0 tx=0x7fc6e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:29.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.523+0000 7fc6f0ff9640 1 -- 172.21.15.94:0/1659761663 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc6dc0c4020 con 0x7fc6f406e390 2024-04-25T16:51:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.683+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/1659761663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6f4160fa0 con 0x7fc6f406e390 2024-04-25T16:51:29.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.687+0000 7fc6f0ff9640 1 -- 172.21.15.94:0/1659761663 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7fc6dc016090 con 0x7fc6f406e390 2024-04-25T16:51:29.684 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:29.684 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:50:59.528743+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:29.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.687+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/1659761663 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc6d406f5a0 msgr2=0x7fc6d4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.687+0000 7fc6f9a2d640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc6d406f5a0 0x7fc6d4071ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4006fd0 tx=0x7fc6e4008040 comp rx=0 tx=0).stop 2024-04-25T16:51:29.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.687+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/1659761663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f406e390 msgr2=0x7fc6f40b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.687+0000 7fc6f9a2d640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f406e390 0x7fc6f40b61b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc01c860 tx=0x7fc6e0005840 comp rx=0 tx=0).stop 2024-04-25T16:51:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.687+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/1659761663 shutdown_connections 2024-04-25T16:51:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.687+0000 7fc6f9a2d640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc6d406f5a0 0x7fc6d4071ab0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.687+0000 7fc6f9a2d640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc6f406e7a0 0x7fc6f40b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.687+0000 7fc6f9a2d640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f406e390 0x7fc6f40b61b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.687+0000 7fc6f9a2d640 1 --2- 172.21.15.94:0/1659761663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e0004810 0x7fc6f4155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.687+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/1659761663 >> 172.21.15.94:0/1659761663 conn(0x7fc6f409e950 msgr2=0x7fc6f40b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.687+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/1659761663 shutdown_connections 2024-04-25T16:51:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.687+0000 7fc6f9a2d640 1 -- 172.21.15.94:0/1659761663 wait complete. 2024-04-25T16:51:29.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:51:29.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.843+0000 7faa8e82c640 1 Processor -- start 2024-04-25T16:51:29.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.843+0000 7faa8e82c640 1 -- start start 2024-04-25T16:51:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.843+0000 7faa8e82c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faa880d3be0 0x7faa880d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.843+0000 7faa8e82c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa880d1d40 con 0x7faa8815ff50 2024-04-25T16:51:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.843+0000 7faa8e82c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa880b2240 con 0x7faa8806f6c0 2024-04-25T16:51:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.843+0000 7faa8e82c640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7faa880d2fd0 con 0x7faa880d3be0 2024-04-25T16:51:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.843+0000 7faa877fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faa880d3be0 0x7faa880d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.843+0000 7faa877fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faa880d3be0 0x7faa880d4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46924/0 (socket says 172.21.15.94:46924) 2024-04-25T16:51:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.843+0000 7faa877fe640 1 -- 172.21.15.94:0/4038663469 learned_addr learned my addr 172.21.15.94:0/4038663469 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.843+0000 7faa877fe640 1 -- 172.21.15.94:0/4038663469 >> v1:172.21.15.138:6789/0 conn(0x7faa8806f6c0 legacy=0x7faa880d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.843+0000 7faa877fe640 1 -- 172.21.15.94:0/4038663469 >> v1:172.21.15.94:6789/0 conn(0x7faa8815ff50 legacy=0x7faa880d5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.843+0000 7faa877fe640 1 -- 172.21.15.94:0/4038663469 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa880d2cf0 con 0x7faa880d3be0 2024-04-25T16:51:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.847+0000 7faa877fe640 1 --2- 172.21.15.94:0/4038663469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa880d3be0 0x7faa880d4df0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faa70009870 tx=0x7faa7001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa9f0bbeb529701 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.847+0000 7faa86ffd640 1 -- 172.21.15.94:0/4038663469 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa700a9400 con 0x7faa880d3be0 2024-04-25T16:51:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.847+0000 7faa86ffd640 1 -- 172.21.15.94:0/4038663469 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa700a95a0 con 0x7faa880d3be0 2024-04-25T16:51:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.847+0000 7faa86ffd640 1 -- 172.21.15.94:0/4038663469 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa700a9970 con 0x7faa880d3be0 2024-04-25T16:51:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.847+0000 7faa8e82c640 1 -- 172.21.15.94:0/4038663469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa880d3be0 msgr2=0x7faa880d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.847+0000 7faa8e82c640 1 --2- 172.21.15.94:0/4038663469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa880d3be0 0x7faa880d4df0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faa70009870 tx=0x7faa7001c520 comp rx=0 tx=0).stop 2024-04-25T16:51:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.847+0000 7faa8e82c640 1 -- 172.21.15.94:0/4038663469 shutdown_connections 2024-04-25T16:51:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.847+0000 7faa8e82c640 1 --2- 172.21.15.94:0/4038663469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa880d3be0 0x7faa880d4df0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.847+0000 7faa8e82c640 1 -- 172.21.15.94:0/4038663469 >> 172.21.15.94:0/4038663469 conn(0x7faa8809e950 msgr2=0x7faa8808d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.847+0000 7faa8e82c640 1 -- 172.21.15.94:0/4038663469 shutdown_connections 2024-04-25T16:51:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.847+0000 7faa8e82c640 1 -- 172.21.15.94:0/4038663469 wait complete. 2024-04-25T16:51:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa8e82c640 1 Processor -- start 2024-04-25T16:51:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa8e82c640 1 -- start start 2024-04-25T16:51:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa8e82c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa8806f6c0 0x7faa8806cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa8e82c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8815ff50 0x7faa8814c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa8e82c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa880c7bf0 0x7faa8814c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa8e82c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa880d2aa0 con 0x7faa8815ff50 2024-04-25T16:51:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa8e82c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa8806fe20 con 0x7faa880c7bf0 2024-04-25T16:51:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa8e82c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa8816e810 con 0x7faa8806f6c0 2024-04-25T16:51:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8815ff50 0x7faa8814c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8815ff50 0x7faa8814c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46926/0 (socket says 172.21.15.94:46926) 2024-04-25T16:51:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa877fe640 1 -- 172.21.15.94:0/2571447719 learned_addr learned my addr 172.21.15.94:0/2571447719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa8cdc9640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa880c7bf0 0x7faa8814c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa87fff640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa8806f6c0 0x7faa8806cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa877fe640 1 -- 172.21.15.94:0/2571447719 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa8806f6c0 msgr2=0x7faa8806cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa877fe640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa8806f6c0 0x7faa8806cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa877fe640 1 -- 172.21.15.94:0/2571447719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa880c7bf0 msgr2=0x7faa8814c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa877fe640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa880c7bf0 0x7faa8814c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa877fe640 1 -- 172.21.15.94:0/2571447719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8806d080 con 0x7faa8815ff50 2024-04-25T16:51:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa87fff640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa8806f6c0 0x7faa8806cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa877fe640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8815ff50 0x7faa8814c0f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faa7001abd0 tx=0x7faa700192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa84ff9640 1 -- 172.21.15.94:0/2571447719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa700b1800 con 0x7faa8815ff50 2024-04-25T16:51:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa84ff9640 1 -- 172.21.15.94:0/2571447719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa700b19a0 con 0x7faa8815ff50 2024-04-25T16:51:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa8e82c640 1 -- 172.21.15.94:0/2571447719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8814cb70 con 0x7faa8815ff50 2024-04-25T16:51:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa84ff9640 1 -- 172.21.15.94:0/2571447719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa700b1d90 con 0x7faa8815ff50 2024-04-25T16:51:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa8e82c640 1 -- 172.21.15.94:0/2571447719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa880cb5b0 con 0x7faa8815ff50 2024-04-25T16:51:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa8e82c640 1 -- 172.21.15.94:0/2571447719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa88076830 con 0x7faa8815ff50 2024-04-25T16:51:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.851+0000 7faa84ff9640 1 -- 172.21.15.94:0/2571447719 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faa700b93e0 con 0x7faa8815ff50 2024-04-25T16:51:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.855+0000 7faa84ff9640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa6806f430 0x7faa68071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.855+0000 7faa84ff9640 1 -- 172.21.15.94:0/2571447719 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faa70129ce0 con 0x7faa8815ff50 2024-04-25T16:51:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.855+0000 7faa87fff640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa6806f430 0x7faa68071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.855+0000 7faa87fff640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa6806f430 0x7faa68071940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faa880d0280 tx=0x7faa78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:29.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:29.859+0000 7faa84ff9640 1 -- 172.21.15.94:0/2571447719 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faa700c0060 con 0x7faa8815ff50 2024-04-25T16:51:30.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.039+0000 7faa8e82c640 1 -- 172.21.15.94:0/2571447719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa88159290 con 0x7faa8815ff50 2024-04-25T16:51:30.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.039+0000 7faa84ff9640 1 -- 172.21.15.94:0/2571447719 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7faa700ad020 con 0x7faa8815ff50 2024-04-25T16:51:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:30.038 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2352186041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2352186041},{"type":"v1","addr":"172.21.15.138:6833","nonce":2352186041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3210927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3210927873},{"type":"v1","addr":"172.21.15.94:6835","nonce":3210927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2237413225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2237413225},{"type":"v1","addr":"172.21.15.94:6837","nonce":2237413225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:50:54.403299+0000","modified":"2024-04-25T16:51:18.139471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/3652545132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3652545132},{"type":"v1","addr":"172.21.15.138:6835","nonce":3652545132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T16:51:30.038 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:51:30.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.039+0000 7faa8e82c640 1 -- 172.21.15.94:0/2571447719 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa6806f430 msgr2=0x7faa68071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:30.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.039+0000 7faa8e82c640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa6806f430 0x7faa68071940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faa880d0280 tx=0x7faa78008040 comp rx=0 tx=0).stop 2024-04-25T16:51:30.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.043+0000 7faa8e82c640 1 -- 172.21.15.94:0/2571447719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8815ff50 msgr2=0x7faa8814c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:30.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.043+0000 7faa8e82c640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8815ff50 0x7faa8814c0f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faa7001abd0 tx=0x7faa700192c0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.043+0000 7faa8e82c640 1 -- 172.21.15.94:0/2571447719 shutdown_connections 2024-04-25T16:51:30.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.043+0000 7faa8e82c640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa880c7bf0 0x7faa8814c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.043+0000 7faa8e82c640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8815ff50 0x7faa8814c0f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.043+0000 7faa8e82c640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa6806f430 0x7faa68071940 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.043+0000 7faa8e82c640 1 --2- 172.21.15.94:0/2571447719 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa8806f6c0 0x7faa8806cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.043+0000 7faa8e82c640 1 -- 172.21.15.94:0/2571447719 >> 172.21.15.94:0/2571447719 conn(0x7faa8809e950 msgr2=0x7faa880b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:30.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.043+0000 7faa8e82c640 1 -- 172.21.15.94:0/2571447719 shutdown_connections 2024-04-25T16:51:30.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.043+0000 7faa8e82c640 1 -- 172.21.15.94:0/2571447719 wait complete. 2024-04-25T16:51:30.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T16:50:54.403299+0000', 'modified': '2024-04-25T16:51:18.139471+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4297}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4297': {'gid': 4297, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6835/3652545132', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 3652545132}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 3652545132}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4297, 'qdb_cluster': [4297]} 2024-04-25T16:51:30.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T16:51:30.461 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T16:51:30.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935c25d640 1 Processor -- start 2024-04-25T16:51:30.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935c25d640 1 -- start start 2024-04-25T16:51:30.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935c25d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f93540b0420 0x7f935406ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935c25d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f935415ec30 0x7f935406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935c25d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9354070630 con 0x7f935406f2a0 2024-04-25T16:51:30.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935c25d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f93540aeaa0 con 0x7f935415ec30 2024-04-25T16:51:30.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935c25d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f93540703e0 con 0x7f93540b0420 2024-04-25T16:51:30.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935a7fa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f935415ec30 0x7f935406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935a7fa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f935415ec30 0x7f935406cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46936/0 (socket says 172.21.15.94:46936) 2024-04-25T16:51:30.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935a7fa640 1 -- 172.21.15.94:0/2657985173 learned_addr learned my addr 172.21.15.94:0/2657985173 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:30.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f9359ff9640 1 --2- 172.21.15.94:0/2657985173 >> v2:172.21.15.138:3300/0 conn(0x7f93540b0420 0x7f935406ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935a7fa640 1 -- 172.21.15.94:0/2657985173 >> v2:172.21.15.138:3300/0 conn(0x7f93540b0420 msgr2=0x7f935406ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:30.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935a7fa640 1 --2- 172.21.15.94:0/2657985173 >> v2:172.21.15.138:3300/0 conn(0x7f93540b0420 0x7f935406ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935a7fa640 1 -- 172.21.15.94:0/2657985173 >> v1:172.21.15.138:6789/0 conn(0x7f935406f2a0 legacy=0x7f935406c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:30.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.623+0000 7f935a7fa640 1 -- 172.21.15.94:0/2657985173 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935406f6b0 con 0x7f935415ec30 2024-04-25T16:51:30.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.627+0000 7f935a7fa640 1 --2- 172.21.15.94:0/2657985173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935415ec30 0x7f935406cd10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9354071670 tx=0x7f934c01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa665f28c4595d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:30.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.627+0000 7f9358ff7640 1 -- 172.21.15.94:0/2657985173 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c0a5840 con 0x7f935415ec30 2024-04-25T16:51:30.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.627+0000 7f9358ff7640 1 -- 172.21.15.94:0/2657985173 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f934c0a59e0 con 0x7f935415ec30 2024-04-25T16:51:30.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.627+0000 7f9358ff7640 1 -- 172.21.15.94:0/2657985173 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c0a5db0 con 0x7f935415ec30 2024-04-25T16:51:30.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.627+0000 7f935c25d640 1 -- 172.21.15.94:0/2657985173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935415ec30 msgr2=0x7f935406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:30.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.627+0000 7f935c25d640 1 --2- 172.21.15.94:0/2657985173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935415ec30 0x7f935406cd10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9354071670 tx=0x7f934c01c980 comp rx=0 tx=0).stop 2024-04-25T16:51:30.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.627+0000 7f935c25d640 1 -- 172.21.15.94:0/2657985173 shutdown_connections 2024-04-25T16:51:30.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.627+0000 7f935c25d640 1 --2- 172.21.15.94:0/2657985173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935415ec30 0x7f935406cd10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.627+0000 7f935c25d640 1 --2- 172.21.15.94:0/2657985173 >> v2:172.21.15.138:3300/0 conn(0x7f93540b0420 0x7f935406ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.627+0000 7f935c25d640 1 -- 172.21.15.94:0/2657985173 >> 172.21.15.94:0/2657985173 conn(0x7f93540b47f0 msgr2=0x7f93540b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:30.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.627+0000 7f935c25d640 1 -- 172.21.15.94:0/2657985173 shutdown_connections 2024-04-25T16:51:30.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.627+0000 7f935c25d640 1 -- 172.21.15.94:0/2657985173 wait complete. 2024-04-25T16:51:30.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f935c25d640 1 Processor -- start 2024-04-25T16:51:30.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f935c25d640 1 -- start start 2024-04-25T16:51:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f935c25d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f935406f2a0 0x7f935406dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f935c25d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93540b0420 0x7f9354155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f935c25d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f935415ec30 0x7f93541493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f935c25d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f93540b0830 con 0x7f93540b0420 2024-04-25T16:51:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f935c25d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93540aef30 con 0x7f935415ec30 2024-04-25T16:51:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f935c25d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f93540b09b0 con 0x7f935406f2a0 2024-04-25T16:51:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f93597f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93540b0420 0x7f9354155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f93597f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93540b0420 0x7f9354155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46944/0 (socket says 172.21.15.94:46944) 2024-04-25T16:51:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f93597f8640 1 -- 172.21.15.94:0/2124813864 learned_addr learned my addr 172.21.15.94:0/2124813864 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f9359ff9640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f935406f2a0 0x7f935406dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f935a7fa640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f935415ec30 0x7f93541493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f93597f8640 1 -- 172.21.15.94:0/2124813864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f935406f2a0 msgr2=0x7f935406dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:30.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f93597f8640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f935406f2a0 0x7f935406dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f93597f8640 1 -- 172.21.15.94:0/2124813864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f935415ec30 msgr2=0x7f93541493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:30.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f93597f8640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f935415ec30 0x7f93541493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f93597f8640 1 -- 172.21.15.94:0/2124813864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93540d0d40 con 0x7f93540b0420 2024-04-25T16:51:30.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f93597f8640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93540b0420 0x7f9354155390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f934400b410 tx=0x7f934400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:30.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f934a7fc640 1 -- 172.21.15.94:0/2124813864 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9344015030 con 0x7f93540b0420 2024-04-25T16:51:30.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f934a7fc640 1 -- 172.21.15.94:0/2124813864 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9344004020 con 0x7f93540b0420 2024-04-25T16:51:30.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f935c25d640 1 -- 172.21.15.94:0/2124813864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93540d0780 con 0x7f93540b0420 2024-04-25T16:51:30.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f934a7fc640 1 -- 172.21.15.94:0/2124813864 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9344004450 con 0x7f93540b0420 2024-04-25T16:51:30.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.631+0000 7f935c25d640 1 -- 172.21.15.94:0/2124813864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935414d1a0 con 0x7f93540b0420 2024-04-25T16:51:30.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.635+0000 7f934a7fc640 1 -- 172.21.15.94:0/2124813864 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9344018070 con 0x7f93540b0420 2024-04-25T16:51:30.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.635+0000 7f934a7fc640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f933806f430 0x7f9338071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.635+0000 7f934a7fc640 1 -- 172.21.15.94:0/2124813864 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f934408f120 con 0x7f93540b0420 2024-04-25T16:51:30.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.635+0000 7f9359ff9640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f933806f430 0x7f9338071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.635+0000 7f9359ff9640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f933806f430 0x7f9338071940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f93540cd2f0 tx=0x7f9350008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:30.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.635+0000 7f93635ba000 1 Processor -- start 2024-04-25T16:51:30.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.635+0000 7f93635ba000 1 -- start start 2024-04-25T16:51:30.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.635+0000 7f93635ba000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c9e0c20 0x56096cb43340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.635+0000 7f93635ba000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9ccea0 0x56096cb43880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.635+0000 7f93635ba000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c5c5280 0x56096cb43dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.635+0000 7f93635ba000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56096c9ae160 con 0x56096c5c5280 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.635+0000 7f93635ba000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56096c943e20 con 0x56096c9ccea0 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.635+0000 7f93635ba000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56096c8fca20 con 0x56096c9e0c20 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f935a7fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c5c5280 0x56096cb43dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f935a7fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c5c5280 0x56096cb43dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46948/0 (socket says 172.21.15.94:46948) 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f935a7fa640 1 -- 172.21.15.94:0/2933679582 learned_addr learned my addr 172.21.15.94:0/2933679582 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f9359ff9640 1 --2- 172.21.15.94:0/2933679582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c9e0c20 0x56096cb43340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93597f8640 1 --2- 172.21.15.94:0/2933679582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9ccea0 0x56096cb43880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f935a7fa640 1 -- 172.21.15.94:0/2933679582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c9e0c20 msgr2=0x56096cb43340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f935a7fa640 1 --2- 172.21.15.94:0/2933679582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c9e0c20 0x56096cb43340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f935a7fa640 1 -- 172.21.15.94:0/2933679582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9ccea0 msgr2=0x56096cb43880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:30.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f935a7fa640 1 --2- 172.21.15.94:0/2933679582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9ccea0 0x56096cb43880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f935a7fa640 1 -- 172.21.15.94:0/2933679582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56096c901100 con 0x56096c5c5280 2024-04-25T16:51:30.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f935a7fa640 1 --2- 172.21.15.94:0/2933679582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c5c5280 0x56096cb43dc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f934c01b030 tx=0x7f934c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b672bc7df3a1bec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:30.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93377fe640 1 -- 172.21.15.94:0/2933679582 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c0bd070 con 0x56096c5c5280 2024-04-25T16:51:30.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93377fe640 1 -- 172.21.15.94:0/2933679582 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f934c0a5840 con 0x56096c5c5280 2024-04-25T16:51:30.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93377fe640 1 -- 172.21.15.94:0/2933679582 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c0a5c30 con 0x56096c5c5280 2024-04-25T16:51:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 -- 172.21.15.94:0/2933679582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c5c5280 msgr2=0x56096cb43dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 --2- 172.21.15.94:0/2933679582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c5c5280 0x56096cb43dc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f934c01b030 tx=0x7f934c004ac0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 -- 172.21.15.94:0/2933679582 shutdown_connections 2024-04-25T16:51:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 --2- 172.21.15.94:0/2933679582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c5c5280 0x56096cb43dc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 --2- 172.21.15.94:0/2933679582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9ccea0 0x56096cb43880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 --2- 172.21.15.94:0/2933679582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c9e0c20 0x56096cb43340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 -- 172.21.15.94:0/2933679582 >> 172.21.15.94:0/2933679582 conn(0x56096c604ff0 msgr2=0x56096caf44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 -- 172.21.15.94:0/2933679582 shutdown_connections 2024-04-25T16:51:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 -- 172.21.15.94:0/2933679582 wait complete. 2024-04-25T16:51:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 -- start start 2024-04-25T16:51:30.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 Processor -- start 2024-04-25T16:51:30.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c5c5280 0x56096cb48810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb48d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9e0c20 0x56096cb03440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56096c951220 con 0x56096c9ccea0 2024-04-25T16:51:30.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56096c8e7ea0 con 0x56096c9e0c20 2024-04-25T16:51:30.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.639+0000 7f93635ba000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56096c93eb00 con 0x56096c5c5280 2024-04-25T16:51:30.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93635ba000 10 client.0 _ll_register_callbacks cb 0x56096c950dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:30.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93597f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb48d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93597f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb48d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46960/0 (socket says 172.21.15.94:46960) 2024-04-25T16:51:30.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93597f8640 1 -- 172.21.15.94:0/2663885759 learned_addr learned my addr 172.21.15.94:0/2663885759 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:30.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f935a7fa640 1 --2- 172.21.15.94:0/2663885759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9e0c20 0x56096cb03440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f9359ff9640 1 --2- 172.21.15.94:0/2663885759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c5c5280 0x56096cb48810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93597f8640 1 -- 172.21.15.94:0/2663885759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c5c5280 msgr2=0x56096cb48810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:30.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93597f8640 1 --2- 172.21.15.94:0/2663885759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c5c5280 0x56096cb48810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93597f8640 1 -- 172.21.15.94:0/2663885759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9e0c20 msgr2=0x56096cb03440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93597f8640 1 --2- 172.21.15.94:0/2663885759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9e0c20 0x56096cb03440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93597f8640 1 -- 172.21.15.94:0/2663885759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56096c8e3500 con 0x56096c9ccea0 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93597f8640 1 --2- 172.21.15.94:0/2663885759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb48d50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9344057bc0 tx=0x7f934405b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa20606adeb672a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f9334ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f9334ff9640 1 -- 172.21.15.94:0/2663885759 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934408b060 con 0x56096c9ccea0 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f9334ff9640 1 -- 172.21.15.94:0/2663885759 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9344056f40 con 0x56096c9ccea0 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93635ba000 1 -- 172.21.15.94:0/2663885759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f934c0a5840 con 0x56096c9ccea0 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f9334ff9640 1 -- 172.21.15.94:0/2663885759 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f9344079050 con 0x56096c9ccea0 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93635ba000 10 client.4818 fetch_fsmap learned FSMap version 9 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93635ba000 1 -- 172.21.15.94:0/2663885759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x56096c8c9ea0 con 0x56096c9ccea0 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f9334ff9640 1 -- 172.21.15.94:0/2663885759 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f934c0a5840 con 0x56096c9ccea0 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f9334ff9640 1 -- 172.21.15.94:0/2663885759 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f9344022060 con 0x56096c9ccea0 2024-04-25T16:51:30.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93635ba000 10 client.4818 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T16:51:30.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93635ba000 10 client.4818 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T16:51:30.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93635ba000 20 client.4818 populate_metadata read hostname 'smithi094' 2024-04-25T16:51:30.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93635ba000 1 --2- 172.21.15.94:0/2663885759 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x56096c5e97d0 0x56096cb041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93635ba000 4 client.4818 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:51:30.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f93635ba000 1 -- 172.21.15.94:0/2663885759 --> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56096c8dcbe0 con 0x56096c5e97d0 2024-04-25T16:51:30.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f9359ff9640 1 --2- 172.21.15.94:0/2663885759 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x56096c5e97d0 0x56096cb041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f9334ff9640 10 client.4818 ms_handle_connect on v2:172.21.15.138:6834/3652545132 2024-04-25T16:51:30.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.643+0000 7f9359ff9640 1 --2- 172.21.15.94:0/2663885759 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x56096c5e97d0 0x56096cb041e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea7338103b02273f server_cookie=d61e1f2ecead7cb5 in_seq=0 out_seq=0 2024-04-25T16:51:30.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f9334ff9640 1 -- 172.21.15.94:0/2663885759 <== mds.0 v2:172.21.15.138:6834/3652545132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56096c8dcbe0 con 0x56096c5e97d0 2024-04-25T16:51:30.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f9334ff9640 10 client.4818 handle_command_reply: tid=0 2024-04-25T16:51:30.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f93635ba000 1 client.4818 shutdown 2024-04-25T16:51:30.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f93635ba000 1 -- 172.21.15.94:0/2663885759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 msgr2=0x56096cb48d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:30.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f93635ba000 1 --2- 172.21.15.94:0/2663885759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb48d50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9344057bc0 tx=0x7f934405b0e0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f93635ba000 1 -- 172.21.15.94:0/2663885759 shutdown_connections 2024-04-25T16:51:30.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f93635ba000 1 --2- 172.21.15.94:0/2663885759 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x56096c5e97d0 0x56096cb041e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f93635ba000 1 --2- 172.21.15.94:0/2663885759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9e0c20 0x56096cb03440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f93635ba000 1 --2- 172.21.15.94:0/2663885759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb48d50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f9334ff9640 0 client.4818 ms_handle_reset on v2:172.21.15.138:6834/3652545132 2024-04-25T16:51:30.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f93635ba000 1 --2- 172.21.15.94:0/2663885759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c5c5280 0x56096cb48810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f93635ba000 1 -- 172.21.15.94:0/2663885759 >> 172.21.15.94:0/2663885759 conn(0x56096c604ff0 msgr2=0x56096cafb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:30.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f93635ba000 1 -- 172.21.15.94:0/2663885759 shutdown_connections 2024-04-25T16:51:30.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f93635ba000 1 -- 172.21.15.94:0/2663885759 wait complete. 2024-04-25T16:51:30.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.647+0000 7f93635ba000 20 client.4818 trim_cache size 0 max 16384 2024-04-25T16:51:30.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93635ba000 1 Processor -- start 2024-04-25T16:51:30.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93635ba000 1 -- start start 2024-04-25T16:51:30.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93635ba000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c5c5280 0x56096cb589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93635ba000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb58ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93635ba000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9e0c20 0x56096cb4a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93635ba000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9354201e40 con 0x56096c9ccea0 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93635ba000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9354074260 con 0x56096c9e0c20 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93635ba000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9354201910 con 0x56096c5c5280 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93597f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb58ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93597f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb58ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46972/0 (socket says 172.21.15.94:46972) 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93597f8640 1 -- 172.21.15.94:0/1950553155 learned_addr learned my addr 172.21.15.94:0/1950553155 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93597f8640 1 -- 172.21.15.94:0/1950553155 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c5c5280 msgr2=0x56096cb589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93597f8640 1 --2- 172.21.15.94:0/1950553155 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c5c5280 0x56096cb589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93597f8640 1 -- 172.21.15.94:0/1950553155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9e0c20 msgr2=0x56096cb4a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93597f8640 1 --2- 172.21.15.94:0/1950553155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9e0c20 0x56096cb4a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93597f8640 1 -- 172.21.15.94:0/1950553155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56096c8fcc70 con 0x56096c9ccea0 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f93597f8640 1 --2- 172.21.15.94:0/1950553155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb58ef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f934405b2d0 tx=0x7f934408f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d29f071842d192 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f9336ffd640 1 -- 172.21.15.94:0/1950553155 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9344056f40 con 0x56096c9ccea0 2024-04-25T16:51:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f9336ffd640 1 -- 172.21.15.94:0/1950553155 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93440570e0 con 0x56096c9ccea0 2024-04-25T16:51:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.663+0000 7f9336ffd640 1 -- 172.21.15.94:0/1950553155 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934405a4a0 con 0x56096c9ccea0 2024-04-25T16:51:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 -- 172.21.15.94:0/1950553155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 msgr2=0x56096cb58ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 --2- 172.21.15.94:0/1950553155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb58ef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f934405b2d0 tx=0x7f934408f000 comp rx=0 tx=0).stop 2024-04-25T16:51:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 -- 172.21.15.94:0/1950553155 shutdown_connections 2024-04-25T16:51:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 --2- 172.21.15.94:0/1950553155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c9e0c20 0x56096cb4a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 --2- 172.21.15.94:0/1950553155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb58ef0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 --2- 172.21.15.94:0/1950553155 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c5c5280 0x56096cb589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 -- 172.21.15.94:0/1950553155 >> 172.21.15.94:0/1950553155 conn(0x56096c604ff0 msgr2=0x56096caf4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 -- 172.21.15.94:0/1950553155 shutdown_connections 2024-04-25T16:51:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 -- 172.21.15.94:0/1950553155 wait complete. 2024-04-25T16:51:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 -- start start 2024-04-25T16:51:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 Processor -- start 2024-04-25T16:51:30.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c5c5280 0x56096cb4e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb71310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c9e0c20 0x56096cb71850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56096c943130 con 0x56096c9ccea0 2024-04-25T16:51:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56096c8d97d0 con 0x56096c5c5280 2024-04-25T16:51:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56096c948450 con 0x56096c9e0c20 2024-04-25T16:51:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 10 client.0 _ll_register_callbacks cb 0x7f9344057c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93597f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb71310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93597f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb71310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46976/0 (socket says 172.21.15.94:46976) 2024-04-25T16:51:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93597f8640 1 -- 172.21.15.94:0/1831967485 learned_addr learned my addr 172.21.15.94:0/1831967485 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f9359ff9640 1 --2- 172.21.15.94:0/1831967485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c5c5280 0x56096cb4e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f935a7fa640 1 --2- 172.21.15.94:0/1831967485 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c9e0c20 0x56096cb71850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93597f8640 1 -- 172.21.15.94:0/1831967485 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c9e0c20 msgr2=0x56096cb71850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93597f8640 1 --2- 172.21.15.94:0/1831967485 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c9e0c20 0x56096cb71850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93597f8640 1 -- 172.21.15.94:0/1831967485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c5c5280 msgr2=0x56096cb4e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93597f8640 1 --2- 172.21.15.94:0/1831967485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c5c5280 0x56096cb4e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93597f8640 1 -- 172.21.15.94:0/1831967485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56096c9c6180 con 0x56096c9ccea0 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93597f8640 1 --2- 172.21.15.94:0/1831967485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb71310 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f934405b560 tx=0x7f934408d1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc3da51895842ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f9334ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f9334ff9640 1 -- 172.21.15.94:0/1831967485 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9344056f40 con 0x56096c9ccea0 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f9334ff9640 1 -- 172.21.15.94:0/1831967485 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f934407dde0 con 0x56096c9ccea0 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.667+0000 7f93635ba000 1 -- 172.21.15.94:0/1831967485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93440570e0 con 0x56096c9ccea0 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f9334ff9640 1 -- 172.21.15.94:0/1831967485 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f934408b020 con 0x56096c9ccea0 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 10 client.4824 fetch_fsmap learned FSMap version 9 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 -- 172.21.15.94:0/1831967485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x56096c5fd0c0 con 0x56096c9ccea0 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f9334ff9640 1 -- 172.21.15.94:0/1831967485 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93440570e0 con 0x56096c9ccea0 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f9334ff9640 1 -- 172.21.15.94:0/1831967485 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f9344070a20 con 0x56096c9ccea0 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 10 client.4824 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 10 client.4824 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 20 client.4824 populate_metadata read hostname 'smithi094' 2024-04-25T16:51:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 --2- 172.21.15.94:0/1831967485 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x56096c5d7830 0x56096cbb4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 4 client.4824 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 -- 172.21.15.94:0/1831967485 --> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] -- command(tid 0: {"prefix": "status"}) -- 0x56096c951a30 con 0x56096c5d7830 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f9359ff9640 1 --2- 172.21.15.94:0/1831967485 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x56096c5d7830 0x56096cbb4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f9359ff9640 1 --2- 172.21.15.94:0/1831967485 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x56096c5d7830 0x56096cbb4740 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdde5395a70dc1e7 server_cookie=6c753aacd6eb3481 in_seq=0 out_seq=0 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f9334ff9640 10 client.4824 ms_handle_connect on v2:172.21.15.138:6834/3652545132 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f9334ff9640 1 -- 172.21.15.94:0/1831967485 <== mds.0 v2:172.21.15.138:6834/3652545132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x56096c951a30 con 0x56096c5d7830 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f9334ff9640 10 client.4824 handle_command_reply: tid=0 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 client.4824 shutdown 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 -- 172.21.15.94:0/1831967485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 msgr2=0x56096cb71310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 --2- 172.21.15.94:0/1831967485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb71310 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f934405b560 tx=0x7f934408d1e0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 -- 172.21.15.94:0/1831967485 shutdown_connections 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 --2- 172.21.15.94:0/1831967485 >> [v2:172.21.15.138:6834/3652545132,v1:172.21.15.138:6835/3652545132] conn(0x56096c5d7830 0x56096cbb4740 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 --2- 172.21.15.94:0/1831967485 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56096c9e0c20 0x56096cb71850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 --2- 172.21.15.94:0/1831967485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56096c9ccea0 0x56096cb71310 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 --2- 172.21.15.94:0/1831967485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56096c5c5280 0x56096cb4e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f9334ff9640 0 client.4824 ms_handle_reset on v2:172.21.15.138:6834/3652545132 2024-04-25T16:51:30.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 -- 172.21.15.94:0/1831967485 >> 172.21.15.94:0/1831967485 conn(0x56096c604ff0 msgr2=0x56096cb579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:30.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 -- 172.21.15.94:0/1831967485 shutdown_connections 2024-04-25T16:51:30.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 1 -- 172.21.15.94:0/1831967485 wait complete. 2024-04-25T16:51:30.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f93635ba000 20 client.4824 trim_cache size 0 max 16384 2024-04-25T16:51:30.674 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:51:30.674 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "cc26689a-ed7a-405e-9abc-b7fa2c1906cf", 2024-04-25T16:51:30.674 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-04-25T16:51:30.674 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4297, 2024-04-25T16:51:30.674 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T16:51:30.674 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T16:51:30.675 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T16:51:30.675 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 35.593788592999999, 2024-04-25T16:51:30.675 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 9, 2024-04-25T16:51:30.675 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 24, 2024-04-25T16:51:30.675 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T16:51:30.675 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 43.223521613000003 2024-04-25T16:51:30.675 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:51:30.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f935c25d640 1 -- 172.21.15.94:0/2124813864 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f933806f430 msgr2=0x7f9338071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:30.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f935c25d640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f933806f430 0x7f9338071940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f93540cd2f0 tx=0x7f9350008040 comp rx=0 tx=0).stop 2024-04-25T16:51:30.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f935c25d640 1 -- 172.21.15.94:0/2124813864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93540b0420 msgr2=0x7f9354155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:30.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f935c25d640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93540b0420 0x7f9354155390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f934400b410 tx=0x7f934400b8d0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f935c25d640 1 -- 172.21.15.94:0/2124813864 shutdown_connections 2024-04-25T16:51:30.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f935c25d640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f935415ec30 0x7f93541493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f935c25d640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93540b0420 0x7f9354155390 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f935c25d640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f933806f430 0x7f9338071940 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f935c25d640 1 --2- 172.21.15.94:0/2124813864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f935406f2a0 0x7f935406dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:30.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f935c25d640 1 -- 172.21.15.94:0/2124813864 >> 172.21.15.94:0/2124813864 conn(0x7f93540b47f0 msgr2=0x7f93540b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:30.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f935c25d640 1 -- 172.21.15.94:0/2124813864 shutdown_connections 2024-04-25T16:51:30.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:30.671+0000 7f935c25d640 1 -- 172.21.15.94:0/2124813864 wait complete. 2024-04-25T16:51:30.689 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T16:51:30.698 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T16:51:30.698 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T16:51:30.699 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T16:51:30.699 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T16:51:30.732 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T16:51:30.733 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T16:51:30.733 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T16:51:30.733 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T16:51:30.733 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T16:51:30.733 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T16:51:30.733 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T16:51:30.733 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T16:51:30.733 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T16:51:30.733 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T16:51:30.733 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T16:51:30.734 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T16:51:30.734 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T16:51:30.734 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T16:51:30.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:30.734 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T16:51:30.740 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T16:51:30.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:30.741 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T16:51:30.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:30.794 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-04-25T16:51:31.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:31.018 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T16:51:31.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:31.025 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-04-25T16:51:31.086 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-04-25T16:51:31.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:51:31.087 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T16:51:31.088 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T16:51:31.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.251+0000 7f1ac707c640 1 Processor -- start 2024-04-25T16:51:31.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.251+0000 7f1ac707c640 1 -- start start 2024-04-25T16:51:31.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.251+0000 7f1ac707c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ac0160bc0 0x7f1ac00cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:31.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.251+0000 7f1ac707c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ac00b66f0 0x7f1ac0163e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:31.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.251+0000 7f1ac707c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ac0160fd0 con 0x7f1ac00cf7e0 2024-04-25T16:51:31.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.251+0000 7f1ac707c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1ac00d0100 con 0x7f1ac0160bc0 2024-04-25T16:51:31.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.251+0000 7f1ac707c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac00d0e80 con 0x7f1ac00b66f0 2024-04-25T16:51:31.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.251+0000 7f1ac5619640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ac00b66f0 0x7f1ac0163e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:31.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.251+0000 7f1ac4e18640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ac0160bc0 0x7f1ac00cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:31.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.251+0000 7f1ac4e18640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ac0160bc0 0x7f1ac00cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58136/0 (socket says 172.21.15.94:58136) 2024-04-25T16:51:31.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.251+0000 7f1ac4e18640 1 -- 172.21.15.94:0/2917215563 learned_addr learned my addr 172.21.15.94:0/2917215563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:31.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.251+0000 7f1ac5619640 1 --2- 172.21.15.94:0/2917215563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ac00b66f0 0x7f1ac0163e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:56636/0 (socket says 172.21.15.94:56636) 2024-04-25T16:51:31.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac5619640 1 -- 172.21.15.94:0/2917215563 >> v2:172.21.15.138:3300/0 conn(0x7f1ac0160bc0 msgr2=0x7f1ac00cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:31.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac5619640 1 --2- 172.21.15.94:0/2917215563 >> v2:172.21.15.138:3300/0 conn(0x7f1ac0160bc0 0x7f1ac00cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:31.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac5619640 1 -- 172.21.15.94:0/2917215563 >> v1:172.21.15.138:6789/0 conn(0x7f1ac00cf7e0 legacy=0x7f1ac0163730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:31.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac5619640 1 -- 172.21.15.94:0/2917215563 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac00d1160 con 0x7f1ac00b66f0 2024-04-25T16:51:31.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac4e18640 1 --2- 172.21.15.94:0/2917215563 >> v2:172.21.15.138:3300/0 conn(0x7f1ac0160bc0 0x7f1ac00cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:31.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac5619640 1 --2- 172.21.15.94:0/2917215563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ac00b66f0 0x7f1ac0163e40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00097c0 tx=0x7f1ab001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65c353b45e4cfea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:31.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1abf7fe640 1 -- 172.21.15.94:0/2917215563 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00b0070 con 0x7f1ac00b66f0 2024-04-25T16:51:31.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1abf7fe640 1 -- 172.21.15.94:0/2917215563 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab00ab600 con 0x7f1ac00b66f0 2024-04-25T16:51:31.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1abf7fe640 1 -- 172.21.15.94:0/2917215563 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00ab9d0 con 0x7f1ac00b66f0 2024-04-25T16:51:31.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac707c640 1 -- 172.21.15.94:0/2917215563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ac00b66f0 msgr2=0x7f1ac0163e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:31.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac707c640 1 --2- 172.21.15.94:0/2917215563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ac00b66f0 0x7f1ac0163e40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00097c0 tx=0x7f1ab001c410 comp rx=0 tx=0).stop 2024-04-25T16:51:31.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac707c640 1 -- 172.21.15.94:0/2917215563 shutdown_connections 2024-04-25T16:51:31.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac707c640 1 --2- 172.21.15.94:0/2917215563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ac00b66f0 0x7f1ac0163e40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:31.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac707c640 1 --2- 172.21.15.94:0/2917215563 >> v2:172.21.15.138:3300/0 conn(0x7f1ac0160bc0 0x7f1ac00cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:31.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac707c640 1 -- 172.21.15.94:0/2917215563 >> 172.21.15.94:0/2917215563 conn(0x7f1ac0071150 msgr2=0x7f1ac0071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:31.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac707c640 1 -- 172.21.15.94:0/2917215563 shutdown_connections 2024-04-25T16:51:31.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.255+0000 7f1ac707c640 1 -- 172.21.15.94:0/2917215563 wait complete. 2024-04-25T16:51:31.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac707c640 1 Processor -- start 2024-04-25T16:51:31.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac707c640 1 -- start start 2024-04-25T16:51:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac707c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ac00b66f0 0x7f1ac00b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac707c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ac00cf7e0 0x7f1ac00b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac707c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ac0160bc0 0x7f1ac0144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac707c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac0164380 con 0x7f1ac00b66f0 2024-04-25T16:51:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac707c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac00d1a90 con 0x7f1ac00cf7e0 2024-04-25T16:51:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac707c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac0164500 con 0x7f1ac0160bc0 2024-04-25T16:51:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac4e18640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ac00b66f0 0x7f1ac00b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac4e18640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ac00b66f0 0x7f1ac00b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46988/0 (socket says 172.21.15.94:46988) 2024-04-25T16:51:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac4e18640 1 -- 172.21.15.94:0/222786401 learned_addr learned my addr 172.21.15.94:0/222786401 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac5619640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ac0160bc0 0x7f1ac0144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1abffff640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ac00cf7e0 0x7f1ac00b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac4e18640 1 -- 172.21.15.94:0/222786401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ac0160bc0 msgr2=0x7f1ac0144e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac4e18640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ac0160bc0 0x7f1ac0144e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac4e18640 1 -- 172.21.15.94:0/222786401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ac00cf7e0 msgr2=0x7f1ac00b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac4e18640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ac00cf7e0 0x7f1ac00b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac4e18640 1 -- 172.21.15.94:0/222786401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac0158ed0 con 0x7f1ac00b66f0 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1abffff640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ac00cf7e0 0x7f1ac00b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac5619640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ac0160bc0 0x7f1ac0144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.259+0000 7f1ac4e18640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ac00b66f0 0x7f1ac00b2730 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800efc0 tx=0x7f1ab800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.263+0000 7f1abd7fa640 1 -- 172.21.15.94:0/222786401 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab800ce60 con 0x7f1ac00b66f0 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.263+0000 7f1abd7fa640 1 -- 172.21.15.94:0/222786401 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab8019070 con 0x7f1ac00b66f0 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.263+0000 7f1abd7fa640 1 -- 172.21.15.94:0/222786401 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80146a0 con 0x7f1ac00b66f0 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.263+0000 7f1ac707c640 1 -- 172.21.15.94:0/222786401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac01593d0 con 0x7f1ac00b66f0 2024-04-25T16:51:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.263+0000 7f1ac707c640 1 -- 172.21.15.94:0/222786401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac0159990 con 0x7f1ac00b66f0 2024-04-25T16:51:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.263+0000 7f1ac707c640 1 -- 172.21.15.94:0/222786401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac006a730 con 0x7f1ac00b66f0 2024-04-25T16:51:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.263+0000 7f1abd7fa640 1 -- 172.21.15.94:0/222786401 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1ab8012050 con 0x7f1ac00b66f0 2024-04-25T16:51:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.263+0000 7f1abd7fa640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1aac06f400 0x7f1aac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.263+0000 7f1abffff640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1aac06f400 0x7f1aac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.263+0000 7f1abd7fa640 1 -- 172.21.15.94:0/222786401 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1ab808fc80 con 0x7f1ac00b66f0 2024-04-25T16:51:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.263+0000 7f1abffff640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1aac06f400 0x7f1aac071910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1ac01fcb70 tx=0x7f1ab4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.267+0000 7f1abd7fa640 1 -- 172.21.15.94:0/222786401 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ab8023020 con 0x7f1ac00b66f0 2024-04-25T16:51:31.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.467+0000 7f1ac707c640 1 -- 172.21.15.94:0/222786401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1ac00b7580 con 0x7f1ac00b66f0 2024-04-25T16:51:31.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.467+0000 7f1abd7fa640 1 -- 172.21.15.94:0/222786401 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f1ab8010040 con 0x7f1ac00b66f0 2024-04-25T16:51:31.466 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T16:51:31.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.471+0000 7f1ac707c640 1 -- 172.21.15.94:0/222786401 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1aac06f400 msgr2=0x7f1aac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:31.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.471+0000 7f1ac707c640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1aac06f400 0x7f1aac071910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1ac01fcb70 tx=0x7f1ab4009040 comp rx=0 tx=0).stop 2024-04-25T16:51:31.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.471+0000 7f1ac707c640 1 -- 172.21.15.94:0/222786401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ac00b66f0 msgr2=0x7f1ac00b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:31.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.471+0000 7f1ac707c640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ac00b66f0 0x7f1ac00b2730 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800efc0 tx=0x7f1ab800c490 comp rx=0 tx=0).stop 2024-04-25T16:51:31.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.471+0000 7f1ac707c640 1 -- 172.21.15.94:0/222786401 shutdown_connections 2024-04-25T16:51:31.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.471+0000 7f1ac707c640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1aac06f400 0x7f1aac071910 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:31.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.471+0000 7f1ac707c640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ac0160bc0 0x7f1ac0144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:31.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.471+0000 7f1ac707c640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ac00cf7e0 0x7f1ac00b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:31.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.471+0000 7f1ac707c640 1 --2- 172.21.15.94:0/222786401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ac00b66f0 0x7f1ac00b2730 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:31.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.471+0000 7f1ac707c640 1 -- 172.21.15.94:0/222786401 >> 172.21.15.94:0/222786401 conn(0x7f1ac0071150 msgr2=0x7f1ac015ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:31.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.471+0000 7f1ac707c640 1 -- 172.21.15.94:0/222786401 shutdown_connections 2024-04-25T16:51:31.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31.471+0000 7f1ac707c640 1 -- 172.21.15.94:0/222786401 wait complete. 2024-04-25T16:51:31.487 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T16:51:31.487 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T16:51:31.487 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-04-25T16:51:31.487 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T16:51:31.487 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T16:51:31.487 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T16:51:31.487 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T16:51:31.487 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T16:51:31.488 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T16:51:31.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:31.488 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-04-25T16:51:31.493 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:51:31.493 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T16:51:31.494 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:31.495 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T16:51:31.495 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42434sec preferred_lft 42434sec 2024-04-25T16:51:31.495 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-04-25T16:51:31.495 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:31.495 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T16:51:31.495 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:f8:37:e2:6a brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:31.495 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T16:51:31.495 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:31.495 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T16:51:31.496 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:31.496 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:31.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-04-25T16:51:31.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-04-25T16:51:31.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-04-25T16:51:31.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T16:51:31.496 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:31.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:31.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:31.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:31.615 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T16:51:31.663 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-04-25T16:51:31.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:31.665 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-04-25T16:51:31.716 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-04-25T16:51:31.716 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T16:51:31.716 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T16:51:31.716 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T16:51:31.716 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T16:51:31.716 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T16:51:31.716 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T16:51:31.716 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:31.716 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:31.716 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:31.716 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T16:51:31.717 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:31.717 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:31.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T16:51:31.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T16:51:31.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T16:51:31.717 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:31.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:31.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:31.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:31.838 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T16:51:31.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:31.885 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-04-25T16:51:31.933 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:31.933 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:31.933 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:31.933 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T16:51:31.933 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:32.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:32.051 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T16:51:32.051 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:32.052 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:32.052 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T16:51:32.052 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T16:51:32.052 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T16:51:32.052 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T16:51:32.052 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T16:51:32.052 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:32.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:32.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:32.141 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:32.141 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:32.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-04-25T16:51:32.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T16:51:32.141 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:32.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:32.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:32.196 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T16:51:32.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:51:32.196 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T16:51:32.260 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T16:51:32.260 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:51:32.260 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T16:51:32.307 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:51:32.365 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:32.413 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:51:32.413 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:51:32.413 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:51:32.413 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:51:32.413 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:51:32.413 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:51:32.413 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:51:32.413 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:51:32.413 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-04-25T16:51:32.413 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:51:32.413 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:51:32.413 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:51:32.414 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:32.414 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:51:32.414 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:51:32.414 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:51:32.414 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:51:32.414 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-04-25T16:51:32.414 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:32.414 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:51:32.414 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:32.414 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-04-25T16:51:32.414 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:51:32.414 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:51:32.414 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:51:32.415 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:51:32.415 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:51:32.415 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:51:32.415 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:51:32.415 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:32.415 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:32.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:32.416 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:51:32.463 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T16:51:32.464 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T16:51:32.509 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:51:32.522 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:51:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:51:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:51:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:51:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-04-25T16:51:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:51:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-04-25T16:51:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:51:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:51:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:51:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:51:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:51:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:51:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:51:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:32.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:32.572 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:51:32.611 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-04-25T16:51:32.611+0000 7f2344529580 -1 init, newargv = 0x7f2334005ec0 newargc=15 2024-04-25T16:51:32.611 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[35597]: starting ceph client 2024-04-25T16:51:32.619 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[35597]: starting fuse 2024-04-25T16:51:33.715 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:51:33.731 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:33.779 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:51:33.779 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:51:33.779 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:51:33.779 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:51:33.779 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:51:33.779 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:51:33.779 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:51:33.779 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:51:33.779 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-04-25T16:51:33.779 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:51:33.779 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:51:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:51:33.781 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:51:33.781 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:51:33.781 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:51:33.781 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:33.781 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:33.781 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T16:51:33.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:33.782 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:51:33.828 INFO:teuthology.orchestra.run.smithi094.stdout:45 2024-04-25T16:51:33.828 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-04-25T16:51:33.829 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-04-25T16:51:33.830 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-04-25T16:51:33.830 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:51:33.830 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-04-25T16:51:33.830 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T16:51:33.830 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T16:51:33.830 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T16:51:33.830 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-04-25T16:51:33.830 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-04-25T16:51:33.830 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-04-25T16:51:33.830 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T16:51:33.830 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:51:33.830 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-04-25T16:51:33.830 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-04-25T16:51:33.950 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.35597.asok 2024-04-25T16:51:33.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:33.962 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35597.asok 2024-04-25T16:51:33.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:33.962 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35597.asok status 2024-04-25T16:51:34.146 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:51:34.146 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-04-25T16:51:34.146 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T16:51:34.146 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T16:51:34.146 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-04-25T16:51:34.146 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-04-25T16:51:34.146 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T16:51:34.146 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "35597", 2024-04-25T16:51:34.146 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-04-25T16:51:34.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4830, 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 4830 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3387369724 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:51:34.147 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-04-25T16:51:34.148 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-04-25T16:51:34.148 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-04-25T16:51:34.148 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3387369724 2024-04-25T16:51:34.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:51:34.148 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.4830 192.168.0.1:0/3387369724", 2024-04-25T16:51:34.148 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/3387369724", 2024-04-25T16:51:34.148 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-04-25T16:51:34.148 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 9, 2024-04-25T16:51:34.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 24, 2024-04-25T16:51:34.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-04-25T16:51:34.148 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-04-25T16:51:34.148 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-04-25T16:51:34.148 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:51:34.162 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T16:51:34.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11e59ef640 1 Processor -- start 2024-04-25T16:51:34.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11e59ef640 1 -- start start 2024-04-25T16:51:34.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11e59ef640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11e00d0af0 0x7f11e00d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:34.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11e59ef640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f11e015f550 0x7f11e015f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:34.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11e59ef640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11e015e400 con 0x7f11e006d610 2024-04-25T16:51:34.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11e59ef640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f11e00ce360 con 0x7f11e015f550 2024-04-25T16:51:34.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11e59ef640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f11e015f010 con 0x7f11e00d0af0 2024-04-25T16:51:34.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11de7fc640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f11e006d610 0x7f11e006da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50436/0 (socket says 172.21.15.94:50436) 2024-04-25T16:51:34.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11de7fc640 1 -- 172.21.15.94:0/1553896004 learned_addr learned my addr 172.21.15.94:0/1553896004 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11df7fe640 1 --2- 172.21.15.94:0/1553896004 >> v2:172.21.15.94:3300/0 conn(0x7f11e015f550 0x7f11e015f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11ddffb640 1 -- 172.21.15.94:0/1553896004 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708495734 0 0) 0x7f11e015e400 con 0x7f11e006d610 2024-04-25T16:51:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11ddffb640 1 -- 172.21.15.94:0/1553896004 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11c8003530 con 0x7f11e006d610 2024-04-25T16:51:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11deffd640 1 --2- 172.21.15.94:0/1553896004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11e00d0af0 0x7f11e00d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11df7fe640 1 -- 172.21.15.94:0/1553896004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11e00d0af0 msgr2=0x7f11e00d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11df7fe640 1 --2- 172.21.15.94:0/1553896004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11e00d0af0 0x7f11e00d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11df7fe640 1 -- 172.21.15.94:0/1553896004 >> v1:172.21.15.94:6789/0 conn(0x7f11e006d610 legacy=0x7f11e006da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.319+0000 7f11df7fe640 1 -- 172.21.15.94:0/1553896004 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e015e6e0 con 0x7f11e015f550 2024-04-25T16:51:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11df7fe640 1 --2- 172.21.15.94:0/1553896004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e015f550 0x7f11e015f960 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009d90 tx=0x7f11d801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297b049270e6f7b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11ddffb640 1 -- 172.21.15.94:0/1553896004 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d80b2070 con 0x7f11e015f550 2024-04-25T16:51:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11ddffb640 1 -- 172.21.15.94:0/1553896004 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11d80a5920 con 0x7f11e015f550 2024-04-25T16:51:34.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11ddffb640 1 -- 172.21.15.94:0/1553896004 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d80a5cd0 con 0x7f11e015f550 2024-04-25T16:51:34.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11e59ef640 1 -- 172.21.15.94:0/1553896004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e015f550 msgr2=0x7f11e015f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11e59ef640 1 --2- 172.21.15.94:0/1553896004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e015f550 0x7f11e015f960 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009d90 tx=0x7f11d801c970 comp rx=0 tx=0).stop 2024-04-25T16:51:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11e59ef640 1 -- 172.21.15.94:0/1553896004 shutdown_connections 2024-04-25T16:51:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11e59ef640 1 --2- 172.21.15.94:0/1553896004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e015f550 0x7f11e015f960 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11e59ef640 1 --2- 172.21.15.94:0/1553896004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11e00d0af0 0x7f11e00d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11e59ef640 1 -- 172.21.15.94:0/1553896004 >> 172.21.15.94:0/1553896004 conn(0x7f11e0072c40 msgr2=0x7f11e0073050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11e59ef640 1 -- 172.21.15.94:0/1553896004 shutdown_connections 2024-04-25T16:51:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11e59ef640 1 -- 172.21.15.94:0/1553896004 wait complete. 2024-04-25T16:51:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11e59ef640 1 Processor -- start 2024-04-25T16:51:34.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.323+0000 7f11e59ef640 1 -- start start 2024-04-25T16:51:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11e59ef640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11e006d610 0x7f11e0167130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11e59ef640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e00d0af0 0x7f11e0143750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11e59ef640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e015f550 0x7f11e0143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11e59ef640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11e016a990 con 0x7f11e015f550 2024-04-25T16:51:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11e59ef640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11e016a810 con 0x7f11e00d0af0 2024-04-25T16:51:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11e59ef640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f11e016ab10 con 0x7f11e006d610 2024-04-25T16:51:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e015f550 0x7f11e0143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e015f550 0x7f11e0143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47000/0 (socket says 172.21.15.94:47000) 2024-04-25T16:51:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11df7fe640 1 -- 172.21.15.94:0/2060610937 learned_addr learned my addr 172.21.15.94:0/2060610937 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11deffd640 1 --2- 172.21.15.94:0/2060610937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11e006d610 0x7f11e0167130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11df7fe640 1 -- 172.21.15.94:0/2060610937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11e006d610 msgr2=0x7f11e0167130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11df7fe640 1 --2- 172.21.15.94:0/2060610937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11e006d610 0x7f11e0167130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11df7fe640 1 -- 172.21.15.94:0/2060610937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e00d0af0 msgr2=0x7f11e0143750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11df7fe640 1 --2- 172.21.15.94:0/2060610937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e00d0af0 0x7f11e0143750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11df7fe640 1 -- 172.21.15.94:0/2060610937 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e015d990 con 0x7f11e015f550 2024-04-25T16:51:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11df7fe640 1 --2- 172.21.15.94:0/2060610937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e015f550 0x7f11e0143c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f11d80b0040 tx=0x7f11d8019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11c7fff640 1 -- 172.21.15.94:0/2060610937 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d80b2050 con 0x7f11e015f550 2024-04-25T16:51:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11c7fff640 1 -- 172.21.15.94:0/2060610937 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11d80be070 con 0x7f11e015f550 2024-04-25T16:51:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11c7fff640 1 -- 172.21.15.94:0/2060610937 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d80ada50 con 0x7f11e015f550 2024-04-25T16:51:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11e59ef640 1 -- 172.21.15.94:0/2060610937 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e0168460 con 0x7f11e015f550 2024-04-25T16:51:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11e59ef640 1 -- 172.21.15.94:0/2060610937 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e0168180 con 0x7f11e015f550 2024-04-25T16:51:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.327+0000 7f11e59ef640 1 -- 172.21.15.94:0/2060610937 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e0076830 con 0x7f11e015f550 2024-04-25T16:51:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.331+0000 7f11c7fff640 1 -- 172.21.15.94:0/2060610937 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f11d80a57a0 con 0x7f11e015f550 2024-04-25T16:51:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.331+0000 7f11c7fff640 1 --2- 172.21.15.94:0/2060610937 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f11cc06f400 0x7f11cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.331+0000 7f11c7fff640 1 -- 172.21.15.94:0/2060610937 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7f11d812b100 con 0x7f11e015f550 2024-04-25T16:51:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.331+0000 7f11deffd640 1 --2- 172.21.15.94:0/2060610937 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f11cc06f400 0x7f11cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.331+0000 7f11deffd640 1 --2- 172.21.15.94:0/2060610937 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f11cc06f400 0x7f11cc071910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f11d00096f0 tx=0x7f11d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.335+0000 7f11c7fff640 1 -- 172.21.15.94:0/2060610937 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f11e015d990 con 0x7f11e015f550 2024-04-25T16:51:34.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.531+0000 7f11e59ef640 1 -- 172.21.15.94:0/2060610937 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f11e00b0b40 con 0x7f11e015f550 2024-04-25T16:51:34.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.531+0000 7f11c7fff640 1 -- 172.21.15.94:0/2060610937 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f11d8016560 con 0x7f11e015f550 2024-04-25T16:51:34.530 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T16:51:34.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.535+0000 7f11e59ef640 1 -- 172.21.15.94:0/2060610937 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f11cc06f400 msgr2=0x7f11cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:34.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.535+0000 7f11e59ef640 1 --2- 172.21.15.94:0/2060610937 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f11cc06f400 0x7f11cc071910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f11d00096f0 tx=0x7f11d0008040 comp rx=0 tx=0).stop 2024-04-25T16:51:34.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.535+0000 7f11e59ef640 1 -- 172.21.15.94:0/2060610937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e015f550 msgr2=0x7f11e0143c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:34.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.535+0000 7f11e59ef640 1 --2- 172.21.15.94:0/2060610937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e015f550 0x7f11e0143c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f11d80b0040 tx=0x7f11d8019710 comp rx=0 tx=0).stop 2024-04-25T16:51:34.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.535+0000 7f11e59ef640 1 -- 172.21.15.94:0/2060610937 shutdown_connections 2024-04-25T16:51:34.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.535+0000 7f11e59ef640 1 --2- 172.21.15.94:0/2060610937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e015f550 0x7f11e0143c90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:34.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.535+0000 7f11e59ef640 1 --2- 172.21.15.94:0/2060610937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e00d0af0 0x7f11e0143750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:34.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.535+0000 7f11e59ef640 1 --2- 172.21.15.94:0/2060610937 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f11cc06f400 0x7f11cc071910 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:34.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.535+0000 7f11e59ef640 1 --2- 172.21.15.94:0/2060610937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11e006d610 0x7f11e0167130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:34.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.535+0000 7f11e59ef640 1 -- 172.21.15.94:0/2060610937 >> 172.21.15.94:0/2060610937 conn(0x7f11e0072c40 msgr2=0x7f11e00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:34.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.535+0000 7f11e59ef640 1 -- 172.21.15.94:0/2060610937 shutdown_connections 2024-04-25T16:51:34.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34.535+0000 7f11e59ef640 1 -- 172.21.15.94:0/2060610937 wait complete. 2024-04-25T16:51:34.551 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T16:51:34.552 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T16:51:34.552 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-04-25T16:51:34.552 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T16:51:34.552 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T16:51:34.552 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T16:51:34.552 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T16:51:34.552 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T16:51:34.552 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T16:51:34.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:34.552 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-04-25T16:51:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:51:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:51:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:51:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-25T16:51:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:34.560 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-04-25T16:51:34.560 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-04-25T16:51:34.560 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:34.560 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T16:51:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T16:51:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42431sec preferred_lft 42431sec 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:f8:37:e2:6a brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T16:51:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T16:51:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:34.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:34.562 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T16:51:34.616 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-04-25T16:51:34.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:34.617 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-04-25T16:51:34.669 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-04-25T16:51:34.669 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T16:51:34.669 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T16:51:34.669 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T16:51:34.669 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T16:51:34.669 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T16:51:34.669 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T16:51:34.669 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:34.669 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:34.669 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:34.669 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T16:51:34.670 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:34.670 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:34.670 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T16:51:34.670 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T16:51:34.670 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T16:51:34.670 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:34.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:34.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:34.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:34.793 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T16:51:34.798 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T16:51:34.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:34.799 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-04-25T16:51:34.844 INFO:teuthology.orchestra.run.smithi094.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T16:51:34.844 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:34.844 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:34.844 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:51:34.844 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T16:51:34.844 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:34.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:34.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:34.958 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T16:51:34.969 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:51:34.969 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:51:34.969 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:51:34.969 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:34.969 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-25T16:51:34.969 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:34.970 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T16:51:34.970 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 8e:28:d4:dc:09:57 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T16:51:34.970 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T16:51:34.970 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:34.970 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::8c28:d4ff:fedc:957/64 scope link 2024-04-25T16:51:34.970 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:34.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:34.971 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T16:51:35.021 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:51:35.021 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:51:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:51:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-25T16:51:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:35.022 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T16:51:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 8e:28:d4:dc:09:57 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T16:51:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T16:51:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::8c28:d4ff:fedc:957/64 scope link 2024-04-25T16:51:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:35.023 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T16:51:35.024 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:35.024 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:35.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T16:51:35.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T16:51:35.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T16:51:35.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T16:51:35.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T16:51:35.024 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:35.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:35.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:35.165 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:35.165 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:51:35.165 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.1 up 2024-04-25T16:51:35.165 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T16:51:35.165 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:51:35.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:35.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:35.274 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T16:51:35.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:51:35.274 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T16:51:35.320 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T16:51:35.321 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:51:35.321 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T16:51:35.368 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:51:35.425 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:35.476 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:51:35.476 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:51:35.476 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:51:35.477 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:51:35.477 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:51:35.477 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:51:35.477 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:51:35.477 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:51:35.477 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-04-25T16:51:35.477 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:51:35.477 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:51:35.477 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:51:35.477 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:35.477 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:51:35.477 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:51:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:51:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:51:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-04-25T16:51:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:51:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-04-25T16:51:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:51:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:51:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:51:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:51:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:51:35.478 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:51:35.479 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:51:35.479 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:35.479 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:35.479 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T16:51:35.479 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:51:35.479 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:51:35.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:35.480 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:51:35.528 INFO:teuthology.orchestra.run.smithi094.stdout:45 2024-04-25T16:51:35.528 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T16:51:35.528 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T16:51:35.573 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:51:35.587 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:35.637 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:51:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:51:35.639 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:35.639 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:35.639 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T16:51:35.639 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:51:35.639 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:51:35.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:35.639 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:51:35.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi094.stderr:2024-04-25T16:51:35.679+0000 7f9e3b92b580 -1 init, newargv = 0x7f9e2c01b580 newargc=15 2024-04-25T16:51:35.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi094.stderr:ceph-fuse[35743]: starting ceph client 2024-04-25T16:51:35.683 INFO:teuthology.orchestra.run.smithi094.stdout:45 2024-04-25T16:51:35.686 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi094.stderr:ceph-fuse[35743]: starting fuse 2024-04-25T16:51:36.784 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:51:36.801 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:36.850 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:51:36.850 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:51:36.850 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:51:36.850 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:51:36.850 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:51:36.850 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:51:36.850 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:51:36.850 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-04-25T16:51:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:51:36.852 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:51:36.852 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:51:36.852 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:51:36.852 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:51:36.852 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:51:36.852 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:51:36.852 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:36.852 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:51:36.852 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T16:51:36.852 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:51:36.852 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:51:36.852 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.2 2024-04-25T16:51:36.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:36.853 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:51:36.897 INFO:teuthology.orchestra.run.smithi094.stdout:45 2024-04-25T16:51:36.897 INFO:teuthology.orchestra.run.smithi094.stdout:46 2024-04-25T16:51:36.897 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T16:51:36.897 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:51:36.898 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-04-25T16:51:36.899 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T16:51:36.899 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T16:51:36.899 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T16:51:36.899 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-04-25T16:51:36.899 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-04-25T16:51:36.899 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-04-25T16:51:36.899 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T16:51:36.899 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:51:36.899 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.2")) 2024-04-25T16:51:36.899 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-04-25T16:51:37.018 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.2.35743.asok 2024-04-25T16:51:37.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:37.030 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35743.asok 2024-04-25T16:51:37.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:37.030 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35743.asok status 2024-04-25T16:51:37.210 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:51:37.210 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-04-25T16:51:37.210 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T16:51:37.210 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T16:51:37.210 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "2", 2024-04-25T16:51:37.210 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-04-25T16:51:37.210 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T16:51:37.210 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "35743", 2024-04-25T16:51:37.210 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-04-25T16:51:37.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:51:37.210 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4845, 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 4845 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.2:0", 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3933291848 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:51:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-04-25T16:51:37.212 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-04-25T16:51:37.212 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.2:0", 2024-04-25T16:51:37.212 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3933291848 2024-04-25T16:51:37.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:51:37.212 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.4845 192.168.0.2:0/3933291848", 2024-04-25T16:51:37.212 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.2:0/3933291848", 2024-04-25T16:51:37.212 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-04-25T16:51:37.212 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 9, 2024-04-25T16:51:37.212 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 26, 2024-04-25T16:51:37.212 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-04-25T16:51:37.212 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-04-25T16:51:37.212 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-04-25T16:51:37.212 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:51:37.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T16:51:37.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd74d1640 1 Processor -- start 2024-04-25T16:51:37.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd74d1640 1 -- start start 2024-04-25T16:51:37.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd74d1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddd0070080 0x7fddd015e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:37.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd74d1640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fddd015e7c0 0x7fddd006c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:37.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd74d1640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd006fc60 con 0x7fddd015ebd0 2024-04-25T16:51:37.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd74d1640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fddd0158550 con 0x7fddd015e7c0 2024-04-25T16:51:37.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd74d1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fddd006f7d0 con 0x7fddd0070080 2024-04-25T16:51:37.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd4a6c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fddd015e7c0 0x7fddd006c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:37.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd526d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddd0070080 0x7fddd015e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:37.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd526d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddd0070080 0x7fddd015e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:56670/0 (socket says 172.21.15.94:56670) 2024-04-25T16:51:37.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd5a6e640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fddd015ebd0 0x7fddd006cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:37126/0 (socket says 172.21.15.94:37126) 2024-04-25T16:51:37.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd526d640 1 -- 172.21.15.94:0/3733587110 learned_addr learned my addr 172.21.15.94:0/3733587110 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:37.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd526d640 1 -- 172.21.15.94:0/3733587110 >> v2:172.21.15.138:3300/0 conn(0x7fddd015e7c0 msgr2=0x7fddd006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:37.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd526d640 1 --2- 172.21.15.94:0/3733587110 >> v2:172.21.15.138:3300/0 conn(0x7fddd015e7c0 0x7fddd006c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:37.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd526d640 1 -- 172.21.15.94:0/3733587110 >> v1:172.21.15.138:6789/0 conn(0x7fddd015ebd0 legacy=0x7fddd006cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:37.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd526d640 1 -- 172.21.15.94:0/3733587110 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd00aef70 con 0x7fddd0070080 2024-04-25T16:51:37.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.379+0000 7fddd4a6c640 1 --2- 172.21.15.94:0/3733587110 >> v2:172.21.15.138:3300/0 conn(0x7fddd015e7c0 0x7fddd006c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:37.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.383+0000 7fddd526d640 1 --2- 172.21.15.94:0/3733587110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddd0070080 0x7fddd015e280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009870 tx=0x7fddc001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0e13979c187826d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:37.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.383+0000 7fddcffff640 1 -- 172.21.15.94:0/3733587110 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc00ab640 con 0x7fddd0070080 2024-04-25T16:51:37.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.383+0000 7fddcffff640 1 -- 172.21.15.94:0/3733587110 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc00b0070 con 0x7fddd0070080 2024-04-25T16:51:37.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.383+0000 7fddcffff640 1 -- 172.21.15.94:0/3733587110 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc00aba00 con 0x7fddd0070080 2024-04-25T16:51:37.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.383+0000 7fddd74d1640 1 -- 172.21.15.94:0/3733587110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddd0070080 msgr2=0x7fddd015e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:37.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.383+0000 7fddd74d1640 1 --2- 172.21.15.94:0/3733587110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddd0070080 0x7fddd015e280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009870 tx=0x7fddc001c530 comp rx=0 tx=0).stop 2024-04-25T16:51:37.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.383+0000 7fddd74d1640 1 -- 172.21.15.94:0/3733587110 shutdown_connections 2024-04-25T16:51:37.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.383+0000 7fddd74d1640 1 --2- 172.21.15.94:0/3733587110 >> v2:172.21.15.138:3300/0 conn(0x7fddd015e7c0 0x7fddd006c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:37.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.383+0000 7fddd74d1640 1 --2- 172.21.15.94:0/3733587110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddd0070080 0x7fddd015e280 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:37.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.383+0000 7fddd74d1640 1 -- 172.21.15.94:0/3733587110 >> 172.21.15.94:0/3733587110 conn(0x7fddd00b47f0 msgr2=0x7fddd00b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:37.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.383+0000 7fddd74d1640 1 -- 172.21.15.94:0/3733587110 shutdown_connections 2024-04-25T16:51:37.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.383+0000 7fddd74d1640 1 -- 172.21.15.94:0/3733587110 wait complete. 2024-04-25T16:51:37.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd74d1640 1 Processor -- start 2024-04-25T16:51:37.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd74d1640 1 -- start start 2024-04-25T16:51:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd74d1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddd0070080 0x7fddd0147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd74d1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddd015e7c0 0x7fddd00d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd74d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd015ebd0 0x7fddd00d0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd74d1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fddd0070490 con 0x7fddd015ebd0 2024-04-25T16:51:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd74d1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fddd00ae6c0 con 0x7fddd015e7c0 2024-04-25T16:51:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd74d1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fddd016ae90 con 0x7fddd0070080 2024-04-25T16:51:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd5a6e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd015ebd0 0x7fddd00d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd5a6e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd015ebd0 0x7fddd00d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47016/0 (socket says 172.21.15.94:47016) 2024-04-25T16:51:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd5a6e640 1 -- 172.21.15.94:0/2469456708 learned_addr learned my addr 172.21.15.94:0/2469456708 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd526d640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddd0070080 0x7fddd0147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd4a6c640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddd015e7c0 0x7fddd00d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd5a6e640 1 -- 172.21.15.94:0/2469456708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddd0070080 msgr2=0x7fddd0147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd5a6e640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddd0070080 0x7fddd0147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd5a6e640 1 -- 172.21.15.94:0/2469456708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddd015e7c0 msgr2=0x7fddd00d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd5a6e640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddd015e7c0 0x7fddd00d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.387+0000 7fddd5a6e640 1 -- 172.21.15.94:0/2469456708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd0147690 con 0x7fddd015ebd0 2024-04-25T16:51:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.391+0000 7fddd5a6e640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd015ebd0 0x7fddd00d0810 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fddc800c760 tx=0x7fddc800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.391+0000 7fddcdffb640 1 -- 172.21.15.94:0/2469456708 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8018070 con 0x7fddd015ebd0 2024-04-25T16:51:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.391+0000 7fddcdffb640 1 -- 172.21.15.94:0/2469456708 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc8004d70 con 0x7fddd015ebd0 2024-04-25T16:51:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.391+0000 7fddd74d1640 1 -- 172.21.15.94:0/2469456708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd014e640 con 0x7fddd015ebd0 2024-04-25T16:51:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.391+0000 7fddcdffb640 1 -- 172.21.15.94:0/2469456708 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc800a880 con 0x7fddd015ebd0 2024-04-25T16:51:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.391+0000 7fddd74d1640 1 -- 172.21.15.94:0/2469456708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd014dda0 con 0x7fddd015ebd0 2024-04-25T16:51:37.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.391+0000 7fddd74d1640 1 -- 172.21.15.94:0/2469456708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd0076830 con 0x7fddd015ebd0 2024-04-25T16:51:37.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.391+0000 7fddcdffb640 1 -- 172.21.15.94:0/2469456708 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fddc801b070 con 0x7fddd015ebd0 2024-04-25T16:51:37.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.391+0000 7fddcdffb640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fddbc06f430 0x7fddbc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:37.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.391+0000 7fddcdffb640 1 -- 172.21.15.94:0/2469456708 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fddc808eb90 con 0x7fddd015ebd0 2024-04-25T16:51:37.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.391+0000 7fddd526d640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fddbc06f430 0x7fddbc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:37.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.391+0000 7fddd526d640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fddbc06f430 0x7fddbc071940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fddc00af000 tx=0x7fddc00a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:37.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.395+0000 7fddcdffb640 1 -- 172.21.15.94:0/2469456708 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fddc805b680 con 0x7fddd015ebd0 2024-04-25T16:51:37.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.587+0000 7fddd74d1640 1 -- 172.21.15.94:0/2469456708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fddd0153fb0 con 0x7fddd015ebd0 2024-04-25T16:51:37.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.587+0000 7fddcdffb640 1 -- 172.21.15.94:0/2469456708 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fddc805af80 con 0x7fddd015ebd0 2024-04-25T16:51:37.587 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T16:51:37.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.591+0000 7fddd74d1640 1 -- 172.21.15.94:0/2469456708 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fddbc06f430 msgr2=0x7fddbc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:37.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.591+0000 7fddd74d1640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fddbc06f430 0x7fddbc071940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fddc00af000 tx=0x7fddc00a7000 comp rx=0 tx=0).stop 2024-04-25T16:51:37.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.591+0000 7fddd74d1640 1 -- 172.21.15.94:0/2469456708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd015ebd0 msgr2=0x7fddd00d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:37.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.591+0000 7fddd74d1640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd015ebd0 0x7fddd00d0810 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fddc800c760 tx=0x7fddc800cb60 comp rx=0 tx=0).stop 2024-04-25T16:51:37.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.591+0000 7fddd74d1640 1 -- 172.21.15.94:0/2469456708 shutdown_connections 2024-04-25T16:51:37.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.591+0000 7fddd74d1640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd015ebd0 0x7fddd00d0810 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:37.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.591+0000 7fddd74d1640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddd015e7c0 0x7fddd00d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:37.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.591+0000 7fddd74d1640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fddbc06f430 0x7fddbc071940 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:37.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.591+0000 7fddd74d1640 1 --2- 172.21.15.94:0/2469456708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddd0070080 0x7fddd0147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:37.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.591+0000 7fddd74d1640 1 -- 172.21.15.94:0/2469456708 >> 172.21.15.94:0/2469456708 conn(0x7fddd00b47f0 msgr2=0x7fddd00b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:37.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.591+0000 7fddd74d1640 1 -- 172.21.15.94:0/2469456708 shutdown_connections 2024-04-25T16:51:37.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:37.591+0000 7fddd74d1640 1 -- 172.21.15.94:0/2469456708 wait complete. 2024-04-25T16:51:37.608 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T16:51:37.608 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T16:51:37.608 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-04-25T16:51:37.608 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T16:51:37.608 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T16:51:37.608 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T16:51:37.608 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T16:51:37.608 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T16:51:37.608 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T16:51:37.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:37.609 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-04-25T16:51:37.615 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:51:37.616 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:51:37.616 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:51:37.616 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:37.616 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-04-25T16:51:37.616 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:37.616 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:37.616 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:37.616 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-04-25T16:51:37.616 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:37.616 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:37.616 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-04-25T16:51:37.616 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:37.616 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:37.617 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T16:51:37.617 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:37.617 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T16:51:37.617 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42435sec preferred_lft 42435sec 2024-04-25T16:51:37.617 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-04-25T16:51:37.617 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:37.617 INFO:teuthology.orchestra.run.smithi138.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T16:51:37.617 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 02:42:f0:29:8e:1a brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:37.617 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T16:51:37.617 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:37.617 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T16:51:37.617 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:37.618 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:51:37.618 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2024-04-25T16:51:37.618 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2024-04-25T16:51:37.618 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2024-04-25T16:51:37.618 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T16:51:37.618 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:51:37.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:37.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:37.743 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T16:51:37.752 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-04-25T16:51:37.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:37.754 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-04-25T16:51:37.835 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-04-25T16:51:37.835 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T16:51:37.835 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T16:51:37.835 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T16:51:37.835 INFO:teuthology.orchestra.run.smithi138.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T16:51:37.835 INFO:teuthology.orchestra.run.smithi138.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T16:51:37.835 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T16:51:37.835 INFO:teuthology.orchestra.run.smithi138.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:37.835 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:37.836 INFO:teuthology.orchestra.run.smithi138.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:37.836 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T16:51:37.836 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:37.836 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:51:37.836 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T16:51:37.836 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T16:51:37.836 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T16:51:37.836 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:51:37.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:37.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:37.954 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T16:51:38.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:38.006 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-04-25T16:51:38.054 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:38.054 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:51:38.054 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:38.054 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T16:51:38.054 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:51:38.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:38.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:38.170 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T16:51:38.170 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:38.170 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:51:38.170 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T16:51:38.170 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T16:51:38.171 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T16:51:38.171 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T16:51:38.171 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T16:51:38.171 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:51:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:38.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:38.279 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:38.279 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:51:38.279 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-04-25T16:51:38.279 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T16:51:38.279 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:51:38.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:38.382 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T16:51:38.382 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:51:38.382 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T16:51:38.438 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T16:51:38.438 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:51:38.438 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T16:51:38.486 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T16:51:38.543 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:38.595 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T16:51:38.595 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T16:51:38.595 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T16:51:38.595 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T16:51:38.595 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T16:51:38.595 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T16:51:38.595 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T16:51:38.595 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T16:51:38.595 INFO:teuthology.orchestra.run.smithi138.stdout:/run/lock 2024-04-25T16:51:38.595 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T16:51:38.595 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T16:51:38.595 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T16:51:38.595 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/fs/nfsd 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/run/rpc_pipefs 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T16:51:38.596 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T16:51:38.597 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:38.597 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:38.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:38.598 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T16:51:38.650 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T16:51:38.650 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T16:51:38.695 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T16:51:38.709 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:38.758 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T16:51:38.758 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T16:51:38.758 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/run/lock 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T16:51:38.759 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/fs/nfsd 2024-04-25T16:51:38.760 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:38.760 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T16:51:38.760 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:38.760 INFO:teuthology.orchestra.run.smithi138.stdout:/run/rpc_pipefs 2024-04-25T16:51:38.760 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T16:51:38.760 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T16:51:38.760 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T16:51:38.760 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T16:51:38.760 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T16:51:38.760 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T16:51:38.760 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T16:51:38.760 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:38.760 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:38.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:38.761 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T16:51:38.802 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:2024-04-25T16:51:38.804+0000 7eff0cfdc580 -1 init, newargv = 0x7efef001b520 newargc=15 2024-04-25T16:51:38.802 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:ceph-fuse[33594]: starting ceph client 2024-04-25T16:51:38.808 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:ceph-fuse[33594]: starting fuse 2024-04-25T16:51:39.907 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T16:51:39.923 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:39.971 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T16:51:39.971 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T16:51:39.971 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T16:51:39.971 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T16:51:39.971 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T16:51:39.971 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T16:51:39.971 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T16:51:39.971 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T16:51:39.971 INFO:teuthology.orchestra.run.smithi138.stdout:/run/lock 2024-04-25T16:51:39.971 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/fs/nfsd 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/run/rpc_pipefs 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T16:51:39.972 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T16:51:39.973 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T16:51:39.973 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T16:51:39.973 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T16:51:39.973 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T16:51:39.973 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:39.973 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:39.973 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T16:51:39.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:39.974 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T16:51:40.022 INFO:teuthology.orchestra.run.smithi138.stdout:45 2024-04-25T16:51:40.022 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T16:51:40.022 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-04-25T16:51:40.022 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-04-25T16:51:40.022 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-04-25T16:51:40.023 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T16:51:40.024 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T16:51:40.024 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T16:51:40.024 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-04-25T16:51:40.024 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-04-25T16:51:40.024 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-04-25T16:51:40.024 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T16:51:40.024 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T16:51:40.024 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.1")) 2024-04-25T16:51:40.024 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-04-25T16:51:40.144 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.1.33594.asok 2024-04-25T16:51:40.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:40.156 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33594.asok 2024-04-25T16:51:40.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:40.156 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33594.asok status 2024-04-25T16:51:40.335 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "1", 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "33594", 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4526, 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-04-25T16:51:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 4526 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2048925504 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2048925504 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.4526 192.168.0.1:0/2048925504", 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/2048925504", 2024-04-25T16:51:40.337 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-04-25T16:51:40.338 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 9, 2024-04-25T16:51:40.338 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 26, 2024-04-25T16:51:40.338 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-04-25T16:51:40.338 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-04-25T16:51:40.338 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-04-25T16:51:40.338 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T16:51:40.351 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T16:51:40.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.511+0000 7f7ac1134640 1 Processor -- start 2024-04-25T16:51:40.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.511+0000 7f7ac1134640 1 -- start start 2024-04-25T16:51:40.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7ac1134640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7abc15cf40 0x7f7abc06d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:40.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7ac1134640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06dc50 0x7f7abc0af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:40.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7ac1134640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7abc06e060 con 0x7f7abc070180 2024-04-25T16:51:40.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7ac1134640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7abc15dec0 con 0x7f7abc15cf40 2024-04-25T16:51:40.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7ac1134640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7abc160760 con 0x7f7abc06dc50 2024-04-25T16:51:40.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7aba59c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7abc15cf40 0x7f7abc06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:40.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7aba59c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7abc15cf40 0x7f7abc06d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48782/0 (socket says 172.21.15.94:48782) 2024-04-25T16:51:40.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7aba59c640 1 -- 172.21.15.94:0/1451858389 learned_addr learned my addr 172.21.15.94:0/1451858389 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:40.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7abb59e640 1 --2- 172.21.15.94:0/1451858389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06dc50 0x7f7abc0af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:40.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7aba59c640 1 -- 172.21.15.94:0/1451858389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06dc50 msgr2=0x7f7abc0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:40.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7aba59c640 1 --2- 172.21.15.94:0/1451858389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06dc50 0x7f7abc0af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:40.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7aba59c640 1 -- 172.21.15.94:0/1451858389 >> v1:172.21.15.138:6789/0 conn(0x7f7abc070180 legacy=0x7f7abc160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:40.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7aba59c640 1 -- 172.21.15.94:0/1451858389 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc161300 con 0x7f7abc15cf40 2024-04-25T16:51:40.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.515+0000 7f7abb59e640 1 --2- 172.21.15.94:0/1451858389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06dc50 0x7f7abc0af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:40.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7aba59c640 1 --2- 172.21.15.94:0/1451858389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc15cf40 0x7f7abc06d710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000bb10 tx=0x7f7ab001e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfacbbcc0c4724a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:40.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7ab9d9b640 1 -- 172.21.15.94:0/1451858389 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00b4070 con 0x7f7abc15cf40 2024-04-25T16:51:40.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7ab9d9b640 1 -- 172.21.15.94:0/1451858389 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab00a7980 con 0x7f7abc15cf40 2024-04-25T16:51:40.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7ab9d9b640 1 -- 172.21.15.94:0/1451858389 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00a7d50 con 0x7f7abc15cf40 2024-04-25T16:51:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7ac1134640 1 -- 172.21.15.94:0/1451858389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc15cf40 msgr2=0x7f7abc06d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7ac1134640 1 --2- 172.21.15.94:0/1451858389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc15cf40 0x7f7abc06d710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000bb10 tx=0x7f7ab001e9f0 comp rx=0 tx=0).stop 2024-04-25T16:51:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7ac1134640 1 -- 172.21.15.94:0/1451858389 shutdown_connections 2024-04-25T16:51:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7ac1134640 1 --2- 172.21.15.94:0/1451858389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06dc50 0x7f7abc0af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:40.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7ac1134640 1 --2- 172.21.15.94:0/1451858389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc15cf40 0x7f7abc06d710 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7ac1134640 1 -- 172.21.15.94:0/1451858389 >> 172.21.15.94:0/1451858389 conn(0x7f7abc0b47f0 msgr2=0x7f7abc0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7ac1134640 1 -- 172.21.15.94:0/1451858389 shutdown_connections 2024-04-25T16:51:40.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7ac1134640 1 -- 172.21.15.94:0/1451858389 wait complete. 2024-04-25T16:51:40.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7ac1134640 1 Processor -- start 2024-04-25T16:51:40.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.519+0000 7f7ac1134640 1 -- start start 2024-04-25T16:51:40.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7ac1134640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06dc50 0x7f7abc1589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:40.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7ac1134640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc070180 0x7f7abc154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:40.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7ac1134640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc15cf40 0x7f7abc09a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:40.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7ac1134640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc06fea0 con 0x7f7abc070180 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7ac1134640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc15dbe0 con 0x7f7abc15cf40 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7ac1134640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7abc0afc10 con 0x7f7abc06dc50 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7aba59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc070180 0x7f7abc154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7aba59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc070180 0x7f7abc154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48798/0 (socket says 172.21.15.94:48798) 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7aba59c640 1 -- 172.21.15.94:0/2771749072 learned_addr learned my addr 172.21.15.94:0/2771749072 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7abad9d640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06dc50 0x7f7abc1589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7abb59e640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc15cf40 0x7f7abc09a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7aba59c640 1 -- 172.21.15.94:0/2771749072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06dc50 msgr2=0x7f7abc1589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7aba59c640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06dc50 0x7f7abc1589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7aba59c640 1 -- 172.21.15.94:0/2771749072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc15cf40 msgr2=0x7f7abc09a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7aba59c640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc15cf40 0x7f7abc09a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7aba59c640 1 -- 172.21.15.94:0/2771749072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc159190 con 0x7f7abc070180 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7aba59c640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc070180 0x7f7abc154280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000b2c0 tx=0x7f7ab001b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:40.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7aa37fe640 1 -- 172.21.15.94:0/2771749072 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00b4040 con 0x7f7abc070180 2024-04-25T16:51:40.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7aa37fe640 1 -- 172.21.15.94:0/2771749072 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab00c1070 con 0x7f7abc070180 2024-04-25T16:51:40.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7ac1134640 1 -- 172.21.15.94:0/2771749072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7abc15f340 con 0x7f7abc070180 2024-04-25T16:51:40.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7aa37fe640 1 -- 172.21.15.94:0/2771749072 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00afa40 con 0x7f7abc070180 2024-04-25T16:51:40.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7ac1134640 1 -- 172.21.15.94:0/2771749072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7abc1573a0 con 0x7f7abc070180 2024-04-25T16:51:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.523+0000 7f7ac1134640 1 -- 172.21.15.94:0/2771749072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7abc076830 con 0x7f7abc070180 2024-04-25T16:51:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.527+0000 7f7aa37fe640 1 -- 172.21.15.94:0/2771749072 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7ab00b7070 con 0x7f7abc070180 2024-04-25T16:51:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.527+0000 7f7aa37fe640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7aa806f400 0x7f7aa8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.527+0000 7f7aa37fe640 1 -- 172.21.15.94:0/2771749072 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7ab012c310 con 0x7f7abc070180 2024-04-25T16:51:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.527+0000 7f7abad9d640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7aa806f400 0x7f7aa8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.527+0000 7f7abad9d640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7aa806f400 0x7f7aa8071910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7abc071c30 tx=0x7f7aac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:40.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.531+0000 7f7aa37fe640 1 -- 172.21.15.94:0/2771749072 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7abc06e060 con 0x7f7abc070180 2024-04-25T16:51:40.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.735+0000 7f7ac1134640 1 -- 172.21.15.94:0/2771749072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7abc158f20 con 0x7f7abc070180 2024-04-25T16:51:40.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.735+0000 7f7aa37fe640 1 -- 172.21.15.94:0/2771749072 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f7ab00f8e10 con 0x7f7abc070180 2024-04-25T16:51:40.733 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T16:51:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.735+0000 7f7ac1134640 1 -- 172.21.15.94:0/2771749072 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7aa806f400 msgr2=0x7f7aa8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.735+0000 7f7ac1134640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7aa806f400 0x7f7aa8071910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7abc071c30 tx=0x7f7aac008040 comp rx=0 tx=0).stop 2024-04-25T16:51:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.735+0000 7f7ac1134640 1 -- 172.21.15.94:0/2771749072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc070180 msgr2=0x7f7abc154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.735+0000 7f7ac1134640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc070180 0x7f7abc154280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000b2c0 tx=0x7f7ab001b790 comp rx=0 tx=0).stop 2024-04-25T16:51:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.735+0000 7f7ac1134640 1 -- 172.21.15.94:0/2771749072 shutdown_connections 2024-04-25T16:51:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.735+0000 7f7ac1134640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc15cf40 0x7f7abc09a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.735+0000 7f7ac1134640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7abc070180 0x7f7abc154280 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.735+0000 7f7ac1134640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f7aa806f400 0x7f7aa8071910 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.735+0000 7f7ac1134640 1 --2- 172.21.15.94:0/2771749072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06dc50 0x7f7abc1589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.735+0000 7f7ac1134640 1 -- 172.21.15.94:0/2771749072 >> 172.21.15.94:0/2771749072 conn(0x7f7abc0b47f0 msgr2=0x7f7abc0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.739+0000 7f7ac1134640 1 -- 172.21.15.94:0/2771749072 shutdown_connections 2024-04-25T16:51:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:40.739+0000 7f7ac1134640 1 -- 172.21.15.94:0/2771749072 wait complete. 2024-04-25T16:51:40.754 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T16:51:40.754 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T16:51:40.754 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-04-25T16:51:40.754 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T16:51:40.754 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T16:51:40.754 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T16:51:40.754 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T16:51:40.754 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T16:51:40.754 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T16:51:40.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:40.754 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-04-25T16:51:40.762 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:51:40.762 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:51:40.762 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:51:40.762 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:40.762 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-04-25T16:51:40.762 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:40.762 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:40.762 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:40.762 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-04-25T16:51:40.762 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42432sec preferred_lft 42432sec 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 02:42:f0:29:8e:1a brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T16:51:40.763 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:40.764 INFO:teuthology.orchestra.run.smithi138.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T16:51:40.764 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T16:51:40.764 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T16:51:40.764 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:40.764 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T16:51:40.764 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:40.764 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T16:51:40.764 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:40.764 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T16:51:40.764 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:40.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:40.764 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T16:51:40.817 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-04-25T16:51:40.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:40.819 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-04-25T16:51:40.871 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-04-25T16:51:40.872 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T16:51:40.872 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T16:51:40.872 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T16:51:40.872 INFO:teuthology.orchestra.run.smithi138.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T16:51:40.872 INFO:teuthology.orchestra.run.smithi138.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T16:51:40.872 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T16:51:40.872 INFO:teuthology.orchestra.run.smithi138.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:40.872 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:40.872 INFO:teuthology.orchestra.run.smithi138.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:51:40.872 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T16:51:40.872 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:40.873 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:51:40.873 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T16:51:40.873 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T16:51:40.873 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T16:51:40.873 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:51:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:40.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:40.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:40.996 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T16:51:41.001 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T16:51:41.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:41.001 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-04-25T16:51:41.049 INFO:teuthology.orchestra.run.smithi138.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T16:51:41.050 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:41.050 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:51:41.050 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T16:51:41.050 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T16:51:41.050 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:51:41.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:41.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:41.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:41.168 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T16:51:41.179 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:51:41.180 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:51:41.180 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:51:41.180 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:41.180 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-04-25T16:51:41.180 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:41.180 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T16:51:41.180 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 9e:92:ad:e1:3a:0b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T16:51:41.180 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T16:51:41.180 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:41.180 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::9c92:adff:fee1:3a0b/64 scope link 2024-04-25T16:51:41.180 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:41.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:41.181 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T16:51:41.236 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:51:41.236 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:51:41.236 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:51:41.236 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:41.236 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-04-25T16:51:41.236 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:41.237 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T16:51:41.237 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 9e:92:ad:e1:3a:0b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T16:51:41.237 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T16:51:41.237 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:41.237 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::9c92:adff:fee1:3a0b/64 scope link 2024-04-25T16:51:41.237 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:51:41.238 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T16:51:41.238 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:41.238 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:51:41.238 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T16:51:41.238 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T16:51:41.238 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T16:51:41.238 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T16:51:41.238 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T16:51:41.239 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:51:41.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:41.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:41.382 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:51:41.382 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T16:51:41.382 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.1 up 2024-04-25T16:51:41.382 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T16:51:41.382 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T16:51:41.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:51:41.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:41.442 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T16:51:41.442 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:51:41.442 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T16:51:41.489 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T16:51:41.490 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:51:41.490 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T16:51:41.537 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T16:51:41.595 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:41.643 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T16:51:41.643 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T16:51:41.643 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T16:51:41.643 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T16:51:41.643 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T16:51:41.643 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T16:51:41.643 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/run/lock 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/fs/nfsd 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:/run/rpc_pipefs 2024-04-25T16:51:41.645 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T16:51:41.645 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T16:51:41.645 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T16:51:41.645 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T16:51:41.645 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T16:51:41.645 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T16:51:41.645 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T16:51:41.645 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:41.645 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:41.645 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T16:51:41.645 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T16:51:41.645 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T16:51:41.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:41.646 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T16:51:41.694 INFO:teuthology.orchestra.run.smithi138.stdout:45 2024-04-25T16:51:41.694 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T16:51:41.694 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T16:51:41.739 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T16:51:41.753 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:41.803 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T16:51:41.803 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T16:51:41.803 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T16:51:41.803 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T16:51:41.803 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T16:51:41.803 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T16:51:41.803 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T16:51:41.803 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T16:51:41.803 INFO:teuthology.orchestra.run.smithi138.stdout:/run/lock 2024-04-25T16:51:41.803 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/fs/nfsd 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/run/rpc_pipefs 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T16:51:41.804 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T16:51:41.805 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T16:51:41.805 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T16:51:41.805 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T16:51:41.805 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T16:51:41.805 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T16:51:41.805 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:41.805 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:41.805 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T16:51:41.805 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T16:51:41.805 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T16:51:41.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:41.805 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T16:51:41.847 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi138.stderr:2024-04-25T16:51:41.848+0000 7f531abd2580 -1 init, newargv = 0x7f52fc01b560 newargc=15 2024-04-25T16:51:41.847 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi138.stderr:ceph-fuse[33713]: starting ceph client 2024-04-25T16:51:41.849 INFO:teuthology.orchestra.run.smithi138.stdout:45 2024-04-25T16:51:41.853 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi138.stderr:ceph-fuse[33713]: starting fuse 2024-04-25T16:51:42.951 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T16:51:42.967 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:51:43.015 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T16:51:43.016 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T16:51:43.016 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T16:51:43.016 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T16:51:43.016 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T16:51:43.016 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T16:51:43.016 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T16:51:43.016 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T16:51:43.016 INFO:teuthology.orchestra.run.smithi138.stdout:/run/lock 2024-04-25T16:51:43.016 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T16:51:43.016 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T16:51:43.016 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T16:51:43.016 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:51:43.016 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/fs/nfsd 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/run/rpc_pipefs 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T16:51:43.017 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:43.018 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T16:51:43.018 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T16:51:43.018 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T16:51:43.018 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T16:51:43.018 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.3 2024-04-25T16:51:43.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:43.019 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T16:51:43.066 INFO:teuthology.orchestra.run.smithi138.stdout:45 2024-04-25T16:51:43.066 INFO:teuthology.orchestra.run.smithi138.stdout:46 2024-04-25T16:51:43.066 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T16:51:43.066 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-04-25T16:51:43.066 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-04-25T16:51:43.066 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T16:51:43.067 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T16:51:43.068 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T16:51:43.068 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-04-25T16:51:43.068 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-04-25T16:51:43.068 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-04-25T16:51:43.068 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T16:51:43.068 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T16:51:43.068 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.3")) 2024-04-25T16:51:43.068 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-04-25T16:51:43.191 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.3.33713.asok 2024-04-25T16:51:43.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T16:51:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:51:43.203 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33713.asok 2024-04-25T16:51:43.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:43.203 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33713.asok status 2024-04-25T16:51:43.386 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T16:51:43.386 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-04-25T16:51:43.386 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T16:51:43.386 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T16:51:43.386 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "3", 2024-04-25T16:51:43.386 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-04-25T16:51:43.386 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T16:51:43.386 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "33713", 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4639, 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 4639 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.2:0", 2024-04-25T16:51:43.387 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 3412226445 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.2:0", 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 3412226445 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.4639 192.168.0.2:0/3412226445", 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.2:0/3412226445", 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 9, 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 26, 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-04-25T16:51:43.388 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T16:51:43.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:43.403 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-04-25T16:51:43.403 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T16:51:43.410 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-04-25T16:51:43.410 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T16:51:43.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:43.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T16:51:43.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:43.465 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-04-25T16:51:43.465 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T16:51:43.513 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-04-25T16:51:43.513 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T16:51:43.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:43.514 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T16:51:43.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:43.568 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-04-25T16:51:43.568 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T16:51:43.576 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-04-25T16:51:43.576 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T16:51:43.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:43.576 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T16:51:43.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:43.635 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-04-25T16:51:43.635 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T16:51:43.683 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-04-25T16:51:43.683 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T16:51:43.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:43.684 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T16:51:43.738 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T16:51:43.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:43.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb36693640 1 Processor -- start 2024-04-25T16:51:43.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb36693640 1 -- start start 2024-04-25T16:51:43.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb36693640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7feb3006e960 0x7feb30072670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:43.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb36693640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feb30160ae0 0x7feb300cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:43.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb36693640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb30072fc0 con 0x7feb30072bb0 2024-04-25T16:51:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb36693640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7feb300d3230 con 0x7feb3006e960 2024-04-25T16:51:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb36693640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feb300d2990 con 0x7feb30160ae0 2024-04-25T16:51:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb2ffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7feb3006e960 0x7feb30072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb2ffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7feb3006e960 0x7feb30072670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48808/0 (socket says 172.21.15.94:48808) 2024-04-25T16:51:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb2ffff640 1 -- 172.21.15.94:0/1848476868 learned_addr learned my addr 172.21.15.94:0/1848476868 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb34c30640 1 --2- 172.21.15.94:0/1848476868 >> v2:172.21.15.138:3300/0 conn(0x7feb30160ae0 0x7feb300cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb2ffff640 1 -- 172.21.15.94:0/1848476868 >> v2:172.21.15.138:3300/0 conn(0x7feb30160ae0 msgr2=0x7feb300cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb2ffff640 1 --2- 172.21.15.94:0/1848476868 >> v2:172.21.15.138:3300/0 conn(0x7feb30160ae0 0x7feb300cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb2ffff640 1 -- 172.21.15.94:0/1848476868 >> v1:172.21.15.138:6789/0 conn(0x7feb30072bb0 legacy=0x7feb300ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb2ffff640 1 -- 172.21.15.94:0/1848476868 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb300d2c70 con 0x7feb3006e960 2024-04-25T16:51:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.927+0000 7feb34c30640 1 --2- 172.21.15.94:0/1848476868 >> v2:172.21.15.138:3300/0 conn(0x7feb30160ae0 0x7feb300cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.931+0000 7feb2ffff640 1 --2- 172.21.15.94:0/1848476868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb3006e960 0x7feb30072670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7feb28009e50 tx=0x7feb2801c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52702bf63aed33ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.931+0000 7feb2effd640 1 -- 172.21.15.94:0/1848476868 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb280a55d0 con 0x7feb3006e960 2024-04-25T16:51:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.931+0000 7feb2effd640 1 -- 172.21.15.94:0/1848476868 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb280a5770 con 0x7feb3006e960 2024-04-25T16:51:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.931+0000 7feb2effd640 1 -- 172.21.15.94:0/1848476868 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb280a5b40 con 0x7feb3006e960 2024-04-25T16:51:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.931+0000 7feb36693640 1 -- 172.21.15.94:0/1848476868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb3006e960 msgr2=0x7feb30072670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.931+0000 7feb36693640 1 --2- 172.21.15.94:0/1848476868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb3006e960 0x7feb30072670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7feb28009e50 tx=0x7feb2801c710 comp rx=0 tx=0).stop 2024-04-25T16:51:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.931+0000 7feb36693640 1 -- 172.21.15.94:0/1848476868 shutdown_connections 2024-04-25T16:51:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.931+0000 7feb36693640 1 --2- 172.21.15.94:0/1848476868 >> v2:172.21.15.138:3300/0 conn(0x7feb30160ae0 0x7feb300cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.931+0000 7feb36693640 1 --2- 172.21.15.94:0/1848476868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb3006e960 0x7feb30072670 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.931+0000 7feb36693640 1 -- 172.21.15.94:0/1848476868 >> 172.21.15.94:0/1848476868 conn(0x7feb300d3850 msgr2=0x7feb300d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.931+0000 7feb36693640 1 -- 172.21.15.94:0/1848476868 shutdown_connections 2024-04-25T16:51:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.931+0000 7feb36693640 1 -- 172.21.15.94:0/1848476868 wait complete. 2024-04-25T16:51:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb36693640 1 Processor -- start 2024-04-25T16:51:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb36693640 1 -- start start 2024-04-25T16:51:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb36693640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb3006e960 0x7feb30159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb36693640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb30072bb0 0x7feb300ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb36693640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb30160ae0 0x7feb301542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb36693640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb3006eef0 con 0x7feb30160ae0 2024-04-25T16:51:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb36693640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb3006ed70 con 0x7feb30072bb0 2024-04-25T16:51:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb36693640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feb3016c7d0 con 0x7feb3006e960 2024-04-25T16:51:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb34c30640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb30160ae0 0x7feb301542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb34c30640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb30160ae0 0x7feb301542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48810/0 (socket says 172.21.15.94:48810) 2024-04-25T16:51:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb34c30640 1 -- 172.21.15.94:0/2466148616 learned_addr learned my addr 172.21.15.94:0/2466148616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb2f7fe640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb30072bb0 0x7feb300ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb2ffff640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb3006e960 0x7feb30159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb34c30640 1 -- 172.21.15.94:0/2466148616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb3006e960 msgr2=0x7feb30159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb34c30640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb3006e960 0x7feb30159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb34c30640 1 -- 172.21.15.94:0/2466148616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb30072bb0 msgr2=0x7feb300ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb34c30640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb30072bb0 0x7feb300ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.935+0000 7feb34c30640 1 -- 172.21.15.94:0/2466148616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3014fd80 con 0x7feb30160ae0 2024-04-25T16:51:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.939+0000 7feb34c30640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb30160ae0 0x7feb301542e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c5c0 tx=0x7feb2000ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.939+0000 7feb2cff9640 1 -- 172.21.15.94:0/2466148616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20015070 con 0x7feb30160ae0 2024-04-25T16:51:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.939+0000 7feb36693640 1 -- 172.21.15.94:0/2466148616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb30150060 con 0x7feb30160ae0 2024-04-25T16:51:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.939+0000 7feb2cff9640 1 -- 172.21.15.94:0/2466148616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb20011820 con 0x7feb30160ae0 2024-04-25T16:51:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.939+0000 7feb2cff9640 1 -- 172.21.15.94:0/2466148616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20011c10 con 0x7feb30160ae0 2024-04-25T16:51:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.939+0000 7feb36693640 1 -- 172.21.15.94:0/2466148616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb30150620 con 0x7feb30160ae0 2024-04-25T16:51:43.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.939+0000 7feb36693640 1 -- 172.21.15.94:0/2466148616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb3006a730 con 0x7feb30160ae0 2024-04-25T16:51:43.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.939+0000 7feb2cff9640 1 -- 172.21.15.94:0/2466148616 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feb20018070 con 0x7feb30160ae0 2024-04-25T16:51:43.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.939+0000 7feb2cff9640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feb1006f400 0x7feb10071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:43.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.939+0000 7feb2cff9640 1 -- 172.21.15.94:0/2466148616 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7feb2008f4c0 con 0x7feb30160ae0 2024-04-25T16:51:43.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.939+0000 7feb2ffff640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feb1006f400 0x7feb10071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:43.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.939+0000 7feb2ffff640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feb1006f400 0x7feb10071910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feb280ae000 tx=0x7feb28004ac0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:43.943+0000 7feb2cff9640 1 -- 172.21.15.94:0/2466148616 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feb2005b970 con 0x7feb30160ae0 2024-04-25T16:51:44.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.107+0000 7feb36693640 1 -- 172.21.15.94:0/2466148616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb300cad90 con 0x7feb30160ae0 2024-04-25T16:51:44.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.107+0000 7feb2cff9640 1 -- 172.21.15.94:0/2466148616 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7feb2005fc70 con 0x7feb30160ae0 2024-04-25T16:51:44.107 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:44.107 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.111+0000 7feb36693640 1 -- 172.21.15.94:0/2466148616 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feb1006f400 msgr2=0x7feb10071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.111+0000 7feb36693640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feb1006f400 0x7feb10071910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feb280ae000 tx=0x7feb28004ac0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.111+0000 7feb36693640 1 -- 172.21.15.94:0/2466148616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb30160ae0 msgr2=0x7feb301542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.111+0000 7feb36693640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb30160ae0 0x7feb301542e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c5c0 tx=0x7feb2000ca80 comp rx=0 tx=0).stop 2024-04-25T16:51:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.111+0000 7feb36693640 1 -- 172.21.15.94:0/2466148616 shutdown_connections 2024-04-25T16:51:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.111+0000 7feb36693640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb30160ae0 0x7feb301542e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.111+0000 7feb36693640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb30072bb0 0x7feb300ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.111+0000 7feb36693640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feb1006f400 0x7feb10071910 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.111+0000 7feb36693640 1 --2- 172.21.15.94:0/2466148616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb3006e960 0x7feb30159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.111+0000 7feb36693640 1 -- 172.21.15.94:0/2466148616 >> 172.21.15.94:0/2466148616 conn(0x7feb300d3850 msgr2=0x7feb30162690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.111+0000 7feb36693640 1 -- 172.21.15.94:0/2466148616 shutdown_connections 2024-04-25T16:51:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.111+0000 7feb36693640 1 -- 172.21.15.94:0/2466148616 wait complete. 2024-04-25T16:51:44.129 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:51:44.129 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:51:44.257 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:51:44.262 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:51:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99c72a640 1 Processor -- start 2024-04-25T16:51:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99c72a640 1 -- start start 2024-04-25T16:51:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99c72a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb99406fc90 0x7fb9940cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99c72a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb994163d40 0x7fb994164150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99c72a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9940b7aa0 con 0x7fb9940d1340 2024-04-25T16:51:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99c72a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fb9940d53e0 con 0x7fb994163d40 2024-04-25T16:51:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99c72a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb9941603c0 con 0x7fb99406fc90 2024-04-25T16:51:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99acc7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb994163d40 0x7fb994164150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99acc7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb994163d40 0x7fb994164150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48816/0 (socket says 172.21.15.94:48816) 2024-04-25T16:51:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99acc7640 1 -- 172.21.15.94:0/2331225270 learned_addr learned my addr 172.21.15.94:0/2331225270 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99a4c6640 1 --2- 172.21.15.94:0/2331225270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb99406fc90 0x7fb9940cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99acc7640 1 -- 172.21.15.94:0/2331225270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb99406fc90 msgr2=0x7fb9940cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99acc7640 1 --2- 172.21.15.94:0/2331225270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb99406fc90 0x7fb9940cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99acc7640 1 -- 172.21.15.94:0/2331225270 >> v1:172.21.15.138:6789/0 conn(0x7fb9940d1340 legacy=0x7fb9940d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.283+0000 7fb99acc7640 1 -- 172.21.15.94:0/2331225270 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9940d4a40 con 0x7fb994163d40 2024-04-25T16:51:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.287+0000 7fb99acc7640 1 --2- 172.21.15.94:0/2331225270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994163d40 0x7fb994164150 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009e50 tx=0x7fb98c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112efa2d0199ab49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.287+0000 7fb9994c4640 1 -- 172.21.15.94:0/2331225270 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0a57b0 con 0x7fb994163d40 2024-04-25T16:51:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.287+0000 7fb9994c4640 1 -- 172.21.15.94:0/2331225270 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98c0a5950 con 0x7fb994163d40 2024-04-25T16:51:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.287+0000 7fb9994c4640 1 -- 172.21.15.94:0/2331225270 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0a5d20 con 0x7fb994163d40 2024-04-25T16:51:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.287+0000 7fb99c72a640 1 -- 172.21.15.94:0/2331225270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994163d40 msgr2=0x7fb994164150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.287+0000 7fb99c72a640 1 --2- 172.21.15.94:0/2331225270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994163d40 0x7fb994164150 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009e50 tx=0x7fb98c01c930 comp rx=0 tx=0).stop 2024-04-25T16:51:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.287+0000 7fb99c72a640 1 -- 172.21.15.94:0/2331225270 shutdown_connections 2024-04-25T16:51:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.287+0000 7fb99c72a640 1 --2- 172.21.15.94:0/2331225270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994163d40 0x7fb994164150 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.287+0000 7fb99c72a640 1 --2- 172.21.15.94:0/2331225270 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb99406fc90 0x7fb9940cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.287+0000 7fb99c72a640 1 -- 172.21.15.94:0/2331225270 >> 172.21.15.94:0/2331225270 conn(0x7fb99408d000 msgr2=0x7fb99409e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.287+0000 7fb99c72a640 1 -- 172.21.15.94:0/2331225270 shutdown_connections 2024-04-25T16:51:44.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.287+0000 7fb99c72a640 1 -- 172.21.15.94:0/2331225270 wait complete. 2024-04-25T16:51:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99c72a640 1 Processor -- start 2024-04-25T16:51:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99c72a640 1 -- start start 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99c72a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb99406fc90 0x7fb99415fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99c72a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9940d1340 0x7fb994152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99c72a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994163d40 0x7fb99415e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99c72a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb9940d3230 con 0x7fb994163d40 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99c72a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9940d30b0 con 0x7fb99406fc90 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99c72a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb9940d4660 con 0x7fb9940d1340 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99acc7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994163d40 0x7fb99415e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99acc7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994163d40 0x7fb99415e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48830/0 (socket says 172.21.15.94:48830) 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99acc7640 1 -- 172.21.15.94:0/147310031 learned_addr learned my addr 172.21.15.94:0/147310031 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb999cc5640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9940d1340 0x7fb994152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99a4c6640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb99406fc90 0x7fb99415fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99acc7640 1 -- 172.21.15.94:0/147310031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9940d1340 msgr2=0x7fb994152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99acc7640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9940d1340 0x7fb994152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99acc7640 1 -- 172.21.15.94:0/147310031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb99406fc90 msgr2=0x7fb99415fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99acc7640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb99406fc90 0x7fb99415fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99acc7640 1 -- 172.21.15.94:0/147310031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99415ec60 con 0x7fb994163d40 2024-04-25T16:51:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb999cc5640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9940d1340 0x7fb994152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99a4c6640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb99406fc90 0x7fb99415fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99acc7640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994163d40 0x7fb99415e310 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb98c01afe0 tx=0x7fb98c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb98affd640 1 -- 172.21.15.94:0/147310031 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0bc020 con 0x7fb994163d40 2024-04-25T16:51:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb98affd640 1 -- 172.21.15.94:0/147310031 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98c0b8670 con 0x7fb994163d40 2024-04-25T16:51:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb98affd640 1 -- 172.21.15.94:0/147310031 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0b8a20 con 0x7fb994163d40 2024-04-25T16:51:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99c72a640 1 -- 172.21.15.94:0/147310031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb99415ef40 con 0x7fb994163d40 2024-04-25T16:51:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.291+0000 7fb99c72a640 1 -- 172.21.15.94:0/147310031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb99415f500 con 0x7fb994163d40 2024-04-25T16:51:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.295+0000 7fb99c72a640 1 -- 172.21.15.94:0/147310031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb99406a730 con 0x7fb994163d40 2024-04-25T16:51:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.295+0000 7fb98affd640 1 -- 172.21.15.94:0/147310031 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb98c0bf020 con 0x7fb994163d40 2024-04-25T16:51:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.295+0000 7fb98affd640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb97806f430 0x7fb978071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.295+0000 7fb98affd640 1 -- 172.21.15.94:0/147310031 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb98c12a770 con 0x7fb994163d40 2024-04-25T16:51:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.295+0000 7fb99a4c6640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb97806f430 0x7fb978071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.295+0000 7fb99a4c6640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb97806f430 0x7fb978071940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb9940c8ed0 tx=0x7fb984008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.299+0000 7fb98affd640 1 -- 172.21.15.94:0/147310031 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb98c0c6030 con 0x7fb994163d40 2024-04-25T16:51:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb99c72a640 1 -- 172.21.15.94:0/147310031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb9940d27e0 con 0x7fb994163d40 2024-04-25T16:51:44.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb98affd640 1 -- 172.21.15.94:0/147310031 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fb98c0165e0 con 0x7fb994163d40 2024-04-25T16:51:44.457 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:51:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb99c72a640 1 -- 172.21.15.94:0/147310031 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb97806f430 msgr2=0x7fb978071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb99c72a640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb97806f430 0x7fb978071940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb9940c8ed0 tx=0x7fb984008040 comp rx=0 tx=0).stop 2024-04-25T16:51:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb99c72a640 1 -- 172.21.15.94:0/147310031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994163d40 msgr2=0x7fb99415e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb99c72a640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994163d40 0x7fb99415e310 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb98c01afe0 tx=0x7fb98c0196d0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb99c72a640 1 -- 172.21.15.94:0/147310031 shutdown_connections 2024-04-25T16:51:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb99c72a640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994163d40 0x7fb99415e310 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb99c72a640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb97806f430 0x7fb978071940 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb99c72a640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9940d1340 0x7fb994152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb99c72a640 1 --2- 172.21.15.94:0/147310031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb99406fc90 0x7fb99415fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb99c72a640 1 -- 172.21.15.94:0/147310031 >> 172.21.15.94:0/147310031 conn(0x7fb99408d000 msgr2=0x7fb9940d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:44.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb99c72a640 1 -- 172.21.15.94:0/147310031 shutdown_connections 2024-04-25T16:51:44.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.459+0000 7fb99c72a640 1 -- 172.21.15.94:0/147310031 wait complete. 2024-04-25T16:51:44.476 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:51:44.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.631+0000 7f8d104ce640 1 Processor -- start 2024-04-25T16:51:44.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.631+0000 7f8d104ce640 1 -- start start 2024-04-25T16:51:44.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d104ce640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8d0806e7a0 0x7f8d0806ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d104ce640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d08074380 con 0x7f8d0806d710 2024-04-25T16:51:44.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d104ce640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d080727e0 con 0x7f8d0815c7e0 2024-04-25T16:51:44.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d104ce640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f8d08072ac0 con 0x7f8d0806e7a0 2024-04-25T16:51:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d0da69640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f8d0806d710 0x7f8d0806db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50356/0 (socket says 172.21.15.94:50356) 2024-04-25T16:51:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d0da69640 1 -- 172.21.15.94:0/712518993 learned_addr learned my addr 172.21.15.94:0/712518993 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d0ea6b640 1 --2- 172.21.15.94:0/712518993 >> v2:172.21.15.94:3300/0 conn(0x7f8d0806e7a0 0x7f8d0806ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d0d268640 1 -- 172.21.15.94:0/712518993 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 738839373 0 0) 0x7f8cf8001120 con 0x7f8d0806d710 2024-04-25T16:51:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d0d268640 1 -- 172.21.15.94:0/712518993 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cf4005740 con 0x7f8d0806d710 2024-04-25T16:51:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d0ea6b640 1 -- 172.21.15.94:0/712518993 >> v1:172.21.15.138:6789/0 conn(0x7f8d0815c7e0 legacy=0x7f8d080afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d0ea6b640 1 -- 172.21.15.94:0/712518993 >> v1:172.21.15.94:6789/0 conn(0x7f8d0806d710 legacy=0x7f8d0806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d0ea6b640 1 -- 172.21.15.94:0/712518993 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d08072da0 con 0x7f8d0806e7a0 2024-04-25T16:51:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d0ea6b640 1 --2- 172.21.15.94:0/712518993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0806e7a0 0x7f8d0806ebb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009d90 tx=0x7f8d0001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf37ac4763469a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d0d268640 1 -- 172.21.15.94:0/712518993 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d000b2070 con 0x7f8d0806e7a0 2024-04-25T16:51:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d0d268640 1 -- 172.21.15.94:0/712518993 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d000a5870 con 0x7f8d0806e7a0 2024-04-25T16:51:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d0d268640 1 -- 172.21.15.94:0/712518993 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d000a5c40 con 0x7f8d0806e7a0 2024-04-25T16:51:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d104ce640 1 -- 172.21.15.94:0/712518993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0806e7a0 msgr2=0x7f8d0806ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d104ce640 1 --2- 172.21.15.94:0/712518993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0806e7a0 0x7f8d0806ebb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009d90 tx=0x7f8d0001c8b0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d104ce640 1 -- 172.21.15.94:0/712518993 shutdown_connections 2024-04-25T16:51:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d104ce640 1 --2- 172.21.15.94:0/712518993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0806e7a0 0x7f8d0806ebb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d104ce640 1 -- 172.21.15.94:0/712518993 >> 172.21.15.94:0/712518993 conn(0x7f8d080b47f0 msgr2=0x7f8d080b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.635+0000 7f8d104ce640 1 -- 172.21.15.94:0/712518993 shutdown_connections 2024-04-25T16:51:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d104ce640 1 -- 172.21.15.94:0/712518993 wait complete. 2024-04-25T16:51:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d104ce640 1 Processor -- start 2024-04-25T16:51:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d104ce640 1 -- start start 2024-04-25T16:51:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d104ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0806d710 0x7f8d0809a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d104ce640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0806e7a0 0x7f8d08143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d104ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0815c7e0 0x7f8d081438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d104ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8d080b0500 con 0x7f8d0806d710 2024-04-25T16:51:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d104ce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0815cbf0 con 0x7f8d0815c7e0 2024-04-25T16:51:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d104ce640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d080b0680 con 0x7f8d0806e7a0 2024-04-25T16:51:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d0e26a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0806d710 0x7f8d0809a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d0e26a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0806d710 0x7f8d0809a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48850/0 (socket says 172.21.15.94:48850) 2024-04-25T16:51:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d0e26a640 1 -- 172.21.15.94:0/19391052 learned_addr learned my addr 172.21.15.94:0/19391052 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d0ea6b640 1 --2- 172.21.15.94:0/19391052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0815c7e0 0x7f8d081438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d0e26a640 1 -- 172.21.15.94:0/19391052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0806e7a0 msgr2=0x7f8d08143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d0e26a640 1 --2- 172.21.15.94:0/19391052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0806e7a0 0x7f8d08143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d0e26a640 1 -- 172.21.15.94:0/19391052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0815c7e0 msgr2=0x7f8d081438c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d0e26a640 1 --2- 172.21.15.94:0/19391052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0815c7e0 0x7f8d081438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d0e26a640 1 -- 172.21.15.94:0/19391052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0815af90 con 0x7f8d0806d710 2024-04-25T16:51:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8d0e26a640 1 --2- 172.21.15.94:0/19391052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0806d710 0x7f8d0809a7f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400c3a0 tx=0x7f8d0400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.639+0000 7f8cfeffd640 1 -- 172.21.15.94:0/19391052 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d040052d0 con 0x7f8d0806d710 2024-04-25T16:51:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.643+0000 7f8cfeffd640 1 -- 172.21.15.94:0/19391052 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d04005470 con 0x7f8d0806d710 2024-04-25T16:51:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.643+0000 7f8cfeffd640 1 -- 172.21.15.94:0/19391052 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04007260 con 0x7f8d0806d710 2024-04-25T16:51:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.643+0000 7f8d104ce640 1 -- 172.21.15.94:0/19391052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0815b270 con 0x7f8d0806d710 2024-04-25T16:51:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.643+0000 7f8d104ce640 1 -- 172.21.15.94:0/19391052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0815b830 con 0x7f8d0806d710 2024-04-25T16:51:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.643+0000 7f8d104ce640 1 -- 172.21.15.94:0/19391052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d08076830 con 0x7f8d0806d710 2024-04-25T16:51:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.643+0000 7f8cfeffd640 1 -- 172.21.15.94:0/19391052 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8d04007400 con 0x7f8d0806d710 2024-04-25T16:51:44.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.643+0000 7f8cfeffd640 1 --2- 172.21.15.94:0/19391052 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8cec06f400 0x7f8cec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.643+0000 7f8d0da69640 1 --2- 172.21.15.94:0/19391052 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8cec06f400 0x7f8cec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.643+0000 7f8cfeffd640 1 -- 172.21.15.94:0/19391052 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8d0408ebf0 con 0x7f8d0806d710 2024-04-25T16:51:44.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.643+0000 7f8d0da69640 1 --2- 172.21.15.94:0/19391052 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8cec06f400 0x7f8cec071910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8005890 tx=0x7f8cf8001160 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:44.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.647+0000 7f8cfeffd640 1 -- 172.21.15.94:0/19391052 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8d0405b660 con 0x7f8d0806d710 2024-04-25T16:51:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8d104ce640 1 -- 172.21.15.94:0/19391052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8d081601d0 con 0x7f8d0806d710 2024-04-25T16:51:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8cfeffd640 1 -- 172.21.15.94:0/19391052 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f8d0405af60 con 0x7f8d0806d710 2024-04-25T16:51:44.809 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8d104ce640 1 -- 172.21.15.94:0/19391052 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8cec06f400 msgr2=0x7f8cec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8d104ce640 1 --2- 172.21.15.94:0/19391052 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8cec06f400 0x7f8cec071910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8005890 tx=0x7f8cf8001160 comp rx=0 tx=0).stop 2024-04-25T16:51:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8d104ce640 1 -- 172.21.15.94:0/19391052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0806d710 msgr2=0x7f8d0809a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8d104ce640 1 --2- 172.21.15.94:0/19391052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0806d710 0x7f8d0809a7f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400c3a0 tx=0x7f8d0400c860 comp rx=0 tx=0).stop 2024-04-25T16:51:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8d104ce640 1 -- 172.21.15.94:0/19391052 shutdown_connections 2024-04-25T16:51:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8d104ce640 1 --2- 172.21.15.94:0/19391052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0815c7e0 0x7f8d081438c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8d104ce640 1 --2- 172.21.15.94:0/19391052 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8cec06f400 0x7f8cec071910 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8d104ce640 1 --2- 172.21.15.94:0/19391052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0806e7a0 0x7f8d08143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8d104ce640 1 --2- 172.21.15.94:0/19391052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0806d710 0x7f8d0809a7f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8d104ce640 1 -- 172.21.15.94:0/19391052 >> 172.21.15.94:0/19391052 conn(0x7f8d080b47f0 msgr2=0x7f8d080af370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8d104ce640 1 -- 172.21.15.94:0/19391052 shutdown_connections 2024-04-25T16:51:44.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.811+0000 7f8d104ce640 1 -- 172.21.15.94:0/19391052 wait complete. 2024-04-25T16:51:44.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:51:44.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.983+0000 7f74c08a0640 1 Processor -- start 2024-04-25T16:51:44.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.983+0000 7f74c08a0640 1 -- start start 2024-04-25T16:51:44.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.983+0000 7f74c08a0640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f74bc06d710 0x7f74bc06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.983+0000 7f74c08a0640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74bc074380 con 0x7f74bc15c7e0 2024-04-25T16:51:44.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.983+0000 7f74c08a0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74bc06e060 con 0x7f74bc06e7a0 2024-04-25T16:51:44.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.983+0000 7f74c08a0640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f74bc0727e0 con 0x7f74bc06d710 2024-04-25T16:51:44.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.983+0000 7f74ba59c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f74bc15c7e0 0x7f74bc0afdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50366/0 (socket says 172.21.15.94:50366) 2024-04-25T16:51:44.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.983+0000 7f74ba59c640 1 -- 172.21.15.94:0/130408800 learned_addr learned my addr 172.21.15.94:0/130408800 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:44.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 245342156 0 0) 0x7f74ac001120 con 0x7f74bc15c7e0 2024-04-25T16:51:44.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74a4003530 con 0x7f74bc15c7e0 2024-04-25T16:51:44.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b9d9b640 1 --2- 172.21.15.94:0/130408800 >> v2:172.21.15.138:3300/0 conn(0x7f74bc06d710 0x7f74bc06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2344043100 0 0) 0x7f74a4003530 con 0x7f74bc15c7e0 2024-04-25T16:51:44.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 >> v2:172.21.15.138:3300/0 conn(0x7f74bc06d710 msgr2=0x7f74bc06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:44.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 --2- 172.21.15.94:0/130408800 >> v2:172.21.15.138:3300/0 conn(0x7f74bc06d710 0x7f74bc06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 >> v1:172.21.15.138:6789/0 conn(0x7f74bc06e7a0 legacy=0x7f74bc06ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc072ac0 con 0x7f74bc15c7e0 2024-04-25T16:51:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f74ac001850 con 0x7f74bc15c7e0 2024-04-25T16:51:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 >> v1:172.21.15.94:6789/0 conn(0x7f74bc15c7e0 legacy=0x7f74bc0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 --2- 172.21.15.94:0/130408800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a40039c0 0x7f74a4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 --2- 172.21.15.94:0/130408800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a4004810 0x7f74a4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 --2- 172.21.15.94:0/130408800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a4005310 0x7f74a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc0727e0 con 0x7f74a40039c0 2024-04-25T16:51:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74a400c2b0 con 0x7f74a4005310 2024-04-25T16:51:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f74a400c4b0 con 0x7f74a4004810 2024-04-25T16:51:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74ba59c640 1 --2- 172.21.15.94:0/130408800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a40039c0 0x7f74a4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b9d9b640 1 --2- 172.21.15.94:0/130408800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a4004810 0x7f74a4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74bad9d640 1 --2- 172.21.15.94:0/130408800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a4005310 0x7f74a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74ba59c640 1 -- 172.21.15.94:0/130408800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a4004810 msgr2=0x7f74a4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:44.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74ba59c640 1 --2- 172.21.15.94:0/130408800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74a4004810 0x7f74a4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74ba59c640 1 -- 172.21.15.94:0/130408800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a4005310 msgr2=0x7f74a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:44.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74ba59c640 1 --2- 172.21.15.94:0/130408800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a4005310 0x7f74a400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74bad9d640 1 -- 172.21.15.94:0/130408800 reap_dead start 2024-04-25T16:51:44.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74ba59c640 1 -- 172.21.15.94:0/130408800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a4003530 con 0x7f74a40039c0 2024-04-25T16:51:44.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74ba59c640 1 --2- 172.21.15.94:0/130408800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a40039c0 0x7f74a4003e30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00c4c0 tx=0x7f74ac01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f71c764b025d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:44.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00c0e0 con 0x7f74a40039c0 2024-04-25T16:51:44.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74ac0a9070 con 0x7f74a40039c0 2024-04-25T16:51:44.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.987+0000 7f74b959a640 1 -- 172.21.15.94:0/130408800 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac005d80 con 0x7f74a40039c0 2024-04-25T16:51:44.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 -- 172.21.15.94:0/130408800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a40039c0 msgr2=0x7f74a4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 --2- 172.21.15.94:0/130408800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a40039c0 0x7f74a4003e30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00c4c0 tx=0x7f74ac01cc50 comp rx=0 tx=0).stop 2024-04-25T16:51:44.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 -- 172.21.15.94:0/130408800 shutdown_connections 2024-04-25T16:51:44.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 --2- 172.21.15.94:0/130408800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a40039c0 0x7f74a4003e30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 -- 172.21.15.94:0/130408800 >> 172.21.15.94:0/130408800 conn(0x7f74bc0b47f0 msgr2=0x7f74bc0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:44.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 -- 172.21.15.94:0/130408800 shutdown_connections 2024-04-25T16:51:44.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 -- 172.21.15.94:0/130408800 wait complete. 2024-04-25T16:51:44.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 Processor -- start 2024-04-25T16:51:44.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 -- start start 2024-04-25T16:51:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a40039c0 0x7f74bc09e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74bc1fa360 0x7f74bc06c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74bc1fa770 0x7f74bc06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc15cbf0 con 0x7f74bc1fa360 2024-04-25T16:51:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc073ca0 con 0x7f74a40039c0 2024-04-25T16:51:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74c08a0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc0afdf0 con 0x7f74bc1fa770 2024-04-25T16:51:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74b9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74bc1fa360 0x7f74bc06c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74b9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74bc1fa360 0x7f74bc06c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48866/0 (socket says 172.21.15.94:48866) 2024-04-25T16:51:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74b9d9b640 1 -- 172.21.15.94:0/1257270779 learned_addr learned my addr 172.21.15.94:0/1257270779 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.991+0000 7f74bad9d640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74bc1fa770 0x7f74bc06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74ba59c640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a40039c0 0x7f74bc09e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74b9d9b640 1 -- 172.21.15.94:0/1257270779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74bc1fa770 msgr2=0x7f74bc06cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74b9d9b640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74bc1fa770 0x7f74bc06cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74b9d9b640 1 -- 172.21.15.94:0/1257270779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a40039c0 msgr2=0x7f74bc09e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74b9d9b640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a40039c0 0x7f74bc09e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74b9d9b640 1 -- 172.21.15.94:0/1257270779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc14fdf0 con 0x7f74bc1fa360 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74bad9d640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74bc1fa770 0x7f74bc06cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74b9d9b640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74bc1fa360 0x7f74bc06c600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f74b000f600 tx=0x7f74b000fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74aaffd640 1 -- 172.21.15.94:0/1257270779 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0019020 con 0x7f74bc1fa360 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74aaffd640 1 -- 172.21.15.94:0/1257270779 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b0014870 con 0x7f74bc1fa360 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74aaffd640 1 -- 172.21.15.94:0/1257270779 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0014c60 con 0x7f74bc1fa360 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74c08a0640 1 -- 172.21.15.94:0/1257270779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc15a040 con 0x7f74bc1fa360 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74c08a0640 1 -- 172.21.15.94:0/1257270779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc15a600 con 0x7f74bc1fa360 2024-04-25T16:51:44.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74c08a0640 1 -- 172.21.15.94:0/1257270779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74bc076830 con 0x7f74bc1fa360 2024-04-25T16:51:44.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74aaffd640 1 -- 172.21.15.94:0/1257270779 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f74b001c020 con 0x7f74bc1fa360 2024-04-25T16:51:44.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74aaffd640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f749c06f400 0x7f749c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:44.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74aaffd640 1 -- 172.21.15.94:0/1257270779 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f74b008fe80 con 0x7f74bc1fa360 2024-04-25T16:51:44.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74ba59c640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f749c06f400 0x7f749c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:44.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:44.995+0000 7f74ba59c640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f749c06f400 0x7f749c071910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0ab000 tx=0x7f74ac0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:45.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.003+0000 7f74aaffd640 1 -- 172.21.15.94:0/1257270779 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f74b0023020 con 0x7f74bc1fa360 2024-04-25T16:51:45.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.163+0000 7f74c08a0640 1 -- 172.21.15.94:0/1257270779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f74bc09ee50 con 0x7f74bc1fa360 2024-04-25T16:51:45.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.163+0000 7f74aaffd640 1 -- 172.21.15.94:0/1257270779 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f74b005ca00 con 0x7f74bc1fa360 2024-04-25T16:51:45.161 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:51:45.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.163+0000 7f74c08a0640 1 -- 172.21.15.94:0/1257270779 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f749c06f400 msgr2=0x7f749c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.163+0000 7f74c08a0640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f749c06f400 0x7f749c071910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0ab000 tx=0x7f74ac0a7070 comp rx=0 tx=0).stop 2024-04-25T16:51:45.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.163+0000 7f74c08a0640 1 -- 172.21.15.94:0/1257270779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74bc1fa360 msgr2=0x7f74bc06c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.163+0000 7f74c08a0640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74bc1fa360 0x7f74bc06c600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f74b000f600 tx=0x7f74b000fac0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.163+0000 7f74c08a0640 1 -- 172.21.15.94:0/1257270779 shutdown_connections 2024-04-25T16:51:45.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.163+0000 7f74c08a0640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f749c06f400 0x7f749c071910 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.163+0000 7f74c08a0640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74bc1fa770 0x7f74bc06cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.163+0000 7f74c08a0640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74bc1fa360 0x7f74bc06c600 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.163+0000 7f74c08a0640 1 --2- 172.21.15.94:0/1257270779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a40039c0 0x7f74bc09e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.167+0000 7f74c08a0640 1 -- 172.21.15.94:0/1257270779 >> 172.21.15.94:0/1257270779 conn(0x7f74bc0b47f0 msgr2=0x7f74bc0b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:45.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.167+0000 7f74c08a0640 1 -- 172.21.15.94:0/1257270779 shutdown_connections 2024-04-25T16:51:45.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.167+0000 7f74c08a0640 1 -- 172.21.15.94:0/1257270779 wait complete. 2024-04-25T16:51:45.183 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:51:45.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f45a6143640 1 Processor -- start 2024-04-25T16:51:45.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f45a6143640 1 -- start start 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f45a6143640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f45a015c7e0 0x7f45a00afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f45a6143640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a00b0330 0x7f45a006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f45a6143640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45a0074380 con 0x7f45a006dc50 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f45a6143640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f45a006e060 con 0x7f45a015c7e0 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f45a6143640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f45a00727e0 con 0x7f45a00b0330 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459ffff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f45a006dc50 0x7f45a006e7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50382/0 (socket says 172.21.15.94:50382) 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459ffff640 1 -- 172.21.15.94:0/2737137536 learned_addr learned my addr 172.21.15.94:0/2737137536 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459f7fe640 1 --2- 172.21.15.94:0/2737137536 >> v2:172.21.15.138:3300/0 conn(0x7f45a015c7e0 0x7f45a00afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f4597fff640 1 --2- 172.21.15.94:0/2737137536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a00b0330 0x7f45a006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4180365673 0 0) 0x7f45980014a0 con 0x7f45a006dc50 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f458c003530 con 0x7f45a006dc50 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3499631093 0 0) 0x7f458c003530 con 0x7f45a006dc50 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a00b0330 msgr2=0x7f45a006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 --2- 172.21.15.94:0/2737137536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a00b0330 0x7f45a006d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 >> v2:172.21.15.138:3300/0 conn(0x7f45a015c7e0 msgr2=0x7f45a00afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 --2- 172.21.15.94:0/2737137536 >> v2:172.21.15.138:3300/0 conn(0x7f45a015c7e0 0x7f45a00afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a0072ac0 con 0x7f45a006dc50 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f4598001bd0 con 0x7f45a006dc50 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 >> v1:172.21.15.94:6789/0 conn(0x7f45a006dc50 legacy=0x7f45a006e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 --2- 172.21.15.94:0/2737137536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c0039c0 0x7f458c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 --2- 172.21.15.94:0/2737137536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a00b0330 0x7f458c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 --2- 172.21.15.94:0/2737137536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c004d50 0x7f458c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f45a006e060 con 0x7f458c0039c0 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45a00727e0 con 0x7f458c004d50 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f458c00ba50 con 0x7f45a00b0330 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459f7fe640 1 --2- 172.21.15.94:0/2737137536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c0039c0 0x7f458c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f4597fff640 1 --2- 172.21.15.94:0/2737137536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a00b0330 0x7f458c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459f7fe640 1 -- 172.21.15.94:0/2737137536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a00b0330 msgr2=0x7f458c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459f7fe640 1 --2- 172.21.15.94:0/2737137536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a00b0330 0x7f458c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459ffff640 1 --2- 172.21.15.94:0/2737137536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c004d50 0x7f458c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459f7fe640 1 -- 172.21.15.94:0/2737137536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c004d50 msgr2=0x7f458c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:45.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459f7fe640 1 --2- 172.21.15.94:0/2737137536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c004d50 0x7f458c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459f7fe640 1 -- 172.21.15.94:0/2737137536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c003530 con 0x7f458c0039c0 2024-04-25T16:51:45.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.339+0000 7f459ffff640 1 --2- 172.21.15.94:0/2737137536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c004d50 0x7f458c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:45.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.343+0000 7f459f7fe640 1 --2- 172.21.15.94:0/2737137536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c0039c0 0x7f458c003e30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4590009ae0 tx=0x7f459001ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39776354b4905e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:45.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.343+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45900b2070 con 0x7f458c0039c0 2024-04-25T16:51:45.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.343+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45900a5650 con 0x7f458c0039c0 2024-04-25T16:51:45.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.343+0000 7f459effd640 1 -- 172.21.15.94:0/2737137536 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45900096b0 con 0x7f458c0039c0 2024-04-25T16:51:45.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.343+0000 7f45a6143640 1 -- 172.21.15.94:0/2737137536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c0039c0 msgr2=0x7f458c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.343+0000 7f45a6143640 1 --2- 172.21.15.94:0/2737137536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c0039c0 0x7f458c003e30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4590009ae0 tx=0x7f459001ca80 comp rx=0 tx=0).stop 2024-04-25T16:51:45.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.343+0000 7f459ffff640 1 -- 172.21.15.94:0/2737137536 reap_dead start 2024-04-25T16:51:45.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.343+0000 7f45a6143640 1 -- 172.21.15.94:0/2737137536 shutdown_connections 2024-04-25T16:51:45.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.343+0000 7f45a6143640 1 -- 172.21.15.94:0/2737137536 >> 172.21.15.94:0/2737137536 conn(0x7f45a00b47f0 msgr2=0x7f45a00b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:45.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.343+0000 7f45a6143640 1 -- 172.21.15.94:0/2737137536 shutdown_connections 2024-04-25T16:51:45.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.343+0000 7f45a6143640 1 -- 172.21.15.94:0/2737137536 wait complete. 2024-04-25T16:51:45.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f45a6143640 1 Processor -- start 2024-04-25T16:51:45.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f45a6143640 1 -- start start 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f45a6143640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c0039c0 0x7f45a01584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f45a6143640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45a014a100 0x7f45a0165cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f45a6143640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a0166210 0x7f45a01568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f45a6143640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f45a006f2c0 con 0x7f45a014a100 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f45a6143640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45a0073ca0 con 0x7f458c0039c0 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f45a6143640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f45a015cbf0 con 0x7f45a0166210 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f4597fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45a014a100 0x7f45a0165cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f4597fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45a014a100 0x7f45a0165cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48884/0 (socket says 172.21.15.94:48884) 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f4597fff640 1 -- 172.21.15.94:0/1105059433 learned_addr learned my addr 172.21.15.94:0/1105059433 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f459f7fe640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c0039c0 0x7f45a01584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f459ffff640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a0166210 0x7f45a01568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f4597fff640 1 -- 172.21.15.94:0/1105059433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a0166210 msgr2=0x7f45a01568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f4597fff640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a0166210 0x7f45a01568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f4597fff640 1 -- 172.21.15.94:0/1105059433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c0039c0 msgr2=0x7f45a01584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f4597fff640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c0039c0 0x7f45a01584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f4597fff640 1 -- 172.21.15.94:0/1105059433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a0166620 con 0x7f45a014a100 2024-04-25T16:51:45.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f4597fff640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45a014a100 0x7f45a0165cd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4588010820 tx=0x7f4588010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:45.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f459cff9640 1 -- 172.21.15.94:0/1105059433 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458800f040 con 0x7f45a014a100 2024-04-25T16:51:45.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f459cff9640 1 -- 172.21.15.94:0/1105059433 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458800eb00 con 0x7f45a014a100 2024-04-25T16:51:45.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f45a6143640 1 -- 172.21.15.94:0/1105059433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a00b1f30 con 0x7f45a014a100 2024-04-25T16:51:45.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f459cff9640 1 -- 172.21.15.94:0/1105059433 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458801c400 con 0x7f45a014a100 2024-04-25T16:51:45.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f45a6143640 1 -- 172.21.15.94:0/1105059433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a00b2710 con 0x7f45a014a100 2024-04-25T16:51:45.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.347+0000 7f45a6143640 1 -- 172.21.15.94:0/1105059433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a00b21a0 con 0x7f45a014a100 2024-04-25T16:51:45.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.351+0000 7f459cff9640 1 -- 172.21.15.94:0/1105059433 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f458801c5a0 con 0x7f45a014a100 2024-04-25T16:51:45.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.351+0000 7f459cff9640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f458006f430 0x7f4580071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.351+0000 7f459cff9640 1 -- 172.21.15.94:0/1105059433 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f458808fc30 con 0x7f45a014a100 2024-04-25T16:51:45.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.351+0000 7f459f7fe640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f458006f430 0x7f4580071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.351+0000 7f459f7fe640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f458006f430 0x7f4580071940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f45900af000 tx=0x7f45900046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:45.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.355+0000 7f459cff9640 1 -- 172.21.15.94:0/1105059433 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f45a0166620 con 0x7f45a014a100 2024-04-25T16:51:45.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.503+0000 7f45a6143640 1 -- 172.21.15.94:0/1105059433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f45a0076830 con 0x7f45a014a100 2024-04-25T16:51:45.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.503+0000 7f459cff9640 1 -- 172.21.15.94:0/1105059433 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f4588018030 con 0x7f45a014a100 2024-04-25T16:51:45.502 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:45.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.507+0000 7f45a6143640 1 -- 172.21.15.94:0/1105059433 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f458006f430 msgr2=0x7f4580071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.507+0000 7f45a6143640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f458006f430 0x7f4580071940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f45900af000 tx=0x7f45900046d0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.507+0000 7f45a6143640 1 -- 172.21.15.94:0/1105059433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45a014a100 msgr2=0x7f45a0165cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.507+0000 7f45a6143640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45a014a100 0x7f45a0165cd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4588010820 tx=0x7f4588010ce0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.507+0000 7f45a6143640 1 -- 172.21.15.94:0/1105059433 shutdown_connections 2024-04-25T16:51:45.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.507+0000 7f45a6143640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f458006f430 0x7f4580071940 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.507+0000 7f45a6143640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a0166210 0x7f45a01568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.507+0000 7f45a6143640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45a014a100 0x7f45a0165cd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.507+0000 7f45a6143640 1 --2- 172.21.15.94:0/1105059433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c0039c0 0x7f45a01584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.507+0000 7f45a6143640 1 -- 172.21.15.94:0/1105059433 >> 172.21.15.94:0/1105059433 conn(0x7f45a00b47f0 msgr2=0x7f45a006f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:45.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.507+0000 7f45a6143640 1 -- 172.21.15.94:0/1105059433 shutdown_connections 2024-04-25T16:51:45.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.507+0000 7f45a6143640 1 -- 172.21.15.94:0/1105059433 wait complete. 2024-04-25T16:51:45.523 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:45.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.667+0000 7f396aafe640 1 Processor -- start 2024-04-25T16:51:45.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.667+0000 7f396aafe640 1 -- start start 2024-04-25T16:51:45.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.667+0000 7f396aafe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3964070570 0x7f396406ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.667+0000 7f396aafe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39640d1340 0x7f39640d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.667+0000 7f396aafe640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39640d1c90 con 0x7f39641615a0 2024-04-25T16:51:45.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.667+0000 7f396aafe640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f396406e830 con 0x7f3964070570 2024-04-25T16:51:45.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.667+0000 7f396aafe640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f396406d790 con 0x7f39640d1340 2024-04-25T16:51:45.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.667+0000 7f396889a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3964070570 0x7f396406ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.667+0000 7f396889a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3964070570 0x7f396406ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48888/0 (socket says 172.21.15.94:48888) 2024-04-25T16:51:45.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.667+0000 7f396889a640 1 -- 172.21.15.94:0/3935040798 learned_addr learned my addr 172.21.15.94:0/3935040798 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:45.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.667+0000 7f3963fff640 1 --2- 172.21.15.94:0/3935040798 >> v2:172.21.15.138:3300/0 conn(0x7f39640d1340 0x7f39640d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.667+0000 7f396889a640 1 -- 172.21.15.94:0/3935040798 >> v2:172.21.15.138:3300/0 conn(0x7f39640d1340 msgr2=0x7f39640d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:45.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f396889a640 1 --2- 172.21.15.94:0/3935040798 >> v2:172.21.15.138:3300/0 conn(0x7f39640d1340 0x7f39640d1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f396889a640 1 -- 172.21.15.94:0/3935040798 >> v1:172.21.15.138:6789/0 conn(0x7f39641615a0 legacy=0x7f39641619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f396889a640 1 -- 172.21.15.94:0/3935040798 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396406da70 con 0x7f3964070570 2024-04-25T16:51:45.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f396889a640 1 --2- 172.21.15.94:0/3935040798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3964070570 0x7f396406ce90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3964072440 tx=0x7f395c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e3c7a064ffdf75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:45.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f39637fe640 1 -- 172.21.15.94:0/3935040798 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c0a57a0 con 0x7f3964070570 2024-04-25T16:51:45.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f39637fe640 1 -- 172.21.15.94:0/3935040798 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f395c0a5940 con 0x7f3964070570 2024-04-25T16:51:45.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f39637fe640 1 -- 172.21.15.94:0/3935040798 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c0a5d10 con 0x7f3964070570 2024-04-25T16:51:45.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f396aafe640 1 -- 172.21.15.94:0/3935040798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3964070570 msgr2=0x7f396406ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f396aafe640 1 --2- 172.21.15.94:0/3935040798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3964070570 0x7f396406ce90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3964072440 tx=0x7f395c01c9f0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f396aafe640 1 -- 172.21.15.94:0/3935040798 shutdown_connections 2024-04-25T16:51:45.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f396aafe640 1 --2- 172.21.15.94:0/3935040798 >> v2:172.21.15.138:3300/0 conn(0x7f39640d1340 0x7f39640d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f396aafe640 1 --2- 172.21.15.94:0/3935040798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3964070570 0x7f396406ce90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f396aafe640 1 -- 172.21.15.94:0/3935040798 >> 172.21.15.94:0/3935040798 conn(0x7f3964071150 msgr2=0x7f3964071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:45.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f396aafe640 1 -- 172.21.15.94:0/3935040798 shutdown_connections 2024-04-25T16:51:45.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.671+0000 7f396aafe640 1 -- 172.21.15.94:0/3935040798 wait complete. 2024-04-25T16:51:45.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396aafe640 1 Processor -- start 2024-04-25T16:51:45.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396aafe640 1 -- start start 2024-04-25T16:51:45.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396aafe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3964070570 0x7f396415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396aafe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39640d1340 0x7f39640b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396aafe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39641615a0 0x7f39640b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396aafe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f396406d550 con 0x7f39641615a0 2024-04-25T16:51:45.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396aafe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f396406d3d0 con 0x7f3964070570 2024-04-25T16:51:45.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396aafe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f39640cfe30 con 0x7f39640d1340 2024-04-25T16:51:45.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396909b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39641615a0 0x7f39640b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396909b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39641615a0 0x7f39640b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48900/0 (socket says 172.21.15.94:48900) 2024-04-25T16:51:45.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396909b640 1 -- 172.21.15.94:0/1480283735 learned_addr learned my addr 172.21.15.94:0/1480283735 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:45.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396889a640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3964070570 0x7f396415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f3963fff640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39640d1340 0x7f39640b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396909b640 1 -- 172.21.15.94:0/1480283735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39640d1340 msgr2=0x7f39640b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396909b640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39640d1340 0x7f39640b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396909b640 1 -- 172.21.15.94:0/1480283735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3964070570 msgr2=0x7f396415b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396909b640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3964070570 0x7f396415b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.675+0000 7f396909b640 1 -- 172.21.15.94:0/1480283735 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3964162f30 con 0x7f39641615a0 2024-04-25T16:51:45.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.679+0000 7f396909b640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39641615a0 0x7f39640b25b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f394c00c340 tx=0x7f394c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:45.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.679+0000 7f39617fa640 1 -- 172.21.15.94:0/1480283735 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c017070 con 0x7f39641615a0 2024-04-25T16:51:45.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.679+0000 7f39617fa640 1 -- 172.21.15.94:0/1480283735 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f394c0052d0 con 0x7f39641615a0 2024-04-25T16:51:45.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.679+0000 7f39617fa640 1 -- 172.21.15.94:0/1480283735 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c003800 con 0x7f39641615a0 2024-04-25T16:51:45.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.679+0000 7f396aafe640 1 -- 172.21.15.94:0/1480283735 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396415b840 con 0x7f39641615a0 2024-04-25T16:51:45.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.679+0000 7f396aafe640 1 -- 172.21.15.94:0/1480283735 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f396415c1e0 con 0x7f39641615a0 2024-04-25T16:51:45.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.679+0000 7f396aafe640 1 -- 172.21.15.94:0/1480283735 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3964144400 con 0x7f39641615a0 2024-04-25T16:51:45.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.679+0000 7f39617fa640 1 -- 172.21.15.94:0/1480283735 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f394c01a070 con 0x7f39641615a0 2024-04-25T16:51:45.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.679+0000 7f39617fa640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f394406f430 0x7f3944071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:45.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.679+0000 7f39617fa640 1 -- 172.21.15.94:0/1480283735 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f394c08ebf0 con 0x7f39641615a0 2024-04-25T16:51:45.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.679+0000 7f396889a640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f394406f430 0x7f3944071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:45.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.679+0000 7f396889a640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f394406f430 0x7f3944071940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f395c0ae000 tx=0x7f395c0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:45.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.683+0000 7f39617fa640 1 -- 172.21.15.94:0/1480283735 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f39640d1c90 con 0x7f39641615a0 2024-04-25T16:51:45.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.839+0000 7f396aafe640 1 -- 172.21.15.94:0/1480283735 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f396406a730 con 0x7f39641615a0 2024-04-25T16:51:45.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.843+0000 7f39617fa640 1 -- 172.21.15.94:0/1480283735 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7f394c005480 con 0x7f39641615a0 2024-04-25T16:51:45.841 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:45.841 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:45.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.843+0000 7f396aafe640 1 -- 172.21.15.94:0/1480283735 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f394406f430 msgr2=0x7f3944071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.843+0000 7f396aafe640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f394406f430 0x7f3944071940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f395c0ae000 tx=0x7f395c0a7070 comp rx=0 tx=0).stop 2024-04-25T16:51:45.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.843+0000 7f396aafe640 1 -- 172.21.15.94:0/1480283735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39641615a0 msgr2=0x7f39640b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:45.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.843+0000 7f396aafe640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39641615a0 0x7f39640b25b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f394c00c340 tx=0x7f394c00c800 comp rx=0 tx=0).stop 2024-04-25T16:51:45.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.843+0000 7f396aafe640 1 -- 172.21.15.94:0/1480283735 shutdown_connections 2024-04-25T16:51:45.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.843+0000 7f396aafe640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39641615a0 0x7f39640b25b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.843+0000 7f396aafe640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f394406f430 0x7f3944071940 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.843+0000 7f396aafe640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39640d1340 0x7f39640b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.843+0000 7f396aafe640 1 --2- 172.21.15.94:0/1480283735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3964070570 0x7f396415b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:45.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.843+0000 7f396aafe640 1 -- 172.21.15.94:0/1480283735 >> 172.21.15.94:0/1480283735 conn(0x7f3964071150 msgr2=0x7f396413fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:45.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.843+0000 7f396aafe640 1 -- 172.21.15.94:0/1480283735 shutdown_connections 2024-04-25T16:51:45.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:45.843+0000 7f396aafe640 1 -- 172.21.15.94:0/1480283735 wait complete. 2024-04-25T16:51:45.860 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:51:45.861 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:51:46.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.007+0000 7f2f81209640 1 Processor -- start 2024-04-25T16:51:46.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.007+0000 7f2f81209640 1 -- start start 2024-04-25T16:51:46.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f81209640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f7c0b0420 0x7f2f7c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f81209640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f7c06f2a0 0x7f2f7c06c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f81209640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c06cb40 0x7f2f7c06cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f81209640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f2f7c06d490 con 0x7f2f7c0b0420 2024-04-25T16:51:46.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f81209640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2f7c0aeaa0 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f81209640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7c0703e0 con 0x7f2f7c06cb40 2024-04-25T16:51:46.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f7ad9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f7c0b0420 0x7f2f7c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f7ad9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f7c0b0420 0x7f2f7c06ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48906/0 (socket says 172.21.15.94:48906) 2024-04-25T16:51:46.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f7ad9d640 1 -- 172.21.15.94:0/4120363607 learned_addr learned my addr 172.21.15.94:0/4120363607 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:46.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f7a59c640 1 --2- 172.21.15.94:0/4120363607 >> v2:172.21.15.138:3300/0 conn(0x7f2f7c06f2a0 0x7f2f7c06c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f7b59e640 1 --2- 172.21.15.94:0/4120363607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c06cb40 0x7f2f7c06cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f7ad9d640 1 -- 172.21.15.94:0/4120363607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c06cb40 msgr2=0x7f2f7c06cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:46.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f7ad9d640 1 --2- 172.21.15.94:0/4120363607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c06cb40 0x7f2f7c06cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f7ad9d640 1 -- 172.21.15.94:0/4120363607 >> v2:172.21.15.138:3300/0 conn(0x7f2f7c06f2a0 msgr2=0x7f2f7c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:46.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f7ad9d640 1 --2- 172.21.15.94:0/4120363607 >> v2:172.21.15.138:3300/0 conn(0x7f2f7c06f2a0 0x7f2f7c06c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f7ad9d640 1 -- 172.21.15.94:0/4120363607 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7c070630 con 0x7f2f7c0b0420 2024-04-25T16:51:46.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f7a59c640 1 --2- 172.21.15.94:0/4120363607 >> v2:172.21.15.138:3300/0 conn(0x7f2f7c06f2a0 0x7f2f7c06c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:46.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.011+0000 7f2f7b59e640 1 --2- 172.21.15.94:0/4120363607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c06cb40 0x7f2f7c06cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:46.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f7ad9d640 1 --2- 172.21.15.94:0/4120363607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c0b0420 0x7f2f7c06ed60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0d1190 tx=0x7f2f6c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ad1de81259a707 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:46.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f79d9b640 1 -- 172.21.15.94:0/4120363607 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0b1070 con 0x7f2f7c0b0420 2024-04-25T16:51:46.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f79d9b640 1 -- 172.21.15.94:0/4120363607 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f6c0ac630 con 0x7f2f7c0b0420 2024-04-25T16:51:46.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f79d9b640 1 -- 172.21.15.94:0/4120363607 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0a8050 con 0x7f2f7c0b0420 2024-04-25T16:51:46.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f81209640 1 -- 172.21.15.94:0/4120363607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c0b0420 msgr2=0x7f2f7c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f81209640 1 --2- 172.21.15.94:0/4120363607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c0b0420 0x7f2f7c06ed60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0d1190 tx=0x7f2f6c01c520 comp rx=0 tx=0).stop 2024-04-25T16:51:46.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f81209640 1 -- 172.21.15.94:0/4120363607 shutdown_connections 2024-04-25T16:51:46.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f81209640 1 --2- 172.21.15.94:0/4120363607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c06cb40 0x7f2f7c06cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f81209640 1 --2- 172.21.15.94:0/4120363607 >> v2:172.21.15.138:3300/0 conn(0x7f2f7c06f2a0 0x7f2f7c06c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f81209640 1 --2- 172.21.15.94:0/4120363607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c0b0420 0x7f2f7c06ed60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f81209640 1 -- 172.21.15.94:0/4120363607 >> 172.21.15.94:0/4120363607 conn(0x7f2f7c0b47f0 msgr2=0x7f2f7c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:46.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f81209640 1 -- 172.21.15.94:0/4120363607 shutdown_connections 2024-04-25T16:51:46.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f81209640 1 -- 172.21.15.94:0/4120363607 wait complete. 2024-04-25T16:51:46.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f81209640 1 Processor -- start 2024-04-25T16:51:46.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.015+0000 7f2f81209640 1 -- start start 2024-04-25T16:51:46.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f81209640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f7c06cb40 0x7f2f7c157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f81209640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c06f2a0 0x7f2f7c158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f81209640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c0b0420 0x7f2f7c1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f81209640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c0b0830 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f81209640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c0aef30 con 0x7f2f7c06cb40 2024-04-25T16:51:46.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f81209640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7c0b09b0 con 0x7f2f7c0b0420 2024-04-25T16:51:46.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f7a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c06f2a0 0x7f2f7c158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f7a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c06f2a0 0x7f2f7c158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48922/0 (socket says 172.21.15.94:48922) 2024-04-25T16:51:46.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f7a59c640 1 -- 172.21.15.94:0/3256155856 learned_addr learned my addr 172.21.15.94:0/3256155856 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:46.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f7b59e640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c0b0420 0x7f2f7c1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f7ad9d640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f7c06cb40 0x7f2f7c157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f7a59c640 1 -- 172.21.15.94:0/3256155856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c0b0420 msgr2=0x7f2f7c1592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f7a59c640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c0b0420 0x7f2f7c1592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f7a59c640 1 -- 172.21.15.94:0/3256155856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f7c06cb40 msgr2=0x7f2f7c157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f7a59c640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f7c06cb40 0x7f2f7c157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f7a59c640 1 -- 172.21.15.94:0/3256155856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7c147870 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f7a59c640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c06f2a0 0x7f2f7c158db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000c7f0 tx=0x7f2f7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:46.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f637fe640 1 -- 172.21.15.94:0/3256155856 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70015070 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f637fe640 1 -- 172.21.15.94:0/3256155856 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f70007820 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f81209640 1 -- 172.21.15.94:0/3256155856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f7c147b50 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f637fe640 1 -- 172.21.15.94:0/3256155856 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70007c10 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f81209640 1 -- 172.21.15.94:0/3256155856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f7c148110 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f81209640 1 -- 172.21.15.94:0/3256155856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f7c09a690 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.019+0000 7f2f637fe640 1 -- 172.21.15.94:0/3256155856 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2f70018070 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.023+0000 7f2f637fe640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f6806f400 0x7f2f68071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.023+0000 7f2f637fe640 1 -- 172.21.15.94:0/3256155856 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2f7008fb60 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.023+0000 7f2f7ad9d640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f6806f400 0x7f2f68071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.023+0000 7f2f7ad9d640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f6806f400 0x7f2f68071910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0ae000 tx=0x7f2f6c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:46.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.027+0000 7f2f637fe640 1 -- 172.21.15.94:0/3256155856 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2f7005c7c0 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.171+0000 7f2f81209640 1 -- 172.21.15.94:0/3256155856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2f7c076830 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.175+0000 7f2f637fe640 1 -- 172.21.15.94:0/3256155856 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f2f7005c0c0 con 0x7f2f7c06f2a0 2024-04-25T16:51:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:51:46.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.175+0000 7f2f81209640 1 -- 172.21.15.94:0/3256155856 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f6806f400 msgr2=0x7f2f68071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.175+0000 7f2f81209640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f6806f400 0x7f2f68071910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0ae000 tx=0x7f2f6c0a7000 comp rx=0 tx=0).stop 2024-04-25T16:51:46.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.175+0000 7f2f81209640 1 -- 172.21.15.94:0/3256155856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c06f2a0 msgr2=0x7f2f7c158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.175+0000 7f2f81209640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c06f2a0 0x7f2f7c158db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000c7f0 tx=0x7f2f7000ccb0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.175+0000 7f2f81209640 1 -- 172.21.15.94:0/3256155856 shutdown_connections 2024-04-25T16:51:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.175+0000 7f2f81209640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f6806f400 0x7f2f68071910 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.175+0000 7f2f81209640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f7c0b0420 0x7f2f7c1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.175+0000 7f2f81209640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f7c06f2a0 0x7f2f7c158db0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.175+0000 7f2f81209640 1 --2- 172.21.15.94:0/3256155856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f7c06cb40 0x7f2f7c157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.175+0000 7f2f81209640 1 -- 172.21.15.94:0/3256155856 >> 172.21.15.94:0/3256155856 conn(0x7f2f7c0b47f0 msgr2=0x7f2f7c15ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.175+0000 7f2f81209640 1 -- 172.21.15.94:0/3256155856 shutdown_connections 2024-04-25T16:51:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.175+0000 7f2f81209640 1 -- 172.21.15.94:0/3256155856 wait complete. 2024-04-25T16:51:46.191 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:51:46.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7fecd28e6640 1 Processor -- start 2024-04-25T16:51:46.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7fecd28e6640 1 -- start start 2024-04-25T16:51:46.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7fecd28e6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feccc0b0bf0 0x7feccc0b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7fecd28e6640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feccc0b6dc0 con 0x7feccc0b07e0 2024-04-25T16:51:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7fecd28e6640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feccc0b1110 con 0x7feccc0b69b0 2024-04-25T16:51:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7fecd28e6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feccc0b1d20 con 0x7feccc0b0bf0 2024-04-25T16:51:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccbfff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7feccc0b07e0 0x7feccc0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50392/0 (socket says 172.21.15.94:50392) 2024-04-25T16:51:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccbfff640 1 -- 172.21.15.94:0/3708489614 learned_addr learned my addr 172.21.15.94:0/3708489614 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccb7fe640 1 --2- 172.21.15.94:0/3708489614 >> v2:172.21.15.138:3300/0 conn(0x7feccc0b0bf0 0x7feccc0b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 586680804 0 0) 0x7feccc0b6dc0 con 0x7feccc0b07e0 2024-04-25T16:51:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecb4003530 con 0x7feccc0b07e0 2024-04-25T16:51:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1356939938 0 0) 0x7fecb4003530 con 0x7feccc0b07e0 2024-04-25T16:51:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 >> v2:172.21.15.138:3300/0 conn(0x7feccc0b0bf0 msgr2=0x7feccc0b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 --2- 172.21.15.94:0/3708489614 >> v2:172.21.15.138:3300/0 conn(0x7feccc0b0bf0 0x7feccc0b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 >> v1:172.21.15.138:6789/0 conn(0x7feccc0b69b0 legacy=0x7feccc0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc0b13f0 con 0x7feccc0b07e0 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fecbc001680 con 0x7feccc0b07e0 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 >> v1:172.21.15.94:6789/0 conn(0x7feccc0b07e0 legacy=0x7feccc0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 --2- 172.21.15.94:0/3708489614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb40039c0 0x7fecb4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 --2- 172.21.15.94:0/3708489614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecb4004810 0x7fecb4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 --2- 172.21.15.94:0/3708489614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecb4005280 0x7fecb400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feccc0b1d20 con 0x7fecb40039c0 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fecb400c1a0 con 0x7fecb4005280 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fecb400c3a0 con 0x7fecb4004810 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccbfff640 1 --2- 172.21.15.94:0/3708489614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb40039c0 0x7fecb4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7fecd0e83640 1 --2- 172.21.15.94:0/3708489614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecb4005280 0x7fecb400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccbfff640 1 -- 172.21.15.94:0/3708489614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecb4004810 msgr2=0x7fecb4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccbfff640 1 --2- 172.21.15.94:0/3708489614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecb4004810 0x7fecb4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccbfff640 1 -- 172.21.15.94:0/3708489614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecb4005280 msgr2=0x7fecb400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccbfff640 1 --2- 172.21.15.94:0/3708489614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecb4005280 0x7fecb400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7feccbfff640 1 -- 172.21.15.94:0/3708489614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb4003530 con 0x7fecb40039c0 2024-04-25T16:51:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.331+0000 7fecd0e83640 1 -- 172.21.15.94:0/3708489614 reap_dead start 2024-04-25T16:51:46.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.335+0000 7feccbfff640 1 --2- 172.21.15.94:0/3708489614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb40039c0 0x7fecb4003e50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c210 tx=0x7fecbc01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d0e667924cff4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:46.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.335+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc007d20 con 0x7fecb40039c0 2024-04-25T16:51:46.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.335+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecbc0a9070 con 0x7fecb40039c0 2024-04-25T16:51:46.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.335+0000 7feccaffd640 1 -- 172.21.15.94:0/3708489614 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc005da0 con 0x7fecb40039c0 2024-04-25T16:51:46.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.335+0000 7fecd28e6640 1 -- 172.21.15.94:0/3708489614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb40039c0 msgr2=0x7fecb4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.335+0000 7fecd28e6640 1 --2- 172.21.15.94:0/3708489614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb40039c0 0x7fecb4003e50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c210 tx=0x7fecbc01cc30 comp rx=0 tx=0).stop 2024-04-25T16:51:46.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.335+0000 7fecd28e6640 1 -- 172.21.15.94:0/3708489614 shutdown_connections 2024-04-25T16:51:46.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.335+0000 7fecd28e6640 1 --2- 172.21.15.94:0/3708489614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb40039c0 0x7fecb4003e50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.335+0000 7fecd28e6640 1 -- 172.21.15.94:0/3708489614 >> 172.21.15.94:0/3708489614 conn(0x7feccc071f00 msgr2=0x7feccc073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:46.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.335+0000 7fecd28e6640 1 -- 172.21.15.94:0/3708489614 shutdown_connections 2024-04-25T16:51:46.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.335+0000 7fecd28e6640 1 -- 172.21.15.94:0/3708489614 wait complete. 2024-04-25T16:51:46.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecd28e6640 1 Processor -- start 2024-04-25T16:51:46.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecd28e6640 1 -- start start 2024-04-25T16:51:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecd28e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb40039c0 0x7feccc13a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecd28e6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feccc15b200 0x7feccc143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecd28e6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feccc1f9420 0x7feccc143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecd28e6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feccc0d1910 con 0x7fecb40039c0 2024-04-25T16:51:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecd28e6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feccc0d1790 con 0x7feccc15b200 2024-04-25T16:51:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecd28e6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feccc0d1a90 con 0x7feccc1f9420 2024-04-25T16:51:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7feccbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb40039c0 0x7feccc13a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7feccbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb40039c0 0x7feccc13a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48936/0 (socket says 172.21.15.94:48936) 2024-04-25T16:51:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7feccbfff640 1 -- 172.21.15.94:0/1237252389 learned_addr learned my addr 172.21.15.94:0/1237252389 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7feccb7fe640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feccc15b200 0x7feccc143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecd0e83640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feccc1f9420 0x7feccc143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7feccbfff640 1 -- 172.21.15.94:0/1237252389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feccc1f9420 msgr2=0x7feccc143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7feccbfff640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feccc1f9420 0x7feccc143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7feccbfff640 1 -- 172.21.15.94:0/1237252389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feccc15b200 msgr2=0x7feccc143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7feccbfff640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feccc15b200 0x7feccc143940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7feccbfff640 1 -- 172.21.15.94:0/1237252389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc1f9c30 con 0x7fecb40039c0 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecd0e83640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feccc1f9420 0x7feccc143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7feccbfff640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb40039c0 0x7feccc13a8c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fecbc001980 tx=0x7fecb4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecc8ff9640 1 -- 172.21.15.94:0/1237252389 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc007d20 con 0x7fecb40039c0 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecc8ff9640 1 -- 172.21.15.94:0/1237252389 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecbc000ee0 con 0x7fecb40039c0 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecd28e6640 1 -- 172.21.15.94:0/1237252389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc1f9f10 con 0x7fecb40039c0 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecc8ff9640 1 -- 172.21.15.94:0/1237252389 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc007180 con 0x7fecb40039c0 2024-04-25T16:51:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecd28e6640 1 -- 172.21.15.94:0/1237252389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc0d1200 con 0x7fecb40039c0 2024-04-25T16:51:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.339+0000 7fecd28e6640 1 -- 172.21.15.94:0/1237252389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc13b120 con 0x7fecb40039c0 2024-04-25T16:51:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.343+0000 7fecc8ff9640 1 -- 172.21.15.94:0/1237252389 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fecbc007320 con 0x7fecb40039c0 2024-04-25T16:51:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.343+0000 7fecc8ff9640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fecb806f400 0x7fecb8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.343+0000 7fecc8ff9640 1 -- 172.21.15.94:0/1237252389 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fecbc128f00 con 0x7fecb40039c0 2024-04-25T16:51:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.343+0000 7feccb7fe640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fecb806f400 0x7fecb8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.343+0000 7feccb7fe640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fecb806f400 0x7fecb8071910 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feccc15d930 tx=0x7fecc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.347+0000 7fecc8ff9640 1 -- 172.21.15.94:0/1237252389 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fecbc016970 con 0x7fecb40039c0 2024-04-25T16:51:46.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.495+0000 7fecd28e6640 1 -- 172.21.15.94:0/1237252389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feccc0cfae0 con 0x7fecb40039c0 2024-04-25T16:51:46.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.495+0000 7fecc8ff9640 1 -- 172.21.15.94:0/1237252389 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fecbc0f59f0 con 0x7fecb40039c0 2024-04-25T16:51:46.495 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.499+0000 7fecd28e6640 1 -- 172.21.15.94:0/1237252389 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fecb806f400 msgr2=0x7fecb8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.499+0000 7fecd28e6640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fecb806f400 0x7fecb8071910 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feccc15d930 tx=0x7fecc0008040 comp rx=0 tx=0).stop 2024-04-25T16:51:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.499+0000 7fecd28e6640 1 -- 172.21.15.94:0/1237252389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb40039c0 msgr2=0x7feccc13a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.499+0000 7fecd28e6640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb40039c0 0x7feccc13a8c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fecbc001980 tx=0x7fecb4005840 comp rx=0 tx=0).stop 2024-04-25T16:51:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.499+0000 7fecd28e6640 1 -- 172.21.15.94:0/1237252389 shutdown_connections 2024-04-25T16:51:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.499+0000 7fecd28e6640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fecb806f400 0x7fecb8071910 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.499+0000 7fecd28e6640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feccc1f9420 0x7feccc143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.499+0000 7fecd28e6640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feccc15b200 0x7feccc143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.499+0000 7fecd28e6640 1 --2- 172.21.15.94:0/1237252389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb40039c0 0x7feccc13a8c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.499+0000 7fecd28e6640 1 -- 172.21.15.94:0/1237252389 >> 172.21.15.94:0/1237252389 conn(0x7feccc071f00 msgr2=0x7feccc0b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.499+0000 7fecd28e6640 1 -- 172.21.15.94:0/1237252389 shutdown_connections 2024-04-25T16:51:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.499+0000 7fecd28e6640 1 -- 172.21.15.94:0/1237252389 wait complete. 2024-04-25T16:51:46.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:51:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.651+0000 7fd700f9b640 1 Processor -- start 2024-04-25T16:51:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.651+0000 7fd700f9b640 1 -- start start 2024-04-25T16:51:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.655+0000 7fd700f9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6fc146ad0 0x7fd6fc0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.655+0000 7fd700f9b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd6fc161d30 0x7fd6fc0b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.655+0000 7fd700f9b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6fc0b7d20 con 0x7fd6fc0b7910 2024-04-25T16:51:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.655+0000 7fd700f9b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd6fc147350 con 0x7fd6fc161d30 2024-04-25T16:51:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.655+0000 7fd700f9b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd6fc146900 con 0x7fd6fc146ad0 2024-04-25T16:51:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.655+0000 7fd6f3fff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd6fc161d30 0x7fd6fc0b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.655+0000 7fd6fa59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6fc146ad0 0x7fd6fc0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.655+0000 7fd6fad9d640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd6fc0b7910 0x7fd6fc071a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:40840/0 (socket says 172.21.15.94:40840) 2024-04-25T16:51:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.655+0000 7fd6fad9d640 1 -- 172.21.15.94:0/866737728 learned_addr learned my addr 172.21.15.94:0/866737728 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:46.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.655+0000 7fd6f3fff640 1 -- 172.21.15.94:0/866737728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6fc146ad0 msgr2=0x7fd6fc0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:46.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.655+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/866737728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6fc146ad0 0x7fd6fc0cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.655+0000 7fd6f3fff640 1 -- 172.21.15.94:0/866737728 >> v1:172.21.15.138:6789/0 conn(0x7fd6fc0b7910 legacy=0x7fd6fc071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.655+0000 7fd6f3fff640 1 -- 172.21.15.94:0/866737728 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6fc0d0920 con 0x7fd6fc161d30 2024-04-25T16:51:46.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.659+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/866737728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc161d30 0x7fd6fc0b73d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40097c0 tx=0x7fd6e401c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=689c01dd120e546e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.659+0000 7fd6f9d9b640 1 -- 172.21.15.94:0/866737728 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e40b0070 con 0x7fd6fc161d30 2024-04-25T16:51:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.659+0000 7fd6f9d9b640 1 -- 172.21.15.94:0/866737728 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e40ab600 con 0x7fd6fc161d30 2024-04-25T16:51:46.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.659+0000 7fd6f9d9b640 1 -- 172.21.15.94:0/866737728 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e40ab9d0 con 0x7fd6fc161d30 2024-04-25T16:51:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.659+0000 7fd700f9b640 1 -- 172.21.15.94:0/866737728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc161d30 msgr2=0x7fd6fc0b73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.659+0000 7fd700f9b640 1 --2- 172.21.15.94:0/866737728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc161d30 0x7fd6fc0b73d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40097c0 tx=0x7fd6e401c350 comp rx=0 tx=0).stop 2024-04-25T16:51:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.659+0000 7fd700f9b640 1 -- 172.21.15.94:0/866737728 shutdown_connections 2024-04-25T16:51:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.659+0000 7fd700f9b640 1 --2- 172.21.15.94:0/866737728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc161d30 0x7fd6fc0b73d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.659+0000 7fd700f9b640 1 --2- 172.21.15.94:0/866737728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6fc146ad0 0x7fd6fc0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.659+0000 7fd700f9b640 1 -- 172.21.15.94:0/866737728 >> 172.21.15.94:0/866737728 conn(0x7fd6fc0d33d0 msgr2=0x7fd6fc0d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:46.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.659+0000 7fd700f9b640 1 -- 172.21.15.94:0/866737728 shutdown_connections 2024-04-25T16:51:46.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.659+0000 7fd700f9b640 1 -- 172.21.15.94:0/866737728 wait complete. 2024-04-25T16:51:46.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd700f9b640 1 Processor -- start 2024-04-25T16:51:46.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd700f9b640 1 -- start start 2024-04-25T16:51:46.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd700f9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc0b7910 0x7fd6fc142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd700f9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc146ad0 0x7fd6fc13fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd700f9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6fc161d30 0x7fd6fc141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd700f9b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc0d0ee0 con 0x7fd6fc0b7910 2024-04-25T16:51:46.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd700f9b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc161810 con 0x7fd6fc146ad0 2024-04-25T16:51:46.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd700f9b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd6fc162140 con 0x7fd6fc161d30 2024-04-25T16:51:46.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6fa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc0b7910 0x7fd6fc142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6fa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc0b7910 0x7fd6fc142f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48940/0 (socket says 172.21.15.94:48940) 2024-04-25T16:51:46.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6fa59c640 1 -- 172.21.15.94:0/957787293 learned_addr learned my addr 172.21.15.94:0/957787293 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:46.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc146ad0 0x7fd6fc13fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6fad9d640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6fc161d30 0x7fd6fc141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6fa59c640 1 -- 172.21.15.94:0/957787293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6fc161d30 msgr2=0x7fd6fc141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6fa59c640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6fc161d30 0x7fd6fc141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6fa59c640 1 -- 172.21.15.94:0/957787293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc146ad0 msgr2=0x7fd6fc13fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6fa59c640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc146ad0 0x7fd6fc13fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6fa59c640 1 -- 172.21.15.94:0/957787293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6fc143480 con 0x7fd6fc0b7910 2024-04-25T16:51:46.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6fad9d640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6fc161d30 0x7fd6fc141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:46.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6fa59c640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc0b7910 0x7fd6fc142f40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00e8a0 tx=0x7fd6ec00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:46.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6f2ffd640 1 -- 172.21.15.94:0/957787293 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec017070 con 0x7fd6fc0b7910 2024-04-25T16:51:46.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6f2ffd640 1 -- 172.21.15.94:0/957787293 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6ec00c940 con 0x7fd6fc0b7910 2024-04-25T16:51:46.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd700f9b640 1 -- 172.21.15.94:0/957787293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6fc144560 con 0x7fd6fc0b7910 2024-04-25T16:51:46.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd6f2ffd640 1 -- 172.21.15.94:0/957787293 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00ccf0 con 0x7fd6fc0b7910 2024-04-25T16:51:46.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.663+0000 7fd700f9b640 1 -- 172.21.15.94:0/957787293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6fc163570 con 0x7fd6fc0b7910 2024-04-25T16:51:46.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.667+0000 7fd700f9b640 1 -- 172.21.15.94:0/957787293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6fc076830 con 0x7fd6fc0b7910 2024-04-25T16:51:46.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.667+0000 7fd6f2ffd640 1 -- 172.21.15.94:0/957787293 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd6ec01a070 con 0x7fd6fc0b7910 2024-04-25T16:51:46.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.667+0000 7fd6f2ffd640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd6dc06f430 0x7fd6dc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.667+0000 7fd6f2ffd640 1 -- 172.21.15.94:0/957787293 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd6ec08f660 con 0x7fd6fc0b7910 2024-04-25T16:51:46.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.667+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd6dc06f430 0x7fd6dc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.667+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd6dc06f430 0x7fd6dc071940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40ad000 tx=0x7fd6e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:46.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.671+0000 7fd6f2ffd640 1 -- 172.21.15.94:0/957787293 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd6ec05c0b0 con 0x7fd6fc0b7910 2024-04-25T16:51:46.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.819+0000 7fd700f9b640 1 -- 172.21.15.94:0/957787293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd6fc0cfcb0 con 0x7fd6fc0b7910 2024-04-25T16:51:46.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.823+0000 7fd6f2ffd640 1 -- 172.21.15.94:0/957787293 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fd6ec05b9b0 con 0x7fd6fc0b7910 2024-04-25T16:51:46.820 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:51:46.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.823+0000 7fd700f9b640 1 -- 172.21.15.94:0/957787293 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd6dc06f430 msgr2=0x7fd6dc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.823+0000 7fd700f9b640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd6dc06f430 0x7fd6dc071940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40ad000 tx=0x7fd6e4008040 comp rx=0 tx=0).stop 2024-04-25T16:51:46.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.823+0000 7fd700f9b640 1 -- 172.21.15.94:0/957787293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc0b7910 msgr2=0x7fd6fc142f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.823+0000 7fd700f9b640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc0b7910 0x7fd6fc142f40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00e8a0 tx=0x7fd6ec00eca0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.823+0000 7fd700f9b640 1 -- 172.21.15.94:0/957787293 shutdown_connections 2024-04-25T16:51:46.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.823+0000 7fd700f9b640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd6dc06f430 0x7fd6dc071940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.823+0000 7fd700f9b640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6fc161d30 0x7fd6fc141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.823+0000 7fd700f9b640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc146ad0 0x7fd6fc13fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.823+0000 7fd700f9b640 1 --2- 172.21.15.94:0/957787293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc0b7910 0x7fd6fc142f40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.823+0000 7fd700f9b640 1 -- 172.21.15.94:0/957787293 >> 172.21.15.94:0/957787293 conn(0x7fd6fc0d33d0 msgr2=0x7fd6fc072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:46.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.823+0000 7fd700f9b640 1 -- 172.21.15.94:0/957787293 shutdown_connections 2024-04-25T16:51:46.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.823+0000 7fd700f9b640 1 -- 172.21.15.94:0/957787293 wait complete. 2024-04-25T16:51:46.840 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:51:46.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.995+0000 7f66936d6640 1 Processor -- start 2024-04-25T16:51:46.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.995+0000 7f66936d6640 1 -- start start 2024-04-25T16:51:46.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.995+0000 7f66936d6640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f668c0b0420 0x7f668c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.995+0000 7f66936d6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f668c15ec30 0x7f668c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:46.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.995+0000 7f66936d6640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f668c070630 con 0x7f668c06f2a0 2024-04-25T16:51:46.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.995+0000 7f66936d6640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f668c0aeaa0 con 0x7f668c0b0420 2024-04-25T16:51:46.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.995+0000 7f66936d6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f668c0703e0 con 0x7f668c15ec30 2024-04-25T16:51:46.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.995+0000 7f6691472640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f668c0b0420 0x7f668c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.995+0000 7f6691472640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f668c0b0420 0x7f668c06ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48950/0 (socket says 172.21.15.94:48950) 2024-04-25T16:51:46.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.995+0000 7f6691472640 1 -- 172.21.15.94:0/3289653689 learned_addr learned my addr 172.21.15.94:0/3289653689 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:46.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f6691c73640 1 --2- 172.21.15.94:0/3289653689 >> v2:172.21.15.138:3300/0 conn(0x7f668c15ec30 0x7f668c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:46.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f6691472640 1 -- 172.21.15.94:0/3289653689 >> v2:172.21.15.138:3300/0 conn(0x7f668c15ec30 msgr2=0x7f668c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:46.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f6691472640 1 --2- 172.21.15.94:0/3289653689 >> v2:172.21.15.138:3300/0 conn(0x7f668c15ec30 0x7f668c06cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f6691472640 1 -- 172.21.15.94:0/3289653689 >> v1:172.21.15.94:6789/0 conn(0x7f668c06f2a0 legacy=0x7f668c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f6683fff640 1 -- 172.21.15.94:0/3289653689 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 795132530 0 0) 0x7f668c070630 con 0x7f668c06f2a0 2024-04-25T16:51:46.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f6691472640 1 -- 172.21.15.94:0/3289653689 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c06f6b0 con 0x7f668c0b0420 2024-04-25T16:51:46.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f6691472640 1 --2- 172.21.15.94:0/3289653689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f668c0b0420 0x7f668c06ed60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f668c0726d0 tx=0x7f667c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f280d858081a7e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:46.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f6683fff640 1 -- 172.21.15.94:0/3289653689 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c0aa650 con 0x7f668c0b0420 2024-04-25T16:51:46.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f6683fff640 1 -- 172.21.15.94:0/3289653689 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f667c0af070 con 0x7f668c0b0420 2024-04-25T16:51:46.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f6683fff640 1 -- 172.21.15.94:0/3289653689 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c0aaa10 con 0x7f668c0b0420 2024-04-25T16:51:46.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f66936d6640 1 -- 172.21.15.94:0/3289653689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f668c0b0420 msgr2=0x7f668c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:46.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f66936d6640 1 --2- 172.21.15.94:0/3289653689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f668c0b0420 0x7f668c06ed60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f668c0726d0 tx=0x7f667c01c4f0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f66936d6640 1 -- 172.21.15.94:0/3289653689 shutdown_connections 2024-04-25T16:51:46.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f66936d6640 1 --2- 172.21.15.94:0/3289653689 >> v2:172.21.15.138:3300/0 conn(0x7f668c15ec30 0x7f668c06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f66936d6640 1 --2- 172.21.15.94:0/3289653689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f668c0b0420 0x7f668c06ed60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:46.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f66936d6640 1 -- 172.21.15.94:0/3289653689 >> 172.21.15.94:0/3289653689 conn(0x7f668c0b47f0 msgr2=0x7f668c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:47.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f66936d6640 1 -- 172.21.15.94:0/3289653689 shutdown_connections 2024-04-25T16:51:47.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:46.999+0000 7f66936d6640 1 -- 172.21.15.94:0/3289653689 wait complete. 2024-04-25T16:51:47.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f66936d6640 1 Processor -- start 2024-04-25T16:51:47.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f66936d6640 1 -- start start 2024-04-25T16:51:47.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f66936d6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f668c06f2a0 0x7f668c09de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f66936d6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c0b0420 0x7f668c1fa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f66936d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f668c15ec30 0x7f668c1543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f66936d6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f668c0b0830 con 0x7f668c15ec30 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f66936d6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f668c0aef30 con 0x7f668c06f2a0 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f66936d6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f668c0b09b0 con 0x7f668c0b0420 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6691c73640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f668c15ec30 0x7f668c1543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6691c73640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f668c15ec30 0x7f668c1543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48964/0 (socket says 172.21.15.94:48964) 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6691c73640 1 -- 172.21.15.94:0/2690463736 learned_addr learned my addr 172.21.15.94:0/2690463736 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6691472640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f668c06f2a0 0x7f668c09de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6690c71640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c0b0420 0x7f668c1fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6691c73640 1 -- 172.21.15.94:0/2690463736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c0b0420 msgr2=0x7f668c1fa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6691c73640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c0b0420 0x7f668c1fa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6691c73640 1 -- 172.21.15.94:0/2690463736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f668c06f2a0 msgr2=0x7f668c09de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6691c73640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f668c06f2a0 0x7f668c09de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6691c73640 1 -- 172.21.15.94:0/2690463736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c1fa880 con 0x7f668c15ec30 2024-04-25T16:51:47.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6691c73640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f668c15ec30 0x7f668c1543f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6684007e60 tx=0x7f668400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:47.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6681ffb640 1 -- 172.21.15.94:0/2690463736 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684014420 con 0x7f668c15ec30 2024-04-25T16:51:47.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6681ffb640 1 -- 172.21.15.94:0/2690463736 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6684019070 con 0x7f668c15ec30 2024-04-25T16:51:47.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f66936d6640 1 -- 172.21.15.94:0/2690463736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c154930 con 0x7f668c15ec30 2024-04-25T16:51:47.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f6681ffb640 1 -- 172.21.15.94:0/2690463736 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684014820 con 0x7f668c15ec30 2024-04-25T16:51:47.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.003+0000 7f66936d6640 1 -- 172.21.15.94:0/2690463736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c09d0a0 con 0x7f668c15ec30 2024-04-25T16:51:47.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.007+0000 7f66936d6640 1 -- 172.21.15.94:0/2690463736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668c076830 con 0x7f668c15ec30 2024-04-25T16:51:47.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.007+0000 7f6681ffb640 1 -- 172.21.15.94:0/2690463736 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f66840149c0 con 0x7f668c15ec30 2024-04-25T16:51:47.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.007+0000 7f6681ffb640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f667006f430 0x7f6670071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.007+0000 7f6681ffb640 1 -- 172.21.15.94:0/2690463736 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f668408fe80 con 0x7f668c15ec30 2024-04-25T16:51:47.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.007+0000 7f6691472640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f667006f430 0x7f6670071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.007+0000 7f6691472640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f667006f430 0x7f6670071940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f667c0ae000 tx=0x7f667c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:47.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.011+0000 7f6681ffb640 1 -- 172.21.15.94:0/2690463736 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f668c1fa880 con 0x7f668c15ec30 2024-04-25T16:51:47.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f66936d6640 1 -- 172.21.15.94:0/2690463736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f668c06fb20 con 0x7f668c15ec30 2024-04-25T16:51:47.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f6681ffb640 1 -- 172.21.15.94:0/2690463736 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f668405ca00 con 0x7f668c15ec30 2024-04-25T16:51:47.165 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:47.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f66936d6640 1 -- 172.21.15.94:0/2690463736 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f667006f430 msgr2=0x7f6670071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f66936d6640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f667006f430 0x7f6670071940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f667c0ae000 tx=0x7f667c0a7000 comp rx=0 tx=0).stop 2024-04-25T16:51:47.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f66936d6640 1 -- 172.21.15.94:0/2690463736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f668c15ec30 msgr2=0x7f668c1543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f66936d6640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f668c15ec30 0x7f668c1543f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6684007e60 tx=0x7f668400e4b0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f66936d6640 1 -- 172.21.15.94:0/2690463736 shutdown_connections 2024-04-25T16:51:47.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f66936d6640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f668c15ec30 0x7f668c1543f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f66936d6640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f667006f430 0x7f6670071940 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f66936d6640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c0b0420 0x7f668c1fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f66936d6640 1 --2- 172.21.15.94:0/2690463736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f668c06f2a0 0x7f668c09de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f66936d6640 1 -- 172.21.15.94:0/2690463736 >> 172.21.15.94:0/2690463736 conn(0x7f668c0b47f0 msgr2=0x7f668c160950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:47.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f66936d6640 1 -- 172.21.15.94:0/2690463736 shutdown_connections 2024-04-25T16:51:47.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.167+0000 7f66936d6640 1 -- 172.21.15.94:0/2690463736 wait complete. 2024-04-25T16:51:47.186 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:47.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.327+0000 7ff507f82640 1 Processor -- start 2024-04-25T16:51:47.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.327+0000 7ff507f82640 1 -- start start 2024-04-25T16:51:47.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.327+0000 7ff507f82640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff50015e990 0x7ff500165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.327+0000 7ff507f82640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff50006d610 con 0x7ff500165b90 2024-04-25T16:51:47.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.327+0000 7ff507f82640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff50006fc60 con 0x7ff500070080 2024-04-25T16:51:47.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.327+0000 7ff507f82640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff500158550 con 0x7ff50015e990 2024-04-25T16:51:47.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.327+0000 7ff50651f640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ff500165b90 0x7ff500167fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50404/0 (socket says 172.21.15.94:50404) 2024-04-25T16:51:47.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.327+0000 7ff50651f640 1 -- 172.21.15.94:0/2668760589 learned_addr learned my addr 172.21.15.94:0/2668760589 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.327+0000 7ff50551d640 1 --2- 172.21.15.94:0/2668760589 >> v2:172.21.15.138:3300/0 conn(0x7ff50015e990 0x7ff500165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3172841135 0 0) 0x7ff4f0001120 con 0x7ff500165b90 2024-04-25T16:51:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4ec003530 con 0x7ff500165b90 2024-04-25T16:51:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 498788017 0 0) 0x7ff50006fc60 con 0x7ff500070080 2024-04-25T16:51:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4f0001120 con 0x7ff500070080 2024-04-25T16:51:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3431174410 0 0) 0x7ff4ec003530 con 0x7ff500165b90 2024-04-25T16:51:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 >> v2:172.21.15.138:3300/0 conn(0x7ff50015e990 msgr2=0x7ff500165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 --2- 172.21.15.94:0/2668760589 >> v2:172.21.15.138:3300/0 conn(0x7ff50015e990 0x7ff500165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 >> v1:172.21.15.138:6789/0 conn(0x7ff500070080 legacy=0x7ff50015e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5000aef70 con 0x7ff500165b90 2024-04-25T16:51:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7ff4f0001850 con 0x7ff500165b90 2024-04-25T16:51:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 >> v1:172.21.15.94:6789/0 conn(0x7ff500165b90 legacy=0x7ff500167fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 --2- 172.21.15.94:0/2668760589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4ec003b50 0x7ff4ec003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 --2- 172.21.15.94:0/2668760589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4ec004810 0x7ff4ec004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 --2- 172.21.15.94:0/2668760589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4ec0051d0 0x7ff4ec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff500158550 con 0x7ff4ec004810 2024-04-25T16:51:47.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec00c170 con 0x7ff4ec003b50 2024-04-25T16:51:47.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ec00c370 con 0x7ff4ec0051d0 2024-04-25T16:51:47.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff50551d640 1 --2- 172.21.15.94:0/2668760589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4ec004810 0x7ff4ec004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff505d1e640 1 --2- 172.21.15.94:0/2668760589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4ec003b50 0x7ff4ec003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff50651f640 1 --2- 172.21.15.94:0/2668760589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4ec0051d0 0x7ff4ec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff50551d640 1 -- 172.21.15.94:0/2668760589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4ec0051d0 msgr2=0x7ff4ec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:47.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff50551d640 1 --2- 172.21.15.94:0/2668760589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4ec0051d0 0x7ff4ec00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff50551d640 1 -- 172.21.15.94:0/2668760589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4ec003b50 msgr2=0x7ff4ec003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:47.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff50551d640 1 --2- 172.21.15.94:0/2668760589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4ec003b50 0x7ff4ec003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff50551d640 1 -- 172.21.15.94:0/2668760589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ec003530 con 0x7ff4ec004810 2024-04-25T16:51:47.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff50651f640 1 --2- 172.21.15.94:0/2668760589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4ec0051d0 0x7ff4ec00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:47.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff50651f640 1 -- 172.21.15.94:0/2668760589 reap_dead start 2024-04-25T16:51:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff50551d640 1 --2- 172.21.15.94:0/2668760589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4ec004810 0x7ff4ec004c20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff5000b5070 tx=0x7ff4f801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c798f409771ac479 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f80b2070 con 0x7ff4ec004810 2024-04-25T16:51:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f80a5c30 con 0x7ff4ec004810 2024-04-25T16:51:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff504d1c640 1 -- 172.21.15.94:0/2668760589 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8009c10 con 0x7ff4ec004810 2024-04-25T16:51:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff507f82640 1 -- 172.21.15.94:0/2668760589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4ec004810 msgr2=0x7ff4ec004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.331+0000 7ff507f82640 1 --2- 172.21.15.94:0/2668760589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4ec004810 0x7ff4ec004c20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff5000b5070 tx=0x7ff4f801cc20 comp rx=0 tx=0).stop 2024-04-25T16:51:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff507f82640 1 -- 172.21.15.94:0/2668760589 shutdown_connections 2024-04-25T16:51:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff507f82640 1 --2- 172.21.15.94:0/2668760589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4ec004810 0x7ff4ec004c20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff507f82640 1 -- 172.21.15.94:0/2668760589 >> 172.21.15.94:0/2668760589 conn(0x7ff5000b47f0 msgr2=0x7ff5000b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff507f82640 1 -- 172.21.15.94:0/2668760589 shutdown_connections 2024-04-25T16:51:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff507f82640 1 -- 172.21.15.94:0/2668760589 wait complete. 2024-04-25T16:51:47.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff507f82640 1 Processor -- start 2024-04-25T16:51:47.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff507f82640 1 -- start start 2024-04-25T16:51:47.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff507f82640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4ec004810 0x7ff50009cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff507f82640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff50014fa50 0x7ff50014c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff507f82640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff50014ca10 0x7ff50009de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff507f82640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5000ae3e0 con 0x7ff50014ca10 2024-04-25T16:51:47.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff507f82640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff50006f7d0 con 0x7ff50014fa50 2024-04-25T16:51:47.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff507f82640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff50006f460 con 0x7ff4ec004810 2024-04-25T16:51:47.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff50651f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff50014ca10 0x7ff50009de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff50651f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff50014ca10 0x7ff50009de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48972/0 (socket says 172.21.15.94:48972) 2024-04-25T16:51:47.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff50651f640 1 -- 172.21.15.94:0/3485508767 learned_addr learned my addr 172.21.15.94:0/3485508767 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:47.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff50551d640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff50014fa50 0x7ff50014c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff50651f640 1 -- 172.21.15.94:0/3485508767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4ec004810 msgr2=0x7ff50009cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff505d1e640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4ec004810 0x7ff50009cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff50651f640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4ec004810 0x7ff50009cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff50651f640 1 -- 172.21.15.94:0/3485508767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff50014fa50 msgr2=0x7ff50014c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff50651f640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff50014fa50 0x7ff50014c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff50651f640 1 -- 172.21.15.94:0/3485508767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50009b740 con 0x7ff50014ca10 2024-04-25T16:51:47.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.335+0000 7ff50651f640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff50014ca10 0x7ff50009de10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff500072540 tx=0x7ff4f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:47.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.339+0000 7ff4f67fc640 1 -- 172.21.15.94:0/3485508767 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0007230 con 0x7ff50014ca10 2024-04-25T16:51:47.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.339+0000 7ff4f67fc640 1 -- 172.21.15.94:0/3485508767 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f00073f0 con 0x7ff50014ca10 2024-04-25T16:51:47.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.339+0000 7ff507f82640 1 -- 172.21.15.94:0/3485508767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50009d190 con 0x7ff50014ca10 2024-04-25T16:51:47.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.339+0000 7ff4f67fc640 1 -- 172.21.15.94:0/3485508767 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00047f0 con 0x7ff50014ca10 2024-04-25T16:51:47.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.339+0000 7ff507f82640 1 -- 172.21.15.94:0/3485508767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50009be90 con 0x7ff50014ca10 2024-04-25T16:51:47.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.339+0000 7ff507f82640 1 -- 172.21.15.94:0/3485508767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff50009b910 con 0x7ff50014ca10 2024-04-25T16:51:47.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.339+0000 7ff4f67fc640 1 -- 172.21.15.94:0/3485508767 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff4f001b020 con 0x7ff50014ca10 2024-04-25T16:51:47.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.339+0000 7ff4f67fc640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff4e406f400 0x7ff4e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.339+0000 7ff4f67fc640 1 -- 172.21.15.94:0/3485508767 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff4f008f340 con 0x7ff50014ca10 2024-04-25T16:51:47.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.339+0000 7ff505d1e640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff4e406f400 0x7ff4e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.339+0000 7ff505d1e640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff4e406f400 0x7ff4e4071910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc005630 tx=0x7ff4fc006ff0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:47.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.343+0000 7ff4f67fc640 1 -- 172.21.15.94:0/3485508767 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff50009b910 con 0x7ff50014ca10 2024-04-25T16:51:47.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.495+0000 7ff507f82640 1 -- 172.21.15.94:0/3485508767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff500076830 con 0x7ff50014ca10 2024-04-25T16:51:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.495+0000 7ff4f67fc640 1 -- 172.21.15.94:0/3485508767 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7ff50006d610 con 0x7ff50014ca10 2024-04-25T16:51:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:47.495 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:47.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.499+0000 7ff507f82640 1 -- 172.21.15.94:0/3485508767 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff4e406f400 msgr2=0x7ff4e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.499+0000 7ff507f82640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff4e406f400 0x7ff4e4071910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc005630 tx=0x7ff4fc006ff0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.499+0000 7ff507f82640 1 -- 172.21.15.94:0/3485508767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff50014ca10 msgr2=0x7ff50009de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.499+0000 7ff507f82640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff50014ca10 0x7ff50009de10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff500072540 tx=0x7ff4f000dc00 comp rx=0 tx=0).stop 2024-04-25T16:51:47.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.499+0000 7ff507f82640 1 -- 172.21.15.94:0/3485508767 shutdown_connections 2024-04-25T16:51:47.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.499+0000 7ff507f82640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff50014ca10 0x7ff50009de10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.499+0000 7ff507f82640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff50014fa50 0x7ff50014c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.499+0000 7ff507f82640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff4e406f400 0x7ff4e4071910 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.499+0000 7ff507f82640 1 --2- 172.21.15.94:0/3485508767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4ec004810 0x7ff50009cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.499+0000 7ff507f82640 1 -- 172.21.15.94:0/3485508767 >> 172.21.15.94:0/3485508767 conn(0x7ff5000b47f0 msgr2=0x7ff50006d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:47.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.499+0000 7ff507f82640 1 -- 172.21.15.94:0/3485508767 shutdown_connections 2024-04-25T16:51:47.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.499+0000 7ff507f82640 1 -- 172.21.15.94:0/3485508767 wait complete. 2024-04-25T16:51:47.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:51:47.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:51:47.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd39b2a640 1 Processor -- start 2024-04-25T16:51:47.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd39b2a640 1 -- start start 2024-04-25T16:51:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd39b2a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd340ccfa0 0x7fcd340d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd39b2a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcd34160a90 0x7fcd340b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd39b2a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcd340b7910 0x7fcd34163600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd39b2a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fcd340b7d20 con 0x7fcd340b7910 2024-04-25T16:51:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd39b2a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcd340d1570 con 0x7fcd34160a90 2024-04-25T16:51:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd39b2a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcd340d1850 con 0x7fcd340ccfa0 2024-04-25T16:51:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd33fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcd340b7910 0x7fcd34163600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd33fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcd340b7910 0x7fcd34163600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48978/0 (socket says 172.21.15.94:48978) 2024-04-25T16:51:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd33fff640 1 -- 172.21.15.94:0/694200880 learned_addr learned my addr 172.21.15.94:0/694200880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd32ffd640 1 --2- 172.21.15.94:0/694200880 >> v2:172.21.15.138:3300/0 conn(0x7fcd34160a90 0x7fcd340b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd337fe640 1 --2- 172.21.15.94:0/694200880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd340ccfa0 0x7fcd340d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd33fff640 1 -- 172.21.15.94:0/694200880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd340ccfa0 msgr2=0x7fcd340d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd33fff640 1 --2- 172.21.15.94:0/694200880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd340ccfa0 0x7fcd340d0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd33fff640 1 -- 172.21.15.94:0/694200880 >> v2:172.21.15.138:3300/0 conn(0x7fcd34160a90 msgr2=0x7fcd340b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd33fff640 1 --2- 172.21.15.94:0/694200880 >> v2:172.21.15.138:3300/0 conn(0x7fcd34160a90 0x7fcd340b73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.667+0000 7fcd33fff640 1 -- 172.21.15.94:0/694200880 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd340d1b30 con 0x7fcd340b7910 2024-04-25T16:51:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd33fff640 1 --2- 172.21.15.94:0/694200880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd340b7910 0x7fcd34163600 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcd340d3fa0 tx=0x7fcd1c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe92c943336f8f47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd327fc640 1 -- 172.21.15.94:0/694200880 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0b0070 con 0x7fcd340b7910 2024-04-25T16:51:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd327fc640 1 -- 172.21.15.94:0/694200880 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd1c0ab600 con 0x7fcd340b7910 2024-04-25T16:51:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd327fc640 1 -- 172.21.15.94:0/694200880 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0ab9d0 con 0x7fcd340b7910 2024-04-25T16:51:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd39b2a640 1 -- 172.21.15.94:0/694200880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd340b7910 msgr2=0x7fcd34163600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd39b2a640 1 --2- 172.21.15.94:0/694200880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd340b7910 0x7fcd34163600 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcd340d3fa0 tx=0x7fcd1c01c260 comp rx=0 tx=0).stop 2024-04-25T16:51:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd39b2a640 1 -- 172.21.15.94:0/694200880 shutdown_connections 2024-04-25T16:51:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd39b2a640 1 --2- 172.21.15.94:0/694200880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd340b7910 0x7fcd34163600 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd39b2a640 1 --2- 172.21.15.94:0/694200880 >> v2:172.21.15.138:3300/0 conn(0x7fcd34160a90 0x7fcd340b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd39b2a640 1 --2- 172.21.15.94:0/694200880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd340ccfa0 0x7fcd340d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd39b2a640 1 -- 172.21.15.94:0/694200880 >> 172.21.15.94:0/694200880 conn(0x7fcd340d33d0 msgr2=0x7fcd340d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd39b2a640 1 -- 172.21.15.94:0/694200880 shutdown_connections 2024-04-25T16:51:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd39b2a640 1 -- 172.21.15.94:0/694200880 wait complete. 2024-04-25T16:51:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd39b2a640 1 Processor -- start 2024-04-25T16:51:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.671+0000 7fcd39b2a640 1 -- start start 2024-04-25T16:51:47.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd39b2a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd340b7910 0x7fcd34140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd39b2a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd340ccfa0 0x7fcd340719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd39b2a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34160a90 0x7fcd34156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd39b2a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd340d24b0 con 0x7fcd34160a90 2024-04-25T16:51:47.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd39b2a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd340d2330 con 0x7fcd340b7910 2024-04-25T16:51:47.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd39b2a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcd340d2630 con 0x7fcd340ccfa0 2024-04-25T16:51:47.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd33fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34160a90 0x7fcd34156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd33fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34160a90 0x7fcd34156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48980/0 (socket says 172.21.15.94:48980) 2024-04-25T16:51:47.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd33fff640 1 -- 172.21.15.94:0/716528613 learned_addr learned my addr 172.21.15.94:0/716528613 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:47.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd337fe640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd340b7910 0x7fcd34140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd32ffd640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd340ccfa0 0x7fcd340719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd33fff640 1 -- 172.21.15.94:0/716528613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd340ccfa0 msgr2=0x7fcd340719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd33fff640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd340ccfa0 0x7fcd340719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd33fff640 1 -- 172.21.15.94:0/716528613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd340b7910 msgr2=0x7fcd34140140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd33fff640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd340b7910 0x7fcd34140140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd33fff640 1 -- 172.21.15.94:0/716528613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd34071ef0 con 0x7fcd34160a90 2024-04-25T16:51:47.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd337fe640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd340b7910 0x7fcd34140140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:47.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd33fff640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34160a90 0x7fcd34156200 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0ae040 tx=0x7fcd1c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:47.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd2bfff640 1 -- 172.21.15.94:0/716528613 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0b0040 con 0x7fcd34160a90 2024-04-25T16:51:47.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd2bfff640 1 -- 172.21.15.94:0/716528613 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd1c0b9070 con 0x7fcd34160a90 2024-04-25T16:51:47.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd39b2a640 1 -- 172.21.15.94:0/716528613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd34156c80 con 0x7fcd34160a90 2024-04-25T16:51:47.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd2bfff640 1 -- 172.21.15.94:0/716528613 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0b4640 con 0x7fcd34160a90 2024-04-25T16:51:47.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd39b2a640 1 -- 172.21.15.94:0/716528613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3406e440 con 0x7fcd34160a90 2024-04-25T16:51:47.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd39b2a640 1 -- 172.21.15.94:0/716528613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3406df50 con 0x7fcd34160a90 2024-04-25T16:51:47.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd2bfff640 1 -- 172.21.15.94:0/716528613 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcd1c0b47e0 con 0x7fcd34160a90 2024-04-25T16:51:47.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd2bfff640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcd1406f400 0x7fcd14071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:47.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd2bfff640 1 -- 172.21.15.94:0/716528613 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fcd1c1294d0 con 0x7fcd34160a90 2024-04-25T16:51:47.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd337fe640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcd1406f400 0x7fcd14071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:47.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.675+0000 7fcd337fe640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcd1406f400 0x7fcd14071910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcd340b8740 tx=0x7fcd2c0073a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:47.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.679+0000 7fcd2bfff640 1 -- 172.21.15.94:0/716528613 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcd1c016110 con 0x7fcd34160a90 2024-04-25T16:51:47.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.839+0000 7fcd39b2a640 1 -- 172.21.15.94:0/716528613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd340d12b0 con 0x7fcd34160a90 2024-04-25T16:51:47.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.843+0000 7fcd2bfff640 1 -- 172.21.15.94:0/716528613 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fcd1c0f5fc0 con 0x7fcd34160a90 2024-04-25T16:51:47.840 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:51:47.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.843+0000 7fcd39b2a640 1 -- 172.21.15.94:0/716528613 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcd1406f400 msgr2=0x7fcd14071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.843+0000 7fcd39b2a640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcd1406f400 0x7fcd14071910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcd340b8740 tx=0x7fcd2c0073a0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.843+0000 7fcd39b2a640 1 -- 172.21.15.94:0/716528613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34160a90 msgr2=0x7fcd34156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:47.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.843+0000 7fcd39b2a640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34160a90 0x7fcd34156200 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0ae040 tx=0x7fcd1c019000 comp rx=0 tx=0).stop 2024-04-25T16:51:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.843+0000 7fcd39b2a640 1 -- 172.21.15.94:0/716528613 shutdown_connections 2024-04-25T16:51:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.843+0000 7fcd39b2a640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34160a90 0x7fcd34156200 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.843+0000 7fcd39b2a640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcd1406f400 0x7fcd14071910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.843+0000 7fcd39b2a640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd340ccfa0 0x7fcd340719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.843+0000 7fcd39b2a640 1 --2- 172.21.15.94:0/716528613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd340b7910 0x7fcd34140140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.843+0000 7fcd39b2a640 1 -- 172.21.15.94:0/716528613 >> 172.21.15.94:0/716528613 conn(0x7fcd340d33d0 msgr2=0x7fcd3414ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.843+0000 7fcd39b2a640 1 -- 172.21.15.94:0/716528613 shutdown_connections 2024-04-25T16:51:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:47.843+0000 7fcd39b2a640 1 -- 172.21.15.94:0/716528613 wait complete. 2024-04-25T16:51:47.861 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:51:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efebae65640 1 Processor -- start 2024-04-25T16:51:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efebae65640 1 -- start start 2024-04-25T16:51:48.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efebae65640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efeb415c7e0 0x7efeb40afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efebae65640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efeb40b0330 0x7efeb406d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efebae65640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb4074380 con 0x7efeb406dc50 2024-04-25T16:51:48.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efebae65640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efeb406e060 con 0x7efeb415c7e0 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efebae65640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efeb40727e0 con 0x7efeb40b0330 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb9402640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7efeb406dc50 0x7efeb406e7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49448/0 (socket says 172.21.15.94:49448) 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb9402640 1 -- 172.21.15.94:0/1055388555 learned_addr learned my addr 172.21.15.94:0/1055388555 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb8c01640 1 --2- 172.21.15.94:0/1055388555 >> v2:172.21.15.138:3300/0 conn(0x7efeb415c7e0 0x7efeb40afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb3fff640 1 --2- 172.21.15.94:0/1055388555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efeb40b0330 0x7efeb406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2172529422 0 0) 0x7efeac0014a0 con 0x7efeb406dc50 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe9c003530 con 0x7efeb406dc50 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 891729423 0 0) 0x7efe9c003530 con 0x7efeb406dc50 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efeb40b0330 msgr2=0x7efeb406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 --2- 172.21.15.94:0/1055388555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efeb40b0330 0x7efeb406d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 >> v2:172.21.15.138:3300/0 conn(0x7efeb415c7e0 msgr2=0x7efeb40afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 --2- 172.21.15.94:0/1055388555 >> v2:172.21.15.138:3300/0 conn(0x7efeb415c7e0 0x7efeb40afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb4072ac0 con 0x7efeb406dc50 2024-04-25T16:51:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7efeac001bd0 con 0x7efeb406dc50 2024-04-25T16:51:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 >> v1:172.21.15.94:6789/0 conn(0x7efeb406dc50 legacy=0x7efeb406e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 --2- 172.21.15.94:0/1055388555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe9c0039c0 0x7efe9c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 --2- 172.21.15.94:0/1055388555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe9c004810 0x7efe9c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 --2- 172.21.15.94:0/1055388555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efeb40b0330 0x7efe9c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efeb406e060 con 0x7efe9c004810 2024-04-25T16:51:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efeb40727e0 con 0x7efe9c0039c0 2024-04-25T16:51:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efe9c00ba80 con 0x7efeb40b0330 2024-04-25T16:51:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb3fff640 1 --2- 172.21.15.94:0/1055388555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe9c004810 0x7efe9c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb8c01640 1 --2- 172.21.15.94:0/1055388555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe9c0039c0 0x7efe9c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb9402640 1 --2- 172.21.15.94:0/1055388555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efeb40b0330 0x7efe9c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb3fff640 1 -- 172.21.15.94:0/1055388555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efeb40b0330 msgr2=0x7efe9c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb3fff640 1 --2- 172.21.15.94:0/1055388555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efeb40b0330 0x7efe9c005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb3fff640 1 -- 172.21.15.94:0/1055388555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe9c0039c0 msgr2=0x7efe9c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:48.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb3fff640 1 --2- 172.21.15.94:0/1055388555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe9c0039c0 0x7efe9c003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.015+0000 7efeb3fff640 1 -- 172.21.15.94:0/1055388555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9c003530 con 0x7efe9c004810 2024-04-25T16:51:48.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.019+0000 7efeb3fff640 1 --2- 172.21.15.94:0/1055388555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe9c004810 0x7efe9c004c20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efea8009ab0 tx=0x7efea801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57b949d32cbec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:48.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.019+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea80b2070 con 0x7efe9c004810 2024-04-25T16:51:48.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.019+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea80a5520 con 0x7efe9c004810 2024-04-25T16:51:48.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.019+0000 7efeb37fe640 1 -- 172.21.15.94:0/1055388555 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea80096b0 con 0x7efe9c004810 2024-04-25T16:51:48.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.019+0000 7efebae65640 1 -- 172.21.15.94:0/1055388555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe9c004810 msgr2=0x7efe9c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.019+0000 7efebae65640 1 --2- 172.21.15.94:0/1055388555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe9c004810 0x7efe9c004c20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efea8009ab0 tx=0x7efea801ca40 comp rx=0 tx=0).stop 2024-04-25T16:51:48.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.019+0000 7efeb9402640 1 -- 172.21.15.94:0/1055388555 reap_dead start 2024-04-25T16:51:48.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.019+0000 7efebae65640 1 -- 172.21.15.94:0/1055388555 shutdown_connections 2024-04-25T16:51:48.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.019+0000 7efebae65640 1 -- 172.21.15.94:0/1055388555 >> 172.21.15.94:0/1055388555 conn(0x7efeb40b47f0 msgr2=0x7efeb40b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:48.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.019+0000 7efebae65640 1 -- 172.21.15.94:0/1055388555 shutdown_connections 2024-04-25T16:51:48.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.019+0000 7efebae65640 1 -- 172.21.15.94:0/1055388555 wait complete. 2024-04-25T16:51:48.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efebae65640 1 Processor -- start 2024-04-25T16:51:48.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efebae65640 1 -- start start 2024-04-25T16:51:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efebae65640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe9c004810 0x7efeb415f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efebae65640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb41f8cc0 0x7efeb41568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efebae65640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb406c7d0 0x7efeb406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efebae65640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efeb406f2c0 con 0x7efeb41f8cc0 2024-04-25T16:51:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efebae65640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4073ca0 con 0x7efeb406c7d0 2024-04-25T16:51:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efebae65640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efeb415cbf0 con 0x7efe9c004810 2024-04-25T16:51:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efeb3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb41f8cc0 0x7efeb41568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efeb3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb41f8cc0 0x7efeb41568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59526/0 (socket says 172.21.15.94:59526) 2024-04-25T16:51:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efeb3fff640 1 -- 172.21.15.94:0/1108313831 learned_addr learned my addr 172.21.15.94:0/1108313831 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efeb3fff640 1 -- 172.21.15.94:0/1108313831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe9c004810 msgr2=0x7efeb415f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efeb9402640 1 --2- 172.21.15.94:0/1108313831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb406c7d0 0x7efeb406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efeb3fff640 1 --2- 172.21.15.94:0/1108313831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe9c004810 0x7efeb415f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efeb3fff640 1 -- 172.21.15.94:0/1108313831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb406c7d0 msgr2=0x7efeb406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efeb3fff640 1 --2- 172.21.15.94:0/1108313831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb406c7d0 0x7efeb406cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efeb3fff640 1 -- 172.21.15.94:0/1108313831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb415fd80 con 0x7efeb41f8cc0 2024-04-25T16:51:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efeb3fff640 1 --2- 172.21.15.94:0/1108313831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb41f8cc0 0x7efeb41568f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efea8000c10 tx=0x7efe9c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efeb17fa640 1 -- 172.21.15.94:0/1108313831 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea80b2040 con 0x7efeb41f8cc0 2024-04-25T16:51:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efeb17fa640 1 -- 172.21.15.94:0/1108313831 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea80a5cd0 con 0x7efeb41f8cc0 2024-04-25T16:51:48.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efebae65640 1 -- 172.21.15.94:0/1108313831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb41f9740 con 0x7efeb41f8cc0 2024-04-25T16:51:48.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efeb17fa640 1 -- 172.21.15.94:0/1108313831 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8004250 con 0x7efeb41f8cc0 2024-04-25T16:51:48.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efebae65640 1 -- 172.21.15.94:0/1108313831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb415f040 con 0x7efeb41f8cc0 2024-04-25T16:51:48.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.023+0000 7efebae65640 1 -- 172.21.15.94:0/1108313831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb41f94b0 con 0x7efeb41f8cc0 2024-04-25T16:51:48.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.027+0000 7efeb17fa640 1 -- 172.21.15.94:0/1108313831 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efea8003620 con 0x7efeb41f8cc0 2024-04-25T16:51:48.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.027+0000 7efeb17fa640 1 --2- 172.21.15.94:0/1108313831 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7efea006f2e0 0x7efea00717f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.027+0000 7efeb17fa640 1 -- 172.21.15.94:0/1108313831 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7efea812a8a0 con 0x7efeb41f8cc0 2024-04-25T16:51:48.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.027+0000 7efeb8c01640 1 --2- 172.21.15.94:0/1108313831 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7efea006f2e0 0x7efea00717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.027+0000 7efeb8c01640 1 --2- 172.21.15.94:0/1108313831 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7efea006f2e0 0x7efea00717f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efea40099f0 tx=0x7efea4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.031+0000 7efeb17fa640 1 -- 172.21.15.94:0/1108313831 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efea8016780 con 0x7efeb41f8cc0 2024-04-25T16:51:48.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.179+0000 7efebae65640 1 -- 172.21.15.94:0/1108313831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efeb4076830 con 0x7efeb41f8cc0 2024-04-25T16:51:48.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.179+0000 7efeb17fa640 1 -- 172.21.15.94:0/1108313831 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7efea80f76b0 con 0x7efeb41f8cc0 2024-04-25T16:51:48.178 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.183+0000 7efebae65640 1 -- 172.21.15.94:0/1108313831 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7efea006f2e0 msgr2=0x7efea00717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.183+0000 7efebae65640 1 --2- 172.21.15.94:0/1108313831 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7efea006f2e0 0x7efea00717f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efea40099f0 tx=0x7efea4008040 comp rx=0 tx=0).stop 2024-04-25T16:51:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.183+0000 7efebae65640 1 -- 172.21.15.94:0/1108313831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb41f8cc0 msgr2=0x7efeb41568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.183+0000 7efebae65640 1 --2- 172.21.15.94:0/1108313831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb41f8cc0 0x7efeb41568f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efea8000c10 tx=0x7efe9c005840 comp rx=0 tx=0).stop 2024-04-25T16:51:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.183+0000 7efebae65640 1 -- 172.21.15.94:0/1108313831 shutdown_connections 2024-04-25T16:51:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.183+0000 7efebae65640 1 --2- 172.21.15.94:0/1108313831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb406c7d0 0x7efeb406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.183+0000 7efebae65640 1 --2- 172.21.15.94:0/1108313831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb41f8cc0 0x7efeb41568f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.183+0000 7efebae65640 1 --2- 172.21.15.94:0/1108313831 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7efea006f2e0 0x7efea00717f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.183+0000 7efebae65640 1 --2- 172.21.15.94:0/1108313831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe9c004810 0x7efeb415f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.183+0000 7efebae65640 1 -- 172.21.15.94:0/1108313831 >> 172.21.15.94:0/1108313831 conn(0x7efeb40b47f0 msgr2=0x7efeb40d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.183+0000 7efebae65640 1 -- 172.21.15.94:0/1108313831 shutdown_connections 2024-04-25T16:51:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.183+0000 7efebae65640 1 -- 172.21.15.94:0/1108313831 wait complete. 2024-04-25T16:51:48.198 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:51:48.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.343+0000 7facf5e90640 1 Processor -- start 2024-04-25T16:51:48.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facf5e90640 1 -- start start 2024-04-25T16:51:48.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facf5e90640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7facf00b0420 0x7facf006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facf5e90640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf0070630 con 0x7facf015ec30 2024-04-25T16:51:48.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facf5e90640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf006f6b0 con 0x7facf006f2a0 2024-04-25T16:51:48.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facf5e90640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7facf00aeaa0 con 0x7facf00b0420 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7faceffff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7facf015ec30 0x7facf006cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49450/0 (socket says 172.21.15.94:49450) 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7faceffff640 1 -- 172.21.15.94:0/3963605647 learned_addr learned my addr 172.21.15.94:0/3963605647 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facef7fe640 1 --2- 172.21.15.94:0/3963605647 >> v2:172.21.15.138:3300/0 conn(0x7facf00b0420 0x7facf006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2183923400 0 0) 0x7facf0070630 con 0x7facf015ec30 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facd8003530 con 0x7facf015ec30 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2736840555 0 0) 0x7facd8003530 con 0x7facf015ec30 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 >> v2:172.21.15.138:3300/0 conn(0x7facf00b0420 msgr2=0x7facf006ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 --2- 172.21.15.94:0/3963605647 >> v2:172.21.15.138:3300/0 conn(0x7facf00b0420 0x7facf006ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 >> v1:172.21.15.138:6789/0 conn(0x7facf006f2a0 legacy=0x7facf006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf006d420 con 0x7facf015ec30 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7face8001a00 con 0x7facf015ec30 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 >> v1:172.21.15.94:6789/0 conn(0x7facf015ec30 legacy=0x7facf006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 --2- 172.21.15.94:0/3963605647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facd80039c0 0x7facd8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 --2- 172.21.15.94:0/3963605647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd8004810 0x7facd8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 --2- 172.21.15.94:0/3963605647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd80051d0 0x7facd800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7facf00aeaa0 con 0x7facd80039c0 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7facd800c170 con 0x7facd80051d0 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7facd800c370 con 0x7facd8004810 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facef7fe640 1 --2- 172.21.15.94:0/3963605647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facd80039c0 0x7facd8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7faceffff640 1 --2- 172.21.15.94:0/3963605647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd80051d0 0x7facd800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7faceeffd640 1 --2- 172.21.15.94:0/3963605647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd8004810 0x7facd8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facef7fe640 1 -- 172.21.15.94:0/3963605647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd8004810 msgr2=0x7facd8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facef7fe640 1 --2- 172.21.15.94:0/3963605647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd8004810 0x7facd8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facef7fe640 1 -- 172.21.15.94:0/3963605647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd80051d0 msgr2=0x7facd800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facef7fe640 1 --2- 172.21.15.94:0/3963605647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd80051d0 0x7facd800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7faceffff640 1 -- 172.21.15.94:0/3963605647 reap_dead start 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7facef7fe640 1 -- 172.21.15.94:0/3963605647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd8003530 con 0x7facd80039c0 2024-04-25T16:51:48.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.347+0000 7faceeffd640 1 --2- 172.21.15.94:0/3963605647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd8004810 0x7facd8004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:48.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.351+0000 7facef7fe640 1 --2- 172.21.15.94:0/3963605647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facd80039c0 0x7facd8003e30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7face0009ab0 tx=0x7face001c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c41c7dad44b4221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:48.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.351+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face00096b0 con 0x7facd80039c0 2024-04-25T16:51:48.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.351+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face00a6460 con 0x7facd80039c0 2024-04-25T16:51:48.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.351+0000 7facee7fc640 1 -- 172.21.15.94:0/3963605647 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face00a6850 con 0x7facd80039c0 2024-04-25T16:51:48.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.351+0000 7facf5e90640 1 -- 172.21.15.94:0/3963605647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facd80039c0 msgr2=0x7facd8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.351+0000 7facf5e90640 1 --2- 172.21.15.94:0/3963605647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facd80039c0 0x7facd8003e30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7face0009ab0 tx=0x7face001c990 comp rx=0 tx=0).stop 2024-04-25T16:51:48.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.351+0000 7facf5e90640 1 -- 172.21.15.94:0/3963605647 shutdown_connections 2024-04-25T16:51:48.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.351+0000 7facf5e90640 1 --2- 172.21.15.94:0/3963605647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facd80039c0 0x7facd8003e30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.351+0000 7facf5e90640 1 -- 172.21.15.94:0/3963605647 >> 172.21.15.94:0/3963605647 conn(0x7facf00b47f0 msgr2=0x7facf00b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:48.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.351+0000 7facf5e90640 1 -- 172.21.15.94:0/3963605647 shutdown_connections 2024-04-25T16:51:48.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.351+0000 7facf5e90640 1 -- 172.21.15.94:0/3963605647 wait complete. 2024-04-25T16:51:48.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facf5e90640 1 Processor -- start 2024-04-25T16:51:48.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facf5e90640 1 -- start start 2024-04-25T16:51:48.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facf5e90640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd80039c0 0x7facf0146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facf5e90640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf01fd160 0x7facf0155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facf5e90640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facf0155960 0x7facf0157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facf5e90640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7facf00b0830 con 0x7facf0155960 2024-04-25T16:51:48.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facf5e90640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7facf00aef30 con 0x7facd80039c0 2024-04-25T16:51:48.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facf5e90640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7facf00b09b0 con 0x7facf01fd160 2024-04-25T16:51:48.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7faceffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facf0155960 0x7facf0157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7faceffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facf0155960 0x7facf0157a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59554/0 (socket says 172.21.15.94:59554) 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7faceffff640 1 -- 172.21.15.94:0/1430762879 learned_addr learned my addr 172.21.15.94:0/1430762879 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facef7fe640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd80039c0 0x7facf0146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7faceeffd640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf01fd160 0x7facf0155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7faceffff640 1 -- 172.21.15.94:0/1430762879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf01fd160 msgr2=0x7facf0155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7faceffff640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf01fd160 0x7facf0155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7faceffff640 1 -- 172.21.15.94:0/1430762879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd80039c0 msgr2=0x7facf0146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7faceffff640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd80039c0 0x7facf0146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7faceffff640 1 -- 172.21.15.94:0/1430762879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf0155d70 con 0x7facf0155960 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7faceeffd640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf01fd160 0x7facf0155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7faceffff640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facf0155960 0x7facf0157a90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7face8000f90 tx=0x7face8007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facd7fff640 1 -- 172.21.15.94:0/1430762879 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face800eaa0 con 0x7facf0155960 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facd7fff640 1 -- 172.21.15.94:0/1430762879 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face800ec60 con 0x7facf0155960 2024-04-25T16:51:48.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facd7fff640 1 -- 172.21.15.94:0/1430762879 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8014430 con 0x7facf0155960 2024-04-25T16:51:48.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facf5e90640 1 -- 172.21.15.94:0/1430762879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf01fd700 con 0x7facf0155960 2024-04-25T16:51:48.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facf5e90640 1 -- 172.21.15.94:0/1430762879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf01fdcc0 con 0x7facf0155960 2024-04-25T16:51:48.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.355+0000 7facf5e90640 1 -- 172.21.15.94:0/1430762879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf0076830 con 0x7facf0155960 2024-04-25T16:51:48.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.359+0000 7facd7fff640 1 -- 172.21.15.94:0/1430762879 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7face80145d0 con 0x7facf0155960 2024-04-25T16:51:48.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.359+0000 7facd7fff640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7facdc06f400 0x7facdc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.359+0000 7facd7fff640 1 -- 172.21.15.94:0/1430762879 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7face808f330 con 0x7facf0155960 2024-04-25T16:51:48.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.359+0000 7facef7fe640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7facdc06f400 0x7facdc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.359+0000 7facef7fe640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7facdc06f400 0x7facdc071910 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7face00ad000 tx=0x7face00046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:48.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.363+0000 7facd7fff640 1 -- 172.21.15.94:0/1430762879 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7facf0155d70 con 0x7facf0155960 2024-04-25T16:51:48.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.523+0000 7facf5e90640 1 -- 172.21.15.94:0/1430762879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7facf006fb20 con 0x7facf0155960 2024-04-25T16:51:48.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.523+0000 7facd7fff640 1 -- 172.21.15.94:0/1430762879 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7face805bf40 con 0x7facf0155960 2024-04-25T16:51:48.523 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:51:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.527+0000 7facf5e90640 1 -- 172.21.15.94:0/1430762879 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7facdc06f400 msgr2=0x7facdc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.527+0000 7facf5e90640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7facdc06f400 0x7facdc071910 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7face00ad000 tx=0x7face00046d0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.527+0000 7facf5e90640 1 -- 172.21.15.94:0/1430762879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facf0155960 msgr2=0x7facf0157a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.527+0000 7facf5e90640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facf0155960 0x7facf0157a90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7face8000f90 tx=0x7face8007b70 comp rx=0 tx=0).stop 2024-04-25T16:51:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.527+0000 7facf5e90640 1 -- 172.21.15.94:0/1430762879 shutdown_connections 2024-04-25T16:51:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.527+0000 7facf5e90640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facf0155960 0x7facf0157a90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.527+0000 7facf5e90640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7facdc06f400 0x7facdc071910 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.527+0000 7facf5e90640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf01fd160 0x7facf0155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.527+0000 7facf5e90640 1 --2- 172.21.15.94:0/1430762879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd80039c0 0x7facf0146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.527+0000 7facf5e90640 1 -- 172.21.15.94:0/1430762879 >> 172.21.15.94:0/1430762879 conn(0x7facf00b47f0 msgr2=0x7facf01515e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:48.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.527+0000 7facf5e90640 1 -- 172.21.15.94:0/1430762879 shutdown_connections 2024-04-25T16:51:48.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.527+0000 7facf5e90640 1 -- 172.21.15.94:0/1430762879 wait complete. 2024-04-25T16:51:48.544 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:51:48.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f8116c3b640 1 Processor -- start 2024-04-25T16:51:48.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f8116c3b640 1 -- start start 2024-04-25T16:51:48.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f8116c3b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8110146ad0 0x7f81100cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f8116c3b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8110161d30 0x7f81100b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f8116c3b640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81100b7d20 con 0x7f81100b7910 2024-04-25T16:51:48.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f8116c3b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f8110147350 con 0x7f8110146ad0 2024-04-25T16:51:48.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f8116c3b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8110146900 con 0x7f8110161d30 2024-04-25T16:51:48.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f81149d7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8110146ad0 0x7f81100cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f81149d7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8110146ad0 0x7f81100cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59562/0 (socket says 172.21.15.94:59562) 2024-04-25T16:51:48.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f81149d7640 1 -- 172.21.15.94:0/3486584204 learned_addr learned my addr 172.21.15.94:0/3486584204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:48.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f810ffff640 1 --2- 172.21.15.94:0/3486584204 >> v2:172.21.15.138:3300/0 conn(0x7f8110161d30 0x7f81100b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f81149d7640 1 -- 172.21.15.94:0/3486584204 >> v2:172.21.15.138:3300/0 conn(0x7f8110161d30 msgr2=0x7f81100b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:48.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f81149d7640 1 --2- 172.21.15.94:0/3486584204 >> v2:172.21.15.138:3300/0 conn(0x7f8110161d30 0x7f81100b73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f81149d7640 1 -- 172.21.15.94:0/3486584204 >> v1:172.21.15.94:6789/0 conn(0x7f81100b7910 legacy=0x7f8110071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f81149d7640 1 -- 172.21.15.94:0/3486584204 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81100d0920 con 0x7f8110146ad0 2024-04-25T16:51:48.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f810f7fe640 1 -- 172.21.15.94:0/3486584204 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2075422276 0 0) 0x7f80f8001120 con 0x7f81100b7910 2024-04-25T16:51:48.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.687+0000 7f810ffff640 1 --2- 172.21.15.94:0/3486584204 >> v2:172.21.15.138:3300/0 conn(0x7f8110161d30 0x7f81100b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:48.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f81149d7640 1 --2- 172.21.15.94:0/3486584204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8110146ad0 0x7f81100cfc10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8110074150 tx=0x7f810801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5dc3aae43c74bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:48.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f810f7fe640 1 -- 172.21.15.94:0/3486584204 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81080ab5f0 con 0x7f8110146ad0 2024-04-25T16:51:48.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f810f7fe640 1 -- 172.21.15.94:0/3486584204 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81080b0070 con 0x7f8110146ad0 2024-04-25T16:51:48.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f810f7fe640 1 -- 172.21.15.94:0/3486584204 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81080ab9b0 con 0x7f8110146ad0 2024-04-25T16:51:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f8116c3b640 1 -- 172.21.15.94:0/3486584204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8110146ad0 msgr2=0x7f81100cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f8116c3b640 1 --2- 172.21.15.94:0/3486584204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8110146ad0 0x7f81100cfc10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8110074150 tx=0x7f810801c3d0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f8116c3b640 1 -- 172.21.15.94:0/3486584204 shutdown_connections 2024-04-25T16:51:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f8116c3b640 1 --2- 172.21.15.94:0/3486584204 >> v2:172.21.15.138:3300/0 conn(0x7f8110161d30 0x7f81100b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f8116c3b640 1 --2- 172.21.15.94:0/3486584204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8110146ad0 0x7f81100cfc10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f8116c3b640 1 -- 172.21.15.94:0/3486584204 >> 172.21.15.94:0/3486584204 conn(0x7f81100d33d0 msgr2=0x7f81100d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f8116c3b640 1 -- 172.21.15.94:0/3486584204 shutdown_connections 2024-04-25T16:51:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f8116c3b640 1 -- 172.21.15.94:0/3486584204 wait complete. 2024-04-25T16:51:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f8116c3b640 1 Processor -- start 2024-04-25T16:51:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.691+0000 7f8116c3b640 1 -- start start 2024-04-25T16:51:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f8116c3b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81100b7910 0x7f81101f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f8116c3b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8110146ad0 0x7f8110131120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f8116c3b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8110161d30 0x7f81100cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f8116c3b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81100d0ee0 con 0x7f81100b7910 2024-04-25T16:51:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f8116c3b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8110161810 con 0x7f8110161d30 2024-04-25T16:51:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f8116c3b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8110162140 con 0x7f8110146ad0 2024-04-25T16:51:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f81149d7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81100b7910 0x7f81101f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f81149d7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81100b7910 0x7f81101f72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59564/0 (socket says 172.21.15.94:59564) 2024-04-25T16:51:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f81149d7640 1 -- 172.21.15.94:0/1364802642 learned_addr learned my addr 172.21.15.94:0/1364802642 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f810ffff640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8110146ad0 0x7f8110131120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f81149d7640 1 -- 172.21.15.94:0/1364802642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8110146ad0 msgr2=0x7f8110131120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f81149d7640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8110146ad0 0x7f8110131120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f81149d7640 1 -- 172.21.15.94:0/1364802642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8110161d30 msgr2=0x7f81100cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f81151d8640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8110161d30 0x7f81100cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f81149d7640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8110161d30 0x7f81100cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f81149d7640 1 -- 172.21.15.94:0/1364802642 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8110142690 con 0x7f81100b7910 2024-04-25T16:51:48.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f81149d7640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81100b7910 0x7f81101f72a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8108018fe0 tx=0x7f810801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:48.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f810d7fa640 1 -- 172.21.15.94:0/1364802642 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81080ba070 con 0x7f81100b7910 2024-04-25T16:51:48.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f810d7fa640 1 -- 172.21.15.94:0/1364802642 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81080abbb0 con 0x7f81100b7910 2024-04-25T16:51:48.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f8116c3b640 1 -- 172.21.15.94:0/1364802642 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8110131d00 con 0x7f81100b7910 2024-04-25T16:51:48.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f810d7fa640 1 -- 172.21.15.94:0/1364802642 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108004020 con 0x7f81100b7910 2024-04-25T16:51:48.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f8116c3b640 1 -- 172.21.15.94:0/1364802642 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81101322c0 con 0x7f81100b7910 2024-04-25T16:51:48.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f8116c3b640 1 -- 172.21.15.94:0/1364802642 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f811006a730 con 0x7f81100b7910 2024-04-25T16:51:48.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.695+0000 7f810d7fa640 1 -- 172.21.15.94:0/1364802642 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f81080041c0 con 0x7f81100b7910 2024-04-25T16:51:48.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.699+0000 7f810d7fa640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f80f006f400 0x7f80f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:48.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.699+0000 7f810d7fa640 1 -- 172.21.15.94:0/1364802642 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8108129570 con 0x7f81100b7910 2024-04-25T16:51:48.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.699+0000 7f810ffff640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f80f006f400 0x7f80f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:48.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.699+0000 7f810ffff640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f80f006f400 0x7f80f0071910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81100b4ea0 tx=0x7f8100008450 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:48.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.703+0000 7f810d7fa640 1 -- 172.21.15.94:0/1364802642 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8110142690 con 0x7f81100b7910 2024-04-25T16:51:48.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.859+0000 7f8116c3b640 1 -- 172.21.15.94:0/1364802642 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8110143270 con 0x7f81100b7910 2024-04-25T16:51:48.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.863+0000 7f810d7fa640 1 -- 172.21.15.94:0/1364802642 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f81080fe070 con 0x7f81100b7910 2024-04-25T16:51:48.860 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:48.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.863+0000 7f8116c3b640 1 -- 172.21.15.94:0/1364802642 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f80f006f400 msgr2=0x7f80f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.863+0000 7f8116c3b640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f80f006f400 0x7f80f0071910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81100b4ea0 tx=0x7f8100008450 comp rx=0 tx=0).stop 2024-04-25T16:51:48.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.863+0000 7f8116c3b640 1 -- 172.21.15.94:0/1364802642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81100b7910 msgr2=0x7f81101f72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:48.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.863+0000 7f8116c3b640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81100b7910 0x7f81101f72a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8108018fe0 tx=0x7f810801ac10 comp rx=0 tx=0).stop 2024-04-25T16:51:48.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.863+0000 7f8116c3b640 1 -- 172.21.15.94:0/1364802642 shutdown_connections 2024-04-25T16:51:48.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.863+0000 7f8116c3b640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8110161d30 0x7f81100cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.863+0000 7f8116c3b640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f80f006f400 0x7f80f0071910 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.863+0000 7f8116c3b640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8110146ad0 0x7f8110131120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.863+0000 7f8116c3b640 1 --2- 172.21.15.94:0/1364802642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81100b7910 0x7f81101f72a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:48.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.863+0000 7f8116c3b640 1 -- 172.21.15.94:0/1364802642 >> 172.21.15.94:0/1364802642 conn(0x7f81100d33d0 msgr2=0x7f8110072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:48.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.863+0000 7f8116c3b640 1 -- 172.21.15.94:0/1364802642 shutdown_connections 2024-04-25T16:51:48.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:48.863+0000 7f8116c3b640 1 -- 172.21.15.94:0/1364802642 wait complete. 2024-04-25T16:51:48.881 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.027+0000 7f07ff5e5640 1 Processor -- start 2024-04-25T16:51:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.027+0000 7f07ff5e5640 1 -- start start 2024-04-25T16:51:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07ff5e5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f07f815e990 0x7f07f8165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07ff5e5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8165b90 0x7f07f8167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07ff5e5640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f806d610 con 0x7f07f8070080 2024-04-25T16:51:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07ff5e5640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f07f8158550 con 0x7f07f815e990 2024-04-25T16:51:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07ff5e5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f07f806f7d0 con 0x7f07f8165b90 2024-04-25T16:51:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07fcb80640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f07f815e990 0x7f07f8165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07fdb82640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8165b90 0x7f07f8167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07fd381640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f07f8070080 0x7f07f815e280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:52404/0 (socket says 172.21.15.94:52404) 2024-04-25T16:51:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07fd381640 1 -- 172.21.15.94:0/265064061 learned_addr learned my addr 172.21.15.94:0/265064061 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07fcb80640 1 -- 172.21.15.94:0/265064061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8165b90 msgr2=0x7f07f8167fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:49.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07fcb80640 1 --2- 172.21.15.94:0/265064061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8165b90 0x7f07f8167fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07fcb80640 1 -- 172.21.15.94:0/265064061 >> v1:172.21.15.138:6789/0 conn(0x7f07f8070080 legacy=0x7f07f815e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07fcb80640 1 -- 172.21.15.94:0/265064061 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f806fc60 con 0x7f07f815e990 2024-04-25T16:51:49.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.031+0000 7f07fdb82640 1 --2- 172.21.15.94:0/265064061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8165b90 0x7f07f8167fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:49.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07fcb80640 1 --2- 172.21.15.94:0/265064061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07f815e990 0x7f07f8165650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f07e80097c0 tx=0x7f07e801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c647fa6f25f88cf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:49.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07effff640 1 -- 172.21.15.94:0/265064061 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e80ab680 con 0x7f07f815e990 2024-04-25T16:51:49.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07effff640 1 -- 172.21.15.94:0/265064061 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e80b0070 con 0x7f07f815e990 2024-04-25T16:51:49.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07effff640 1 -- 172.21.15.94:0/265064061 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e80aba40 con 0x7f07f815e990 2024-04-25T16:51:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07ff5e5640 1 -- 172.21.15.94:0/265064061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07f815e990 msgr2=0x7f07f8165650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07ff5e5640 1 --2- 172.21.15.94:0/265064061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07f815e990 0x7f07f8165650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f07e80097c0 tx=0x7f07e801c480 comp rx=0 tx=0).stop 2024-04-25T16:51:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07ff5e5640 1 -- 172.21.15.94:0/265064061 shutdown_connections 2024-04-25T16:51:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07ff5e5640 1 --2- 172.21.15.94:0/265064061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8165b90 0x7f07f8167fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07ff5e5640 1 --2- 172.21.15.94:0/265064061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07f815e990 0x7f07f8165650 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07ff5e5640 1 -- 172.21.15.94:0/265064061 >> 172.21.15.94:0/265064061 conn(0x7f07f80b47f0 msgr2=0x7f07f80b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07ff5e5640 1 -- 172.21.15.94:0/265064061 shutdown_connections 2024-04-25T16:51:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07ff5e5640 1 -- 172.21.15.94:0/265064061 wait complete. 2024-04-25T16:51:49.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07ff5e5640 1 Processor -- start 2024-04-25T16:51:49.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.035+0000 7f07ff5e5640 1 -- start start 2024-04-25T16:51:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07ff5e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07f8070080 0x7f07f81fd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07ff5e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07f815e990 0x7f07f8156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07ff5e5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8165b90 0x7f07f8074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07ff5e5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f07f80ae6c0 con 0x7f07f8070080 2024-04-25T16:51:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07ff5e5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f07f806f460 con 0x7f07f815e990 2024-04-25T16:51:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07ff5e5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f07f8070490 con 0x7f07f8165b90 2024-04-25T16:51:49.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07fd381640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07f8070080 0x7f07f81fd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07fd381640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07f8070080 0x7f07f81fd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59576/0 (socket says 172.21.15.94:59576) 2024-04-25T16:51:49.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07fd381640 1 -- 172.21.15.94:0/4259433893 learned_addr learned my addr 172.21.15.94:0/4259433893 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:49.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07fdb82640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8165b90 0x7f07f8074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07fcb80640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07f815e990 0x7f07f8156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07fd381640 1 -- 172.21.15.94:0/4259433893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8165b90 msgr2=0x7f07f8074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07fd381640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8165b90 0x7f07f8074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07fd381640 1 -- 172.21.15.94:0/4259433893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07f815e990 msgr2=0x7f07f8156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07fd381640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07f815e990 0x7f07f8156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07fd381640 1 -- 172.21.15.94:0/4259433893 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f80d0a20 con 0x7f07f8070080 2024-04-25T16:51:49.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07fcb80640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07f815e990 0x7f07f8156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:49.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07fd381640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07f8070080 0x7f07f81fd5b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f07f400c340 tx=0x7f07f400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:49.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07edffb640 1 -- 172.21.15.94:0/4259433893 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4017070 con 0x7f07f8070080 2024-04-25T16:51:49.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07edffb640 1 -- 172.21.15.94:0/4259433893 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f4005d10 con 0x7f07f8070080 2024-04-25T16:51:49.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07edffb640 1 -- 172.21.15.94:0/4259433893 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4007310 con 0x7f07f8070080 2024-04-25T16:51:49.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07ff5e5640 1 -- 172.21.15.94:0/4259433893 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f81fcad0 con 0x7f07f8070080 2024-04-25T16:51:49.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07ff5e5640 1 -- 172.21.15.94:0/4259433893 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f81fd090 con 0x7f07f8070080 2024-04-25T16:51:49.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07ff5e5640 1 -- 172.21.15.94:0/4259433893 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f8076830 con 0x7f07f8070080 2024-04-25T16:51:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.039+0000 7f07edffb640 1 -- 172.21.15.94:0/4259433893 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f07f401a070 con 0x7f07f8070080 2024-04-25T16:51:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.043+0000 7f07edffb640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f07dc06f400 0x7f07dc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.043+0000 7f07edffb640 1 -- 172.21.15.94:0/4259433893 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f07f408ea10 con 0x7f07f8070080 2024-04-25T16:51:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.043+0000 7f07fcb80640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f07dc06f400 0x7f07dc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.043+0000 7f07fcb80640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f07dc06f400 0x7f07dc071910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07e80af000 tx=0x7f07e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:49.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.047+0000 7f07edffb640 1 -- 172.21.15.94:0/4259433893 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f07f806d610 con 0x7f07f8070080 2024-04-25T16:51:49.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.195+0000 7f07ff5e5640 1 -- 172.21.15.94:0/4259433893 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07f815f3f0 con 0x7f07f8070080 2024-04-25T16:51:49.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.195+0000 7f07edffb640 1 -- 172.21.15.94:0/4259433893 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7f07f405b500 con 0x7f07f8070080 2024-04-25T16:51:49.195 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:49.195 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.199+0000 7f07ff5e5640 1 -- 172.21.15.94:0/4259433893 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f07dc06f400 msgr2=0x7f07dc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.199+0000 7f07ff5e5640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f07dc06f400 0x7f07dc071910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07e80af000 tx=0x7f07e8008040 comp rx=0 tx=0).stop 2024-04-25T16:51:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.199+0000 7f07ff5e5640 1 -- 172.21.15.94:0/4259433893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07f8070080 msgr2=0x7f07f81fd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.199+0000 7f07ff5e5640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07f8070080 0x7f07f81fd5b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f07f400c340 tx=0x7f07f400c800 comp rx=0 tx=0).stop 2024-04-25T16:51:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.199+0000 7f07ff5e5640 1 -- 172.21.15.94:0/4259433893 shutdown_connections 2024-04-25T16:51:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.199+0000 7f07ff5e5640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f07dc06f400 0x7f07dc071910 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.199+0000 7f07ff5e5640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07f8165b90 0x7f07f8074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.199+0000 7f07ff5e5640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07f815e990 0x7f07f8156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.199+0000 7f07ff5e5640 1 --2- 172.21.15.94:0/4259433893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07f8070080 0x7f07f81fd5b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.199+0000 7f07ff5e5640 1 -- 172.21.15.94:0/4259433893 >> 172.21.15.94:0/4259433893 conn(0x7f07f80b47f0 msgr2=0x7f07f80d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.199+0000 7f07ff5e5640 1 -- 172.21.15.94:0/4259433893 shutdown_connections 2024-04-25T16:51:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.199+0000 7f07ff5e5640 1 -- 172.21.15.94:0/4259433893 wait complete. 2024-04-25T16:51:49.215 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:51:49.216 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:51:49.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.367+0000 7f51e1483640 1 Processor -- start 2024-04-25T16:51:49.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.367+0000 7f51e1483640 1 -- start start 2024-04-25T16:51:49.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.367+0000 7f51e1483640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f51dc0b5260 0x7f51dc0b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.367+0000 7f51e1483640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc073ae0 0x7f51dc073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.367+0000 7f51e1483640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51dc074430 con 0x7f51dc06fb50 2024-04-25T16:51:49.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.367+0000 7f51e1483640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f51dc06e740 con 0x7f51dc0b5260 2024-04-25T16:51:49.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.367+0000 7f51e1483640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f51dc06ff60 con 0x7f51dc073ae0 2024-04-25T16:51:49.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.367+0000 7f51daffd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f51dc06fb50 0x7f51dc06e030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49464/0 (socket says 172.21.15.94:49464) 2024-04-25T16:51:49.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.367+0000 7f51daffd640 1 -- 172.21.15.94:0/3433319695 learned_addr learned my addr 172.21.15.94:0/3433319695 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:49.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51da7fc640 1 --2- 172.21.15.94:0/3433319695 >> v2:172.21.15.138:3300/0 conn(0x7f51dc0b5260 0x7f51dc0b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51db7fe640 1 --2- 172.21.15.94:0/3433319695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc073ae0 0x7f51dc073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2045703797 0 0) 0x7f51cc001120 con 0x7f51dc06fb50 2024-04-25T16:51:49.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51c4003530 con 0x7f51dc06fb50 2024-04-25T16:51:49.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2145203322 0 0) 0x7f51c4003530 con 0x7f51dc06fb50 2024-04-25T16:51:49.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc073ae0 msgr2=0x7f51dc073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 --2- 172.21.15.94:0/3433319695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc073ae0 0x7f51dc073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 >> v2:172.21.15.138:3300/0 conn(0x7f51dc0b5260 msgr2=0x7f51dc0b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 --2- 172.21.15.94:0/3433319695 >> v2:172.21.15.138:3300/0 conn(0x7f51dc0b5260 0x7f51dc0b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc070230 con 0x7f51dc06fb50 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f51cc001850 con 0x7f51dc06fb50 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 >> v1:172.21.15.94:6789/0 conn(0x7f51dc06fb50 legacy=0x7f51dc06e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 --2- 172.21.15.94:0/3433319695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c40039c0 0x7f51c4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 --2- 172.21.15.94:0/3433319695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c4004810 0x7f51c4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 --2- 172.21.15.94:0/3433319695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc073ae0 0x7f51c4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc06ff60 con 0x7f51c40039c0 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51c400ba80 con 0x7f51c4004810 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f51c400bc30 con 0x7f51dc073ae0 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51daffd640 1 --2- 172.21.15.94:0/3433319695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c40039c0 0x7f51c4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51da7fc640 1 --2- 172.21.15.94:0/3433319695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c4004810 0x7f51c4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51db7fe640 1 --2- 172.21.15.94:0/3433319695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc073ae0 0x7f51c4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51daffd640 1 -- 172.21.15.94:0/3433319695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc073ae0 msgr2=0x7f51c4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:49.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51daffd640 1 --2- 172.21.15.94:0/3433319695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc073ae0 0x7f51c4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51daffd640 1 -- 172.21.15.94:0/3433319695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c4004810 msgr2=0x7f51c4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:49.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51daffd640 1 --2- 172.21.15.94:0/3433319695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c4004810 0x7f51c4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51daffd640 1 -- 172.21.15.94:0/3433319695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c4003530 con 0x7f51c40039c0 2024-04-25T16:51:49.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51daffd640 1 --2- 172.21.15.94:0/3433319695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c40039c0 0x7f51c4003e30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c490 tx=0x7f51cc01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cae265c0830c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:49.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc00c0e0 con 0x7f51c40039c0 2024-04-25T16:51:49.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51cc0a9020 con 0x7f51c40039c0 2024-04-25T16:51:49.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51d9ffb640 1 -- 172.21.15.94:0/3433319695 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc005b20 con 0x7f51c40039c0 2024-04-25T16:51:49.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51e1483640 1 -- 172.21.15.94:0/3433319695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c40039c0 msgr2=0x7f51c4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.371+0000 7f51e1483640 1 --2- 172.21.15.94:0/3433319695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c40039c0 0x7f51c4003e30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c490 tx=0x7f51cc01cca0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51db7fe640 1 -- 172.21.15.94:0/3433319695 reap_dead start 2024-04-25T16:51:49.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51e1483640 1 -- 172.21.15.94:0/3433319695 shutdown_connections 2024-04-25T16:51:49.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51e1483640 1 -- 172.21.15.94:0/3433319695 >> 172.21.15.94:0/3433319695 conn(0x7f51dc0d2690 msgr2=0x7f51dc0d2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:49.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51e1483640 1 -- 172.21.15.94:0/3433319695 shutdown_connections 2024-04-25T16:51:49.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51e1483640 1 -- 172.21.15.94:0/3433319695 wait complete. 2024-04-25T16:51:49.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51e1483640 1 Processor -- start 2024-04-25T16:51:49.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51e1483640 1 -- start start 2024-04-25T16:51:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51e1483640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c40039c0 0x7f51dc15f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51e1483640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51dc1500c0 0x7f51dc1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51e1483640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc156ae0 0x7f51dc146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51e1483640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc167660 con 0x7f51c40039c0 2024-04-25T16:51:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51e1483640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc1674e0 con 0x7f51dc1500c0 2024-04-25T16:51:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51e1483640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f51dc1677e0 con 0x7f51dc156ae0 2024-04-25T16:51:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51daffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c40039c0 0x7f51dc15f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51daffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c40039c0 0x7f51dc15f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59598/0 (socket says 172.21.15.94:59598) 2024-04-25T16:51:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51daffd640 1 -- 172.21.15.94:0/3925208338 learned_addr learned my addr 172.21.15.94:0/3925208338 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51db7fe640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc156ae0 0x7f51dc146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51da7fc640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51dc1500c0 0x7f51dc1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51daffd640 1 -- 172.21.15.94:0/3925208338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc156ae0 msgr2=0x7f51dc146760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51daffd640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc156ae0 0x7f51dc146760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51daffd640 1 -- 172.21.15.94:0/3925208338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51dc1500c0 msgr2=0x7f51dc1565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51daffd640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51dc1500c0 0x7f51dc1565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51daffd640 1 -- 172.21.15.94:0/3925208338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc153860 con 0x7f51c40039c0 2024-04-25T16:51:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.375+0000 7f51da7fc640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51dc1500c0 0x7f51dc1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.379+0000 7f51daffd640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c40039c0 0x7f51dc15f7f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f51cc0ab040 tx=0x7f51c4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.379+0000 7f51c37fe640 1 -- 172.21.15.94:0/3925208338 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc005d00 con 0x7f51c40039c0 2024-04-25T16:51:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.379+0000 7f51c37fe640 1 -- 172.21.15.94:0/3925208338 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51cc0047f0 con 0x7f51c40039c0 2024-04-25T16:51:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.379+0000 7f51c37fe640 1 -- 172.21.15.94:0/3925208338 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc004ba0 con 0x7f51c40039c0 2024-04-25T16:51:49.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.379+0000 7f51e1483640 1 -- 172.21.15.94:0/3925208338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51dc15fd30 con 0x7f51c40039c0 2024-04-25T16:51:49.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.379+0000 7f51e1483640 1 -- 172.21.15.94:0/3925208338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51dc152f30 con 0x7f51c40039c0 2024-04-25T16:51:49.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.379+0000 7f51e1483640 1 -- 172.21.15.94:0/3925208338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51dc06a730 con 0x7f51c40039c0 2024-04-25T16:51:49.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.379+0000 7f51c37fe640 1 -- 172.21.15.94:0/3925208338 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f51cc0a5780 con 0x7f51c40039c0 2024-04-25T16:51:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.379+0000 7f51c37fe640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f51c806f400 0x7f51c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.379+0000 7f51c37fe640 1 -- 172.21.15.94:0/3925208338 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f51cc128d90 con 0x7f51c40039c0 2024-04-25T16:51:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.379+0000 7f51da7fc640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f51c806f400 0x7f51c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.383+0000 7f51da7fc640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f51c806f400 0x7f51c8071910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f51dc15f060 tx=0x7f51d000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:49.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.383+0000 7f51c37fe640 1 -- 172.21.15.94:0/3925208338 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f51cc0b4070 con 0x7f51c40039c0 2024-04-25T16:51:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.547+0000 7f51e1483640 1 -- 172.21.15.94:0/3925208338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51dc157fa0 con 0x7f51c40039c0 2024-04-25T16:51:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.547+0000 7f51c37fe640 1 -- 172.21.15.94:0/3925208338 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f51dc074430 con 0x7f51c40039c0 2024-04-25T16:51:49.547 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:51:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.551+0000 7f51e1483640 1 -- 172.21.15.94:0/3925208338 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f51c806f400 msgr2=0x7f51c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.551+0000 7f51e1483640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f51c806f400 0x7f51c8071910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f51dc15f060 tx=0x7f51d000b040 comp rx=0 tx=0).stop 2024-04-25T16:51:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.551+0000 7f51e1483640 1 -- 172.21.15.94:0/3925208338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c40039c0 msgr2=0x7f51dc15f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.551+0000 7f51e1483640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c40039c0 0x7f51dc15f7f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f51cc0ab040 tx=0x7f51c4005840 comp rx=0 tx=0).stop 2024-04-25T16:51:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.551+0000 7f51e1483640 1 -- 172.21.15.94:0/3925208338 shutdown_connections 2024-04-25T16:51:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.551+0000 7f51e1483640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f51c806f400 0x7f51c8071910 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.551+0000 7f51e1483640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51dc156ae0 0x7f51dc146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.551+0000 7f51e1483640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51dc1500c0 0x7f51dc1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.551+0000 7f51e1483640 1 --2- 172.21.15.94:0/3925208338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c40039c0 0x7f51dc15f7f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.551+0000 7f51e1483640 1 -- 172.21.15.94:0/3925208338 >> 172.21.15.94:0/3925208338 conn(0x7f51dc0d2690 msgr2=0x7f51dc155b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:49.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.551+0000 7f51e1483640 1 -- 172.21.15.94:0/3925208338 shutdown_connections 2024-04-25T16:51:49.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.551+0000 7f51e1483640 1 -- 172.21.15.94:0/3925208338 wait complete. 2024-04-25T16:51:49.569 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:51:49.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff781102640 1 Processor -- start 2024-04-25T16:51:49.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff781102640 1 -- start start 2024-04-25T16:51:49.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff781102640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff77c0b0420 0x7ff77c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff781102640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff77c15ec30 0x7ff77c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff781102640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff77c070630 con 0x7ff77c06f2a0 2024-04-25T16:51:49.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff781102640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff77c0aeaa0 con 0x7ff77c15ec30 2024-04-25T16:51:49.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff781102640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff77c0703e0 con 0x7ff77c0b0420 2024-04-25T16:51:49.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff77ad9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff77c0b0420 0x7ff77c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff77b59e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff77c15ec30 0x7ff77c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff77ad9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff77c0b0420 0x7ff77c06ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:50238/0 (socket says 172.21.15.94:50238) 2024-04-25T16:51:49.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff77ad9d640 1 -- 172.21.15.94:0/2341371619 learned_addr learned my addr 172.21.15.94:0/2341371619 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:49.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff77a59c640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff77c06f2a0 0x7ff77c06c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:52416/0 (socket says 172.21.15.94:52416) 2024-04-25T16:51:49.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff77ad9d640 1 -- 172.21.15.94:0/2341371619 >> v2:172.21.15.138:3300/0 conn(0x7ff77c15ec30 msgr2=0x7ff77c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:49.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff77ad9d640 1 --2- 172.21.15.94:0/2341371619 >> v2:172.21.15.138:3300/0 conn(0x7ff77c15ec30 0x7ff77c06cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.723+0000 7ff77ad9d640 1 -- 172.21.15.94:0/2341371619 >> v1:172.21.15.138:6789/0 conn(0x7ff77c06f2a0 legacy=0x7ff77c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff77ad9d640 1 -- 172.21.15.94:0/2341371619 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c06f6b0 con 0x7ff77c0b0420 2024-04-25T16:51:49.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff77b59e640 1 --2- 172.21.15.94:0/2341371619 >> v2:172.21.15.138:3300/0 conn(0x7ff77c15ec30 0x7ff77c06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:49.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff77ad9d640 1 --2- 172.21.15.94:0/2341371619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff77c0b0420 0x7ff77c06ed60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0aec20 tx=0x7ff76401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a768baa7fbcca579 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:49.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff779d9b640 1 -- 172.21.15.94:0/2341371619 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7640aa670 con 0x7ff77c0b0420 2024-04-25T16:51:49.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff779d9b640 1 -- 172.21.15.94:0/2341371619 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7640af070 con 0x7ff77c0b0420 2024-04-25T16:51:49.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff779d9b640 1 -- 172.21.15.94:0/2341371619 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7640aaa30 con 0x7ff77c0b0420 2024-04-25T16:51:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff781102640 1 -- 172.21.15.94:0/2341371619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff77c0b0420 msgr2=0x7ff77c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff781102640 1 --2- 172.21.15.94:0/2341371619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff77c0b0420 0x7ff77c06ed60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0aec20 tx=0x7ff76401c510 comp rx=0 tx=0).stop 2024-04-25T16:51:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff781102640 1 -- 172.21.15.94:0/2341371619 shutdown_connections 2024-04-25T16:51:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff781102640 1 --2- 172.21.15.94:0/2341371619 >> v2:172.21.15.138:3300/0 conn(0x7ff77c15ec30 0x7ff77c06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff781102640 1 --2- 172.21.15.94:0/2341371619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff77c0b0420 0x7ff77c06ed60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff781102640 1 -- 172.21.15.94:0/2341371619 >> 172.21.15.94:0/2341371619 conn(0x7ff77c0b47f0 msgr2=0x7ff77c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff781102640 1 -- 172.21.15.94:0/2341371619 shutdown_connections 2024-04-25T16:51:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.727+0000 7ff781102640 1 -- 172.21.15.94:0/2341371619 wait complete. 2024-04-25T16:51:49.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.731+0000 7ff781102640 1 Processor -- start 2024-04-25T16:51:49.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.731+0000 7ff781102640 1 -- start start 2024-04-25T16:51:49.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.731+0000 7ff781102640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff77c06f2a0 0x7ff77c13ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.731+0000 7ff781102640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff77c0b0420 0x7ff77c13b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.731+0000 7ff781102640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff77c15ec30 0x7ff77c14cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.731+0000 7ff781102640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c0b0830 con 0x7ff77c0b0420 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.731+0000 7ff781102640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c0aef30 con 0x7ff77c06f2a0 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.731+0000 7ff781102640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff77c0b09b0 con 0x7ff77c15ec30 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.731+0000 7ff77a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff77c0b0420 0x7ff77c13b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.731+0000 7ff77a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff77c0b0420 0x7ff77c13b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59600/0 (socket says 172.21.15.94:59600) 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.731+0000 7ff77a59c640 1 -- 172.21.15.94:0/469701383 learned_addr learned my addr 172.21.15.94:0/469701383 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff77ad9d640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff77c06f2a0 0x7ff77c13ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff77b59e640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff77c15ec30 0x7ff77c14cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff77a59c640 1 -- 172.21.15.94:0/469701383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff77c15ec30 msgr2=0x7ff77c14cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff77a59c640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff77c15ec30 0x7ff77c14cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff77a59c640 1 -- 172.21.15.94:0/469701383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff77c06f2a0 msgr2=0x7ff77c13ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff77a59c640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff77c06f2a0 0x7ff77c13ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff77a59c640 1 -- 172.21.15.94:0/469701383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c14d9e0 con 0x7ff77c0b0420 2024-04-25T16:51:49.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff77ad9d640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff77c06f2a0 0x7ff77c13ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:49.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff77a59c640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff77c0b0420 0x7ff77c13b250 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00c310 tx=0x7ff76c00c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:49.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff7737fe640 1 -- 172.21.15.94:0/469701383 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0074e0 con 0x7ff77c0b0420 2024-04-25T16:51:49.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff7737fe640 1 -- 172.21.15.94:0/469701383 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76c007680 con 0x7ff77c0b0420 2024-04-25T16:51:49.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff7737fe640 1 -- 172.21.15.94:0/469701383 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c004a30 con 0x7ff77c0b0420 2024-04-25T16:51:49.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff781102640 1 -- 172.21.15.94:0/469701383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77c14dcc0 con 0x7ff77c0b0420 2024-04-25T16:51:49.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff781102640 1 -- 172.21.15.94:0/469701383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77c1567a0 con 0x7ff77c0b0420 2024-04-25T16:51:49.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff781102640 1 -- 172.21.15.94:0/469701383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77c076830 con 0x7ff77c0b0420 2024-04-25T16:51:49.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff7737fe640 1 -- 172.21.15.94:0/469701383 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff76c01a020 con 0x7ff77c0b0420 2024-04-25T16:51:49.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff7737fe640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff75c06f400 0x7ff75c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:49.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff77ad9d640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff75c06f400 0x7ff75c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:49.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff7737fe640 1 -- 172.21.15.94:0/469701383 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff76c08ec90 con 0x7ff77c0b0420 2024-04-25T16:51:49.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.735+0000 7ff77ad9d640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff75c06f400 0x7ff75c071910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff7640ae000 tx=0x7ff7640a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.739+0000 7ff7737fe640 1 -- 172.21.15.94:0/469701383 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff76c05b810 con 0x7ff77c0b0420 2024-04-25T16:51:49.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.891+0000 7ff781102640 1 -- 172.21.15.94:0/469701383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff77c0d0d50 con 0x7ff77c0b0420 2024-04-25T16:51:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.891+0000 7ff7737fe640 1 -- 172.21.15.94:0/469701383 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7ff76c05b110 con 0x7ff77c0b0420 2024-04-25T16:51:49.891 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:49.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.895+0000 7ff781102640 1 -- 172.21.15.94:0/469701383 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff75c06f400 msgr2=0x7ff75c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.895+0000 7ff781102640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff75c06f400 0x7ff75c071910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff7640ae000 tx=0x7ff7640a7000 comp rx=0 tx=0).stop 2024-04-25T16:51:49.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.895+0000 7ff781102640 1 -- 172.21.15.94:0/469701383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff77c0b0420 msgr2=0x7ff77c13b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:49.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.895+0000 7ff781102640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff77c0b0420 0x7ff77c13b250 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00c310 tx=0x7ff76c00c710 comp rx=0 tx=0).stop 2024-04-25T16:51:49.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.895+0000 7ff781102640 1 -- 172.21.15.94:0/469701383 shutdown_connections 2024-04-25T16:51:49.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.895+0000 7ff781102640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff75c06f400 0x7ff75c071910 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.895+0000 7ff781102640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff77c15ec30 0x7ff77c14cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.895+0000 7ff781102640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff77c0b0420 0x7ff77c13b250 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.895+0000 7ff781102640 1 --2- 172.21.15.94:0/469701383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff77c06f2a0 0x7ff77c13ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:49.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.895+0000 7ff781102640 1 -- 172.21.15.94:0/469701383 >> 172.21.15.94:0/469701383 conn(0x7ff77c0b47f0 msgr2=0x7ff77c1547e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:49.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.895+0000 7ff781102640 1 -- 172.21.15.94:0/469701383 shutdown_connections 2024-04-25T16:51:49.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:49.895+0000 7ff781102640 1 -- 172.21.15.94:0/469701383 wait complete. 2024-04-25T16:51:49.911 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:51:50.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.063+0000 7f919eb7c640 1 Processor -- start 2024-04-25T16:51:50.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.063+0000 7f919eb7c640 1 -- start start 2024-04-25T16:51:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f919eb7c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f919806d710 0x7f919806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f919eb7c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9198074380 con 0x7f919806e7a0 2024-04-25T16:51:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f919eb7c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f919806e060 con 0x7f919815c7e0 2024-04-25T16:51:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f919eb7c640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f91980727e0 con 0x7f919806d710 2024-04-25T16:51:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f9197fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f919806d710 0x7f919806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f919d119640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f919806e7a0 0x7f919806ebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49474/0 (socket says 172.21.15.94:49474) 2024-04-25T16:51:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f919d119640 1 -- 172.21.15.94:0/912352888 learned_addr learned my addr 172.21.15.94:0/912352888 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f9197fff640 1 -- 172.21.15.94:0/912352888 >> v1:172.21.15.138:6789/0 conn(0x7f919815c7e0 legacy=0x7f91980afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f9197fff640 1 -- 172.21.15.94:0/912352888 >> v1:172.21.15.94:6789/0 conn(0x7f919806e7a0 legacy=0x7f919806ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f9197fff640 1 -- 172.21.15.94:0/912352888 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9198072ac0 con 0x7f919806d710 2024-04-25T16:51:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f9197fff640 1 --2- 172.21.15.94:0/912352888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919806d710 0x7f919806db20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f918c0097c0 tx=0x7f918c01c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e21b639d5ffb182 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f91977fe640 1 -- 172.21.15.94:0/912352888 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c0b0070 con 0x7f919806d710 2024-04-25T16:51:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f91977fe640 1 -- 172.21.15.94:0/912352888 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f918c0a4e50 con 0x7f919806d710 2024-04-25T16:51:50.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.067+0000 7f91977fe640 1 -- 172.21.15.94:0/912352888 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c0ab820 con 0x7f919806d710 2024-04-25T16:51:50.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 -- 172.21.15.94:0/912352888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919806d710 msgr2=0x7f919806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 --2- 172.21.15.94:0/912352888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919806d710 0x7f919806db20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f918c0097c0 tx=0x7f918c01c1c0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 -- 172.21.15.94:0/912352888 shutdown_connections 2024-04-25T16:51:50.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 --2- 172.21.15.94:0/912352888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919806d710 0x7f919806db20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 -- 172.21.15.94:0/912352888 >> 172.21.15.94:0/912352888 conn(0x7f91980b47f0 msgr2=0x7f91980b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:50.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 -- 172.21.15.94:0/912352888 shutdown_connections 2024-04-25T16:51:50.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 -- 172.21.15.94:0/912352888 wait complete. 2024-04-25T16:51:50.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 Processor -- start 2024-04-25T16:51:50.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 -- start start 2024-04-25T16:51:50.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f919806d710 0x7f91981580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f919806e7a0 0x7f919806c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919815c7e0 0x7f919806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f919806f2c0 con 0x7f919815c7e0 2024-04-25T16:51:50.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9198073ca0 con 0x7f919806d710 2024-04-25T16:51:50.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919eb7c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f919815cbf0 con 0x7f919806e7a0 2024-04-25T16:51:50.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919d119640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919815c7e0 0x7f919806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919d119640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919815c7e0 0x7f919806cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59624/0 (socket says 172.21.15.94:59624) 2024-04-25T16:51:50.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.071+0000 7f919d119640 1 -- 172.21.15.94:0/2870661317 learned_addr learned my addr 172.21.15.94:0/2870661317 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:50.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f919d119640 1 -- 172.21.15.94:0/2870661317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f919806e7a0 msgr2=0x7f919806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f9197fff640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f919806e7a0 0x7f919806c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f919d119640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f919806e7a0 0x7f919806c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f919c918640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f919806d710 0x7f91981580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f919d119640 1 -- 172.21.15.94:0/2870661317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f919806d710 msgr2=0x7f91981580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f919d119640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f919806d710 0x7f91981580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f919d119640 1 -- 172.21.15.94:0/2870661317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9198167ba0 con 0x7f919815c7e0 2024-04-25T16:51:50.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f919d119640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919815c7e0 0x7f919806cc40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f919000c3f0 tx=0x7f919000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:50.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f91957fa640 1 -- 172.21.15.94:0/2870661317 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190017070 con 0x7f919815c7e0 2024-04-25T16:51:50.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f91957fa640 1 -- 172.21.15.94:0/2870661317 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9190005cf0 con 0x7f919815c7e0 2024-04-25T16:51:50.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f919eb7c640 1 -- 172.21.15.94:0/2870661317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919815f070 con 0x7f919815c7e0 2024-04-25T16:51:50.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f91957fa640 1 -- 172.21.15.94:0/2870661317 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91900072f0 con 0x7f919815c7e0 2024-04-25T16:51:50.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f919eb7c640 1 -- 172.21.15.94:0/2870661317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919809e670 con 0x7f919815c7e0 2024-04-25T16:51:50.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f919eb7c640 1 -- 172.21.15.94:0/2870661317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9198076830 con 0x7f919815c7e0 2024-04-25T16:51:50.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f91957fa640 1 -- 172.21.15.94:0/2870661317 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f919001a070 con 0x7f919815c7e0 2024-04-25T16:51:50.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f91957fa640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f918406f400 0x7f9184071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f919c918640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f918406f400 0x7f9184071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f91957fa640 1 -- 172.21.15.94:0/2870661317 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f919008ea10 con 0x7f919815c7e0 2024-04-25T16:51:50.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.075+0000 7f919c918640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f918406f400 0x7f9184071910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9188007bf0 tx=0x7f9188007240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:50.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.079+0000 7f91957fa640 1 -- 172.21.15.94:0/2870661317 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9198074380 con 0x7f919815c7e0 2024-04-25T16:51:50.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.239+0000 7f919eb7c640 1 -- 172.21.15.94:0/2870661317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f91981601d0 con 0x7f919815c7e0 2024-04-25T16:51:50.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.243+0000 7f91957fa640 1 -- 172.21.15.94:0/2870661317 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f919005b500 con 0x7f919815c7e0 2024-04-25T16:51:50.240 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:51:50.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.243+0000 7f919eb7c640 1 -- 172.21.15.94:0/2870661317 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f918406f400 msgr2=0x7f9184071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.243+0000 7f919eb7c640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f918406f400 0x7f9184071910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9188007bf0 tx=0x7f9188007240 comp rx=0 tx=0).stop 2024-04-25T16:51:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.243+0000 7f919eb7c640 1 -- 172.21.15.94:0/2870661317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919815c7e0 msgr2=0x7f919806cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.243+0000 7f919eb7c640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919815c7e0 0x7f919806cc40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f919000c3f0 tx=0x7f919000c8b0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.243+0000 7f919eb7c640 1 -- 172.21.15.94:0/2870661317 shutdown_connections 2024-04-25T16:51:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.243+0000 7f919eb7c640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f919815c7e0 0x7f919806cc40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.243+0000 7f919eb7c640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f918406f400 0x7f9184071910 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.243+0000 7f919eb7c640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f919806e7a0 0x7f919806c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.243+0000 7f919eb7c640 1 --2- 172.21.15.94:0/2870661317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f919806d710 0x7f91981580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.243+0000 7f919eb7c640 1 -- 172.21.15.94:0/2870661317 >> 172.21.15.94:0/2870661317 conn(0x7f91980b47f0 msgr2=0x7f9198158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.243+0000 7f919eb7c640 1 -- 172.21.15.94:0/2870661317 shutdown_connections 2024-04-25T16:51:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.243+0000 7f919eb7c640 1 -- 172.21.15.94:0/2870661317 wait complete. 2024-04-25T16:51:50.262 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:51:50.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f9622315640 1 Processor -- start 2024-04-25T16:51:50.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f9622315640 1 -- start start 2024-04-25T16:51:50.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f9622315640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f961c15c7e0 0x7f961c0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f9622315640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f961c0b0330 0x7f961c06d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f9622315640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f961c074380 con 0x7f961c06dc50 2024-04-25T16:51:50.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f9622315640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f961c06e060 con 0x7f961c15c7e0 2024-04-25T16:51:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f9622315640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f961c0727e0 con 0x7f961c0b0330 2024-04-25T16:51:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f961bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f961c15c7e0 0x7f961c0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f961bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f961c15c7e0 0x7f961c0afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59630/0 (socket says 172.21.15.94:59630) 2024-04-25T16:51:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f961bfff640 1 -- 172.21.15.94:0/2280659968 learned_addr learned my addr 172.21.15.94:0/2280659968 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f961b7fe640 1 --2- 172.21.15.94:0/2280659968 >> v2:172.21.15.138:3300/0 conn(0x7f961c0b0330 0x7f961c06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f961bfff640 1 -- 172.21.15.94:0/2280659968 >> v2:172.21.15.138:3300/0 conn(0x7f961c0b0330 msgr2=0x7f961c06d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f961bfff640 1 --2- 172.21.15.94:0/2280659968 >> v2:172.21.15.138:3300/0 conn(0x7f961c0b0330 0x7f961c06d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f961bfff640 1 -- 172.21.15.94:0/2280659968 >> v1:172.21.15.138:6789/0 conn(0x7f961c06dc50 legacy=0x7f961c06e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:50.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.411+0000 7f961bfff640 1 -- 172.21.15.94:0/2280659968 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c072ac0 con 0x7f961c15c7e0 2024-04-25T16:51:50.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.415+0000 7f961bfff640 1 --2- 172.21.15.94:0/2280659968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961c15c7e0 0x7f961c0afdf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f961c0b5020 tx=0x7f961401c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155e1d05d60ecd4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:50.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.415+0000 7f961affd640 1 -- 172.21.15.94:0/2280659968 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96140b0070 con 0x7f961c15c7e0 2024-04-25T16:51:50.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.415+0000 7f961affd640 1 -- 172.21.15.94:0/2280659968 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96140ab600 con 0x7f961c15c7e0 2024-04-25T16:51:50.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.415+0000 7f961affd640 1 -- 172.21.15.94:0/2280659968 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96140ab9d0 con 0x7f961c15c7e0 2024-04-25T16:51:50.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.415+0000 7f9622315640 1 -- 172.21.15.94:0/2280659968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961c15c7e0 msgr2=0x7f961c0afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.415+0000 7f9622315640 1 --2- 172.21.15.94:0/2280659968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961c15c7e0 0x7f961c0afdf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f961c0b5020 tx=0x7f961401c240 comp rx=0 tx=0).stop 2024-04-25T16:51:50.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.415+0000 7f9622315640 1 -- 172.21.15.94:0/2280659968 shutdown_connections 2024-04-25T16:51:50.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.415+0000 7f9622315640 1 --2- 172.21.15.94:0/2280659968 >> v2:172.21.15.138:3300/0 conn(0x7f961c0b0330 0x7f961c06d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.415+0000 7f9622315640 1 --2- 172.21.15.94:0/2280659968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961c15c7e0 0x7f961c0afdf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.415+0000 7f9622315640 1 -- 172.21.15.94:0/2280659968 >> 172.21.15.94:0/2280659968 conn(0x7f961c0b47f0 msgr2=0x7f961c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:50.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.415+0000 7f9622315640 1 -- 172.21.15.94:0/2280659968 shutdown_connections 2024-04-25T16:51:50.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.415+0000 7f9622315640 1 -- 172.21.15.94:0/2280659968 wait complete. 2024-04-25T16:51:50.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9622315640 1 Processor -- start 2024-04-25T16:51:50.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9622315640 1 -- start start 2024-04-25T16:51:50.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9622315640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f961c06dc50 0x7f961c14e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9622315640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961c0b0330 0x7f961c157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9622315640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f961c15c7e0 0x7f961c1451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9622315640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f961c06f2c0 con 0x7f961c0b0330 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9622315640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f961c073ca0 con 0x7f961c15c7e0 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9622315640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f961c15cbf0 con 0x7f961c06dc50 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f961b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961c0b0330 0x7f961c157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f961b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961c0b0330 0x7f961c157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59640/0 (socket says 172.21.15.94:59640) 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f961b7fe640 1 -- 172.21.15.94:0/2211015976 learned_addr learned my addr 172.21.15.94:0/2211015976 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f961bfff640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f961c06dc50 0x7f961c14e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f96208b2640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f961c15c7e0 0x7f961c1451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f961b7fe640 1 -- 172.21.15.94:0/2211015976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f961c06dc50 msgr2=0x7f961c14e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f961b7fe640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f961c06dc50 0x7f961c14e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f961b7fe640 1 -- 172.21.15.94:0/2211015976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f961c15c7e0 msgr2=0x7f961c1451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f961b7fe640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f961c15c7e0 0x7f961c1451d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f961b7fe640 1 -- 172.21.15.94:0/2211015976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c14d390 con 0x7f961c0b0330 2024-04-25T16:51:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f961b7fe640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961c0b0330 0x7f961c157b80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f961000ccd0 tx=0x7f961000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9618ff9640 1 -- 172.21.15.94:0/2211015976 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96100039a0 con 0x7f961c0b0330 2024-04-25T16:51:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9618ff9640 1 -- 172.21.15.94:0/2211015976 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9610003b40 con 0x7f961c0b0330 2024-04-25T16:51:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9618ff9640 1 -- 172.21.15.94:0/2211015976 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610004020 con 0x7f961c0b0330 2024-04-25T16:51:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9622315640 1 -- 172.21.15.94:0/2211015976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961c14d890 con 0x7f961c0b0330 2024-04-25T16:51:50.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9622315640 1 -- 172.21.15.94:0/2211015976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961c14de50 con 0x7f961c0b0330 2024-04-25T16:51:50.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.419+0000 7f9622315640 1 -- 172.21.15.94:0/2211015976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f961c076830 con 0x7f961c0b0330 2024-04-25T16:51:50.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.423+0000 7f9618ff9640 1 -- 172.21.15.94:0/2211015976 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f96100041c0 con 0x7f961c0b0330 2024-04-25T16:51:50.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.423+0000 7f9618ff9640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f960806f430 0x7f9608071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.423+0000 7f9618ff9640 1 -- 172.21.15.94:0/2211015976 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f961008fb80 con 0x7f961c0b0330 2024-04-25T16:51:50.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.423+0000 7f961bfff640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f960806f430 0x7f9608071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.423+0000 7f961bfff640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f960806f430 0x7f9608071940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f96140ad000 tx=0x7f9614008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:50.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.427+0000 7f9618ff9640 1 -- 172.21.15.94:0/2211015976 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9610014030 con 0x7f961c0b0330 2024-04-25T16:51:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.587+0000 7f9622315640 1 -- 172.21.15.94:0/2211015976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f961c1601d0 con 0x7f961c0b0330 2024-04-25T16:51:50.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.587+0000 7f9618ff9640 1 -- 172.21.15.94:0/2211015976 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f961005c670 con 0x7f961c0b0330 2024-04-25T16:51:50.586 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:50.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.591+0000 7f9622315640 1 -- 172.21.15.94:0/2211015976 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f960806f430 msgr2=0x7f9608071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.591+0000 7f9622315640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f960806f430 0x7f9608071940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f96140ad000 tx=0x7f9614008040 comp rx=0 tx=0).stop 2024-04-25T16:51:50.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.591+0000 7f9622315640 1 -- 172.21.15.94:0/2211015976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961c0b0330 msgr2=0x7f961c157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.591+0000 7f9622315640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961c0b0330 0x7f961c157b80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f961000ccd0 tx=0x7f961000a5c0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.591+0000 7f9622315640 1 -- 172.21.15.94:0/2211015976 shutdown_connections 2024-04-25T16:51:50.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.591+0000 7f9622315640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f961c15c7e0 0x7f961c1451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.591+0000 7f9622315640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961c0b0330 0x7f961c157b80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.591+0000 7f9622315640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f960806f430 0x7f9608071940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.591+0000 7f9622315640 1 --2- 172.21.15.94:0/2211015976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f961c06dc50 0x7f961c14e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.591+0000 7f9622315640 1 -- 172.21.15.94:0/2211015976 >> 172.21.15.94:0/2211015976 conn(0x7f961c0b47f0 msgr2=0x7f961c0b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:50.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.591+0000 7f9622315640 1 -- 172.21.15.94:0/2211015976 shutdown_connections 2024-04-25T16:51:50.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.591+0000 7f9622315640 1 -- 172.21.15.94:0/2211015976 wait complete. 2024-04-25T16:51:50.643 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-04-25T16:51:50.643 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:50.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff660fee640 1 Processor -- start 2024-04-25T16:51:50.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff660fee640 1 -- start start 2024-04-25T16:51:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff660fee640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff65c070180 0x7ff65c15cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff660fee640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65c160bf0 0x7ff65c06d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff660fee640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff65c15d620 con 0x7ff65c161000 2024-04-25T16:51:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff660fee640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff65c15dec0 con 0x7ff65c070180 2024-04-25T16:51:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff660fee640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff65c160760 con 0x7ff65c160bf0 2024-04-25T16:51:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff659d9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65c160bf0 0x7ff65c06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff65a59c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff65c070180 0x7ff65c15cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff65ad9d640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff65c161000 0x7ff65c0af2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:52448/0 (socket says 172.21.15.94:52448) 2024-04-25T16:51:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff65ad9d640 1 -- 172.21.15.94:0/3305764918 learned_addr learned my addr 172.21.15.94:0/3305764918 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff659d9b640 1 -- 172.21.15.94:0/3305764918 >> v2:172.21.15.138:3300/0 conn(0x7ff65c070180 msgr2=0x7ff65c15cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff659d9b640 1 --2- 172.21.15.94:0/3305764918 >> v2:172.21.15.138:3300/0 conn(0x7ff65c070180 0x7ff65c15cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff659d9b640 1 -- 172.21.15.94:0/3305764918 >> v1:172.21.15.138:6789/0 conn(0x7ff65c161000 legacy=0x7ff65c0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff659d9b640 1 -- 172.21.15.94:0/3305764918 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65c15d900 con 0x7ff65c160bf0 2024-04-25T16:51:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.783+0000 7ff65a59c640 1 --2- 172.21.15.94:0/3305764918 >> v2:172.21.15.138:3300/0 conn(0x7ff65c070180 0x7ff65c15cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.787+0000 7ff659d9b640 1 --2- 172.21.15.94:0/3305764918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65c160bf0 0x7ff65c06d710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff6500097c0 tx=0x7ff65001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eb6041484b40c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.787+0000 7ff65959a640 1 -- 172.21.15.94:0/3305764918 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6500b1070 con 0x7ff65c160bf0 2024-04-25T16:51:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.787+0000 7ff65959a640 1 -- 172.21.15.94:0/3305764918 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6500ac410 con 0x7ff65c160bf0 2024-04-25T16:51:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.787+0000 7ff65959a640 1 -- 172.21.15.94:0/3305764918 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6500ac7e0 con 0x7ff65c160bf0 2024-04-25T16:51:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.787+0000 7ff660fee640 1 -- 172.21.15.94:0/3305764918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65c160bf0 msgr2=0x7ff65c06d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.787+0000 7ff660fee640 1 --2- 172.21.15.94:0/3305764918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65c160bf0 0x7ff65c06d710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff6500097c0 tx=0x7ff65001c4a0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.787+0000 7ff660fee640 1 -- 172.21.15.94:0/3305764918 shutdown_connections 2024-04-25T16:51:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.787+0000 7ff660fee640 1 --2- 172.21.15.94:0/3305764918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65c160bf0 0x7ff65c06d710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.787+0000 7ff660fee640 1 --2- 172.21.15.94:0/3305764918 >> v2:172.21.15.138:3300/0 conn(0x7ff65c070180 0x7ff65c15cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.787+0000 7ff660fee640 1 -- 172.21.15.94:0/3305764918 >> 172.21.15.94:0/3305764918 conn(0x7ff65c0b47f0 msgr2=0x7ff65c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.787+0000 7ff660fee640 1 -- 172.21.15.94:0/3305764918 shutdown_connections 2024-04-25T16:51:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.787+0000 7ff660fee640 1 -- 172.21.15.94:0/3305764918 wait complete. 2024-04-25T16:51:50.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff660fee640 1 Processor -- start 2024-04-25T16:51:50.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff660fee640 1 -- start start 2024-04-25T16:51:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff660fee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff65c070180 0x7ff65c1f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff660fee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff65c160bf0 0x7ff65c14edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff660fee640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65c161000 0x7ff65c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff660fee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c16a2f0 con 0x7ff65c070180 2024-04-25T16:51:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff660fee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c06e060 con 0x7ff65c160bf0 2024-04-25T16:51:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff660fee640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff65c16a470 con 0x7ff65c161000 2024-04-25T16:51:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff65a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff65c070180 0x7ff65c1f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff65a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff65c070180 0x7ff65c1f7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59646/0 (socket says 172.21.15.94:59646) 2024-04-25T16:51:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff65a59c640 1 -- 172.21.15.94:0/4078423061 learned_addr learned my addr 172.21.15.94:0/4078423061 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff65a59c640 1 -- 172.21.15.94:0/4078423061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65c161000 msgr2=0x7ff65c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff65ad9d640 1 --2- 172.21.15.94:0/4078423061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65c161000 0x7ff65c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff65a59c640 1 --2- 172.21.15.94:0/4078423061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65c161000 0x7ff65c0d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff65a59c640 1 -- 172.21.15.94:0/4078423061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff65c160bf0 msgr2=0x7ff65c14edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff65a59c640 1 --2- 172.21.15.94:0/4078423061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff65c160bf0 0x7ff65c14edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff65a59c640 1 -- 172.21.15.94:0/4078423061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65c1f98f0 con 0x7ff65c070180 2024-04-25T16:51:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff65a59c640 1 --2- 172.21.15.94:0/4078423061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff65c070180 0x7ff65c1f7f30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff64c00e930 tx=0x7ff64c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff64affd640 1 -- 172.21.15.94:0/4078423061 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c017070 con 0x7ff65c070180 2024-04-25T16:51:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff64affd640 1 -- 172.21.15.94:0/4078423061 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff64c00ca00 con 0x7ff65c070180 2024-04-25T16:51:50.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff660fee640 1 -- 172.21.15.94:0/4078423061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65c1f9bd0 con 0x7ff65c070180 2024-04-25T16:51:50.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff64affd640 1 -- 172.21.15.94:0/4078423061 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c00cdb0 con 0x7ff65c070180 2024-04-25T16:51:50.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.791+0000 7ff660fee640 1 -- 172.21.15.94:0/4078423061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65c1fa190 con 0x7ff65c070180 2024-04-25T16:51:50.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.795+0000 7ff660fee640 1 -- 172.21.15.94:0/4078423061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff65c076830 con 0x7ff65c070180 2024-04-25T16:51:50.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.795+0000 7ff64affd640 1 -- 172.21.15.94:0/4078423061 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff64c01a070 con 0x7ff65c070180 2024-04-25T16:51:50.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.795+0000 7ff64affd640 1 --2- 172.21.15.94:0/4078423061 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff63c06f400 0x7ff63c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:50.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.795+0000 7ff64affd640 1 -- 172.21.15.94:0/4078423061 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff64c08fcf0 con 0x7ff65c070180 2024-04-25T16:51:50.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.795+0000 7ff659d9b640 1 --2- 172.21.15.94:0/4078423061 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff63c06f400 0x7ff63c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:50.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.795+0000 7ff659d9b640 1 --2- 172.21.15.94:0/4078423061 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff63c06f400 0x7ff63c071910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6500ae000 tx=0x7ff650004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:50.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.799+0000 7ff64affd640 1 -- 172.21.15.94:0/4078423061 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff64c05c8c0 con 0x7ff65c070180 2024-04-25T16:51:50.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.951+0000 7ff660fee640 1 -- 172.21.15.94:0/4078423061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff65c06fa50 con 0x7ff65c070180 2024-04-25T16:51:50.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.951+0000 7ff64affd640 1 -- 172.21.15.94:0/4078423061 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7ff64c05c1c0 con 0x7ff65c070180 2024-04-25T16:51:50.950 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:50.950 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:50.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.955+0000 7ff660fee640 1 -- 172.21.15.94:0/4078423061 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff63c06f400 msgr2=0x7ff63c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.955+0000 7ff660fee640 1 --2- 172.21.15.94:0/4078423061 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff63c06f400 0x7ff63c071910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6500ae000 tx=0x7ff650004740 comp rx=0 tx=0).stop 2024-04-25T16:51:50.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.955+0000 7ff660fee640 1 -- 172.21.15.94:0/4078423061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff65c070180 msgr2=0x7ff65c1f7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:50.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.955+0000 7ff660fee640 1 --2- 172.21.15.94:0/4078423061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff65c070180 0x7ff65c1f7f30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff64c00e930 tx=0x7ff64c00edf0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.955+0000 7ff660fee640 1 -- 172.21.15.94:0/4078423061 shutdown_connections 2024-04-25T16:51:50.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.955+0000 7ff660fee640 1 --2- 172.21.15.94:0/4078423061 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff63c06f400 0x7ff63c071910 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.955+0000 7ff660fee640 1 --2- 172.21.15.94:0/4078423061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff65c161000 0x7ff65c0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.955+0000 7ff660fee640 1 --2- 172.21.15.94:0/4078423061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff65c160bf0 0x7ff65c14edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.955+0000 7ff660fee640 1 --2- 172.21.15.94:0/4078423061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff65c070180 0x7ff65c1f7f30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:50.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.955+0000 7ff660fee640 1 -- 172.21.15.94:0/4078423061 >> 172.21.15.94:0/4078423061 conn(0x7ff65c0b47f0 msgr2=0x7ff65c0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:50.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.955+0000 7ff660fee640 1 -- 172.21.15.94:0/4078423061 shutdown_connections 2024-04-25T16:51:50.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:50.955+0000 7ff660fee640 1 -- 172.21.15.94:0/4078423061 wait complete. 2024-04-25T16:51:50.970 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:51:50.971 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:51:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e61447640 1 Processor -- start 2024-04-25T16:51:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e61447640 1 -- start start 2024-04-25T16:51:51.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e61447640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9e5c143f20 0x7f9e5c0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e61447640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e5c0701b0 0x7f9e5c0705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e61447640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e5c144330 con 0x7f9e5c0b73d0 2024-04-25T16:51:51.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e61447640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9e5c15e980 con 0x7f9e5c143f20 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e61447640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9e5c15ee10 con 0x7f9e5c0701b0 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e5a7fc640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f9e5c0b73d0 0x7f9e5c0b77e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49490/0 (socket says 172.21.15.94:49490) 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e5a7fc640 1 -- 172.21.15.94:0/130229399 learned_addr learned my addr 172.21.15.94:0/130229399 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1640506900 0 0) 0x7f9e5c144330 con 0x7f9e5c0b73d0 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e44003530 con 0x7f9e5c0b73d0 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e5b7fe640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e5c0701b0 0x7f9e5c0705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e5affd640 1 --2- 172.21.15.94:0/130229399 >> v2:172.21.15.138:3300/0 conn(0x7f9e5c143f20 0x7f9e5c0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 763800192 0 0) 0x7f9e44003530 con 0x7f9e5c0b73d0 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e5c0701b0 msgr2=0x7f9e5c0705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.127+0000 7f9e59ffb640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e5c0701b0 0x7f9e5c0705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 >> v2:172.21.15.138:3300/0 conn(0x7f9e5c143f20 msgr2=0x7f9e5c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 --2- 172.21.15.94:0/130229399 >> v2:172.21.15.138:3300/0 conn(0x7f9e5c143f20 0x7f9e5c0d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c15e050 con 0x7f9e5c0b73d0 2024-04-25T16:51:51.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f9e50001680 con 0x7f9e5c0b73d0 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 >> v1:172.21.15.94:6789/0 conn(0x7f9e5c0b73d0 legacy=0x7f9e5c0b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e440039c0 0x7f9e44003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e44004810 0x7f9e44004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e5c0701b0 0x7f9e440051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c15e980 con 0x7f9e440039c0 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c15ee10 con 0x7f9e44004810 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4400bb90 con 0x7f9e5c0701b0 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e5affd640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e440039c0 0x7f9e44003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e5a7fc640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e44004810 0x7f9e44004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e5b7fe640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e5c0701b0 0x7f9e440051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e5affd640 1 -- 172.21.15.94:0/130229399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e5c0701b0 msgr2=0x7f9e440051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e5affd640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e5c0701b0 0x7f9e440051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e5affd640 1 -- 172.21.15.94:0/130229399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e44004810 msgr2=0x7f9e44004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e5affd640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e44004810 0x7f9e44004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e5affd640 1 -- 172.21.15.94:0/130229399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e44003530 con 0x7f9e440039c0 2024-04-25T16:51:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e5b7fe640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e5c0701b0 0x7f9e440051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e5a7fc640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e44004810 0x7f9e44004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e5affd640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e440039c0 0x7f9e44003e30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9e54007330 tx=0x7f9e5401ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75df286c4432f604 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e540b2070 con 0x7f9e440039c0 2024-04-25T16:51:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e540a5d10 con 0x7f9e440039c0 2024-04-25T16:51:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e59ffb640 1 -- 172.21.15.94:0/130229399 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54009be0 con 0x7f9e440039c0 2024-04-25T16:51:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e61447640 1 -- 172.21.15.94:0/130229399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e440039c0 msgr2=0x7f9e44003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e61447640 1 --2- 172.21.15.94:0/130229399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e440039c0 0x7f9e44003e30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9e54007330 tx=0x7f9e5401ce30 comp rx=0 tx=0).stop 2024-04-25T16:51:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e5b7fe640 1 -- 172.21.15.94:0/130229399 reap_dead start 2024-04-25T16:51:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e61447640 1 -- 172.21.15.94:0/130229399 shutdown_connections 2024-04-25T16:51:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e61447640 1 -- 172.21.15.94:0/130229399 >> 172.21.15.94:0/130229399 conn(0x7f9e5c0d33d0 msgr2=0x7f9e5c0d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e61447640 1 -- 172.21.15.94:0/130229399 shutdown_connections 2024-04-25T16:51:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.131+0000 7f9e61447640 1 -- 172.21.15.94:0/130229399 wait complete. 2024-04-25T16:51:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e61447640 1 Processor -- start 2024-04-25T16:51:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e61447640 1 -- start start 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e61447640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e440039c0 0x7f9e5c146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e61447640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c15bf10 0x7f9e5c15b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e61447640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e5c1f74d0 0x7f9e5c1f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e61447640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c070dd0 con 0x7f9e5c15bf10 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e61447640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c0d0ca0 con 0x7f9e5c1f74d0 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e61447640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9e5c070f50 con 0x7f9e440039c0 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e5a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c15bf10 0x7f9e5c15b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e5a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c15bf10 0x7f9e5c15b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59658/0 (socket says 172.21.15.94:59658) 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e5a7fc640 1 -- 172.21.15.94:0/2877997414 learned_addr learned my addr 172.21.15.94:0/2877997414 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e5b7fe640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e5c1f74d0 0x7f9e5c1f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e5a7fc640 1 -- 172.21.15.94:0/2877997414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e440039c0 msgr2=0x7f9e5c146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e5affd640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e440039c0 0x7f9e5c146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e5a7fc640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e440039c0 0x7f9e5c146f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e5a7fc640 1 -- 172.21.15.94:0/2877997414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e5c1f74d0 msgr2=0x7f9e5c1f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e5a7fc640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e5c1f74d0 0x7f9e5c1f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e5a7fc640 1 -- 172.21.15.94:0/2877997414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c145990 con 0x7f9e5c15bf10 2024-04-25T16:51:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e5b7fe640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e5c1f74d0 0x7f9e5c1f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e5a7fc640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c15bf10 0x7f9e5c15b760 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9e50007fd0 tx=0x7f9e5000dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e437fe640 1 -- 172.21.15.94:0/2877997414 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50019070 con 0x7f9e5c15bf10 2024-04-25T16:51:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e437fe640 1 -- 172.21.15.94:0/2877997414 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e500074e0 con 0x7f9e5c15bf10 2024-04-25T16:51:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e61447640 1 -- 172.21.15.94:0/2877997414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5c145e90 con 0x7f9e5c15bf10 2024-04-25T16:51:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e437fe640 1 -- 172.21.15.94:0/2877997414 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e500049f0 con 0x7f9e5c15bf10 2024-04-25T16:51:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.135+0000 7f9e61447640 1 -- 172.21.15.94:0/2877997414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5c146450 con 0x7f9e5c15bf10 2024-04-25T16:51:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.139+0000 7f9e61447640 1 -- 172.21.15.94:0/2877997414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e5c145be0 con 0x7f9e5c15bf10 2024-04-25T16:51:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.139+0000 7f9e437fe640 1 -- 172.21.15.94:0/2877997414 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9e5001c070 con 0x7f9e5c15bf10 2024-04-25T16:51:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.139+0000 7f9e437fe640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9e4806f430 0x7f9e48071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.139+0000 7f9e437fe640 1 -- 172.21.15.94:0/2877997414 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9e5008f170 con 0x7f9e5c15bf10 2024-04-25T16:51:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.139+0000 7f9e5affd640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9e4806f430 0x7f9e48071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.139+0000 7f9e5affd640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9e4806f430 0x7f9e48071940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e540af000 tx=0x7f9e54004a50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:51.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.143+0000 7f9e437fe640 1 -- 172.21.15.94:0/2877997414 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9e5005bcf0 con 0x7f9e5c15bf10 2024-04-25T16:51:51.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.307+0000 7f9e61447640 1 -- 172.21.15.94:0/2877997414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9e5c06a730 con 0x7f9e5c15bf10 2024-04-25T16:51:51.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.307+0000 7f9e437fe640 1 -- 172.21.15.94:0/2877997414 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f9e5005b5f0 con 0x7f9e5c15bf10 2024-04-25T16:51:51.306 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:51:51.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.311+0000 7f9e61447640 1 -- 172.21.15.94:0/2877997414 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9e4806f430 msgr2=0x7f9e48071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.311+0000 7f9e61447640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9e4806f430 0x7f9e48071940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e540af000 tx=0x7f9e54004a50 comp rx=0 tx=0).stop 2024-04-25T16:51:51.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.311+0000 7f9e61447640 1 -- 172.21.15.94:0/2877997414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c15bf10 msgr2=0x7f9e5c15b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.311+0000 7f9e61447640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c15bf10 0x7f9e5c15b760 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9e50007fd0 tx=0x7f9e5000dc70 comp rx=0 tx=0).stop 2024-04-25T16:51:51.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.311+0000 7f9e61447640 1 -- 172.21.15.94:0/2877997414 shutdown_connections 2024-04-25T16:51:51.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.311+0000 7f9e61447640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e5c1f74d0 0x7f9e5c1f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.311+0000 7f9e61447640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c15bf10 0x7f9e5c15b760 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.311+0000 7f9e61447640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9e4806f430 0x7f9e48071940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.311+0000 7f9e61447640 1 --2- 172.21.15.94:0/2877997414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e440039c0 0x7f9e5c146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.311+0000 7f9e61447640 1 -- 172.21.15.94:0/2877997414 >> 172.21.15.94:0/2877997414 conn(0x7f9e5c0d33d0 msgr2=0x7f9e5c15cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:51.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.311+0000 7f9e61447640 1 -- 172.21.15.94:0/2877997414 shutdown_connections 2024-04-25T16:51:51.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.311+0000 7f9e61447640 1 -- 172.21.15.94:0/2877997414 wait complete. 2024-04-25T16:51:51.327 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:51:51.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.467+0000 7ff4015a3640 1 Processor -- start 2024-04-25T16:51:51.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.467+0000 7ff4015a3640 1 -- start start 2024-04-25T16:51:51.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.467+0000 7ff4015a3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff3fc0b7050 0x7ff3fc15fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.467+0000 7ff4015a3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3fc0b7460 con 0x7ff3fc0afc90 2024-04-25T16:51:51.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.467+0000 7ff4015a3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3fc0b18d0 con 0x7ff3fc0b5130 2024-04-25T16:51:51.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.467+0000 7ff4015a3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff3fc1644d0 con 0x7ff3fc0b7050 2024-04-25T16:51:51.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.467+0000 7ff3fb7fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ff3fc0afc90 0x7ff3fc0b00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49506/0 (socket says 172.21.15.94:49506) 2024-04-25T16:51:51.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.467+0000 7ff3fb7fe640 1 -- 172.21.15.94:0/560820973 learned_addr learned my addr 172.21.15.94:0/560820973 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:51.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.467+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3167670555 0 0) 0x7ff3ec001120 con 0x7ff3fc0afc90 2024-04-25T16:51:51.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3e4003530 con 0x7ff3fc0afc90 2024-04-25T16:51:51.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3faffd640 1 --2- 172.21.15.94:0/560820973 >> v2:172.21.15.138:3300/0 conn(0x7ff3fc0b7050 0x7ff3fc15fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1250030152 0 0) 0x7ff3e4003530 con 0x7ff3fc0afc90 2024-04-25T16:51:51.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 >> v2:172.21.15.138:3300/0 conn(0x7ff3fc0b7050 msgr2=0x7ff3fc15fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:51.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 --2- 172.21.15.94:0/560820973 >> v2:172.21.15.138:3300/0 conn(0x7ff3fc0b7050 0x7ff3fc15fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 >> v1:172.21.15.138:6789/0 conn(0x7ff3fc0b5130 legacy=0x7ff3fc0b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3fc0af1b0 con 0x7ff3fc0afc90 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7ff3ec001850 con 0x7ff3fc0afc90 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 >> v1:172.21.15.94:6789/0 conn(0x7ff3fc0afc90 legacy=0x7ff3fc0b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 --2- 172.21.15.94:0/560820973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e40039c0 0x7ff3e4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 --2- 172.21.15.94:0/560820973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3e4004810 0x7ff3e4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 --2- 172.21.15.94:0/560820973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3e4005310 0x7ff3e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1644d0 con 0x7ff3e40039c0 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e400c2b0 con 0x7ff3e4005310 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e400c4b0 con 0x7ff3e4004810 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3faffd640 1 --2- 172.21.15.94:0/560820973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e40039c0 0x7ff3e4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3fb7fe640 1 --2- 172.21.15.94:0/560820973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3e4005310 0x7ff3e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3faffd640 1 -- 172.21.15.94:0/560820973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3e4004810 msgr2=0x7ff3e4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:51.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3faffd640 1 --2- 172.21.15.94:0/560820973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3e4004810 0x7ff3e4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3faffd640 1 -- 172.21.15.94:0/560820973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3e4005310 msgr2=0x7ff3e400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:51.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3faffd640 1 --2- 172.21.15.94:0/560820973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3e4005310 0x7ff3e400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3faffd640 1 -- 172.21.15.94:0/560820973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e4003530 con 0x7ff3e40039c0 2024-04-25T16:51:51.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3fb7fe640 1 -- 172.21.15.94:0/560820973 reap_dead start 2024-04-25T16:51:51.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3faffd640 1 --2- 172.21.15.94:0/560820973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e40039c0 0x7ff3e4003e30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4007360 tx=0x7ff3f401ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d6abf0420d8f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:51.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f40b2070 con 0x7ff3e40039c0 2024-04-25T16:51:51.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f40a5c30 con 0x7ff3e40039c0 2024-04-25T16:51:51.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff3f9ffb640 1 -- 172.21.15.94:0/560820973 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4009be0 con 0x7ff3e40039c0 2024-04-25T16:51:51.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff4015a3640 1 -- 172.21.15.94:0/560820973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e40039c0 msgr2=0x7ff3e4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff4015a3640 1 --2- 172.21.15.94:0/560820973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e40039c0 0x7ff3e4003e30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4007360 tx=0x7ff3f401ce40 comp rx=0 tx=0).stop 2024-04-25T16:51:51.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff4015a3640 1 -- 172.21.15.94:0/560820973 shutdown_connections 2024-04-25T16:51:51.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff4015a3640 1 --2- 172.21.15.94:0/560820973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e40039c0 0x7ff3e4003e30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.471+0000 7ff4015a3640 1 -- 172.21.15.94:0/560820973 >> 172.21.15.94:0/560820973 conn(0x7ff3fc0b7690 msgr2=0x7ff3fc0b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:51.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff4015a3640 1 -- 172.21.15.94:0/560820973 shutdown_connections 2024-04-25T16:51:51.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff4015a3640 1 -- 172.21.15.94:0/560820973 wait complete. 2024-04-25T16:51:51.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff4015a3640 1 Processor -- start 2024-04-25T16:51:51.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff4015a3640 1 -- start start 2024-04-25T16:51:51.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff4015a3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3e40039c0 0x7ff3fc15dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff4015a3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3fc15e320 0x7ff3fc0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff4015a3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3fc1fab60 0x7ff3fc0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff4015a3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc0b5950 con 0x7ff3fc15e320 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff4015a3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1647b0 con 0x7ff3e40039c0 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff4015a3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff3fc1fb2d0 con 0x7ff3fc1fab60 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff3fa7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3fc15e320 0x7ff3fc0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff3fa7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3fc15e320 0x7ff3fc0cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59670/0 (socket says 172.21.15.94:59670) 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff3fa7fc640 1 -- 172.21.15.94:0/3716394546 learned_addr learned my addr 172.21.15.94:0/3716394546 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff3faffd640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3e40039c0 0x7ff3fc15dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff3fb7fe640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3fc1fab60 0x7ff3fc0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff3fa7fc640 1 -- 172.21.15.94:0/3716394546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3fc1fab60 msgr2=0x7ff3fc0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff3fa7fc640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3fc1fab60 0x7ff3fc0cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff3fa7fc640 1 -- 172.21.15.94:0/3716394546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3e40039c0 msgr2=0x7ff3fc15dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff3fa7fc640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3e40039c0 0x7ff3fc15dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff3fa7fc640 1 -- 172.21.15.94:0/3716394546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3fc0cb690 con 0x7ff3fc15e320 2024-04-25T16:51:51.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.475+0000 7ff3fa7fc640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3fc15e320 0x7ff3fc0cc450 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000c4d0 tx=0x7ff3f000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:51.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.479+0000 7ff3e37fe640 1 -- 172.21.15.94:0/3716394546 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f00052d0 con 0x7ff3fc15e320 2024-04-25T16:51:51.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.479+0000 7ff3e37fe640 1 -- 172.21.15.94:0/3716394546 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f0005470 con 0x7ff3fc15e320 2024-04-25T16:51:51.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.479+0000 7ff4015a3640 1 -- 172.21.15.94:0/3716394546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3fc0cb970 con 0x7ff3fc15e320 2024-04-25T16:51:51.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.479+0000 7ff3e37fe640 1 -- 172.21.15.94:0/3716394546 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0012870 con 0x7ff3fc15e320 2024-04-25T16:51:51.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.479+0000 7ff4015a3640 1 -- 172.21.15.94:0/3716394546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3fc0cbf30 con 0x7ff3fc15e320 2024-04-25T16:51:51.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.479+0000 7ff4015a3640 1 -- 172.21.15.94:0/3716394546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3fc076830 con 0x7ff3fc15e320 2024-04-25T16:51:51.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.479+0000 7ff3e37fe640 1 -- 172.21.15.94:0/3716394546 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff3f0012a10 con 0x7ff3fc15e320 2024-04-25T16:51:51.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.479+0000 7ff3e37fe640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff3e806f400 0x7ff3e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.479+0000 7ff3faffd640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff3e806f400 0x7ff3e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.479+0000 7ff3e37fe640 1 -- 172.21.15.94:0/3716394546 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff3f008f3b0 con 0x7ff3fc15e320 2024-04-25T16:51:51.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.479+0000 7ff3faffd640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff3e806f400 0x7ff3e8071910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40af000 tx=0x7ff3f4004a50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:51.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.483+0000 7ff3e37fe640 1 -- 172.21.15.94:0/3716394546 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff3f005bea0 con 0x7ff3fc15e320 2024-04-25T16:51:51.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.643+0000 7ff4015a3640 1 -- 172.21.15.94:0/3716394546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff3fc0714e0 con 0x7ff3fc15e320 2024-04-25T16:51:51.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.643+0000 7ff3e37fe640 1 -- 172.21.15.94:0/3716394546 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7ff3f005b7a0 con 0x7ff3fc15e320 2024-04-25T16:51:51.642 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:51.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.647+0000 7ff4015a3640 1 -- 172.21.15.94:0/3716394546 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff3e806f400 msgr2=0x7ff3e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.647+0000 7ff4015a3640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff3e806f400 0x7ff3e8071910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40af000 tx=0x7ff3f4004a50 comp rx=0 tx=0).stop 2024-04-25T16:51:51.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.647+0000 7ff4015a3640 1 -- 172.21.15.94:0/3716394546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3fc15e320 msgr2=0x7ff3fc0cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.647+0000 7ff4015a3640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3fc15e320 0x7ff3fc0cc450 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000c4d0 tx=0x7ff3f000c990 comp rx=0 tx=0).stop 2024-04-25T16:51:51.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.647+0000 7ff4015a3640 1 -- 172.21.15.94:0/3716394546 shutdown_connections 2024-04-25T16:51:51.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.647+0000 7ff4015a3640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff3e806f400 0x7ff3e8071910 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.647+0000 7ff4015a3640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3fc1fab60 0x7ff3fc0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.647+0000 7ff4015a3640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3fc15e320 0x7ff3fc0cc450 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.647+0000 7ff4015a3640 1 --2- 172.21.15.94:0/3716394546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3e40039c0 0x7ff3fc15dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.647+0000 7ff4015a3640 1 -- 172.21.15.94:0/3716394546 >> 172.21.15.94:0/3716394546 conn(0x7ff3fc0b7690 msgr2=0x7ff3fc15f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:51.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.647+0000 7ff4015a3640 1 -- 172.21.15.94:0/3716394546 shutdown_connections 2024-04-25T16:51:51.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.647+0000 7ff4015a3640 1 -- 172.21.15.94:0/3716394546 wait complete. 2024-04-25T16:51:51.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:51:51.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.819+0000 7f6671dfb640 1 Processor -- start 2024-04-25T16:51:51.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.819+0000 7f6671dfb640 1 -- start start 2024-04-25T16:51:51.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.819+0000 7f6671dfb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666c15c7e0 0x7f666c0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.819+0000 7f6671dfb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f666c0b0330 0x7f666c06d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.819+0000 7f6671dfb640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666c074380 con 0x7f666c06dc50 2024-04-25T16:51:51.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.819+0000 7f6671dfb640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f666c06e060 con 0x7f666c0b0330 2024-04-25T16:51:51.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.819+0000 7f6671dfb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f666c0727e0 con 0x7f666c15c7e0 2024-04-25T16:51:51.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f666affd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f666c0b0330 0x7f666c06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f666affd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f666c0b0330 0x7f666c06d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59678/0 (socket says 172.21.15.94:59678) 2024-04-25T16:51:51.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f666affd640 1 -- 172.21.15.94:0/731638535 learned_addr learned my addr 172.21.15.94:0/731638535 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:51.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f666b7fe640 1 --2- 172.21.15.94:0/731638535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666c15c7e0 0x7f666c0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f666affd640 1 -- 172.21.15.94:0/731638535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666c15c7e0 msgr2=0x7f666c0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:51.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f666affd640 1 --2- 172.21.15.94:0/731638535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666c15c7e0 0x7f666c0afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f666affd640 1 -- 172.21.15.94:0/731638535 >> v1:172.21.15.138:6789/0 conn(0x7f666c06dc50 legacy=0x7f666c06e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f666affd640 1 -- 172.21.15.94:0/731638535 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c072ac0 con 0x7f666c0b0330 2024-04-25T16:51:51.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f666affd640 1 --2- 172.21.15.94:0/731638535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0b0330 0x7f666c06d710 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6660009870 tx=0x7f666001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9f0a58b247586c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:51.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f666a7fc640 1 -- 172.21.15.94:0/731638535 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66600a95f0 con 0x7f666c0b0330 2024-04-25T16:51:51.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f666a7fc640 1 -- 172.21.15.94:0/731638535 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66600a9790 con 0x7f666c0b0330 2024-04-25T16:51:51.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f666a7fc640 1 -- 172.21.15.94:0/731638535 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66600a9b60 con 0x7f666c0b0330 2024-04-25T16:51:51.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f6671dfb640 1 -- 172.21.15.94:0/731638535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0b0330 msgr2=0x7f666c06d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f6671dfb640 1 --2- 172.21.15.94:0/731638535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0b0330 0x7f666c06d710 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6660009870 tx=0x7f666001c530 comp rx=0 tx=0).stop 2024-04-25T16:51:51.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f6671dfb640 1 -- 172.21.15.94:0/731638535 shutdown_connections 2024-04-25T16:51:51.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f6671dfb640 1 --2- 172.21.15.94:0/731638535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0b0330 0x7f666c06d710 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f6671dfb640 1 --2- 172.21.15.94:0/731638535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666c15c7e0 0x7f666c0afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f6671dfb640 1 -- 172.21.15.94:0/731638535 >> 172.21.15.94:0/731638535 conn(0x7f666c0b47f0 msgr2=0x7f666c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:51.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f6671dfb640 1 -- 172.21.15.94:0/731638535 shutdown_connections 2024-04-25T16:51:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.823+0000 7f6671dfb640 1 -- 172.21.15.94:0/731638535 wait complete. 2024-04-25T16:51:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f6671dfb640 1 Processor -- start 2024-04-25T16:51:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f6671dfb640 1 -- start start 2024-04-25T16:51:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f6671dfb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666c06dc50 0x7f666c148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f6671dfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0b0330 0x7f666c0d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f6671dfb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666c15c7e0 0x7f666c1603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f6671dfb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f666c06f2c0 con 0x7f666c0b0330 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f6671dfb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f666c073ca0 con 0x7f666c15c7e0 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f6671dfb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f666c15cbf0 con 0x7f666c06dc50 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f666affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0b0330 0x7f666c0d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f666affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0b0330 0x7f666c0d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59690/0 (socket says 172.21.15.94:59690) 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f666affd640 1 -- 172.21.15.94:0/1363431492 learned_addr learned my addr 172.21.15.94:0/1363431492 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f666bfff640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666c15c7e0 0x7f666c1603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f666b7fe640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666c06dc50 0x7f666c148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f666affd640 1 -- 172.21.15.94:0/1363431492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666c06dc50 msgr2=0x7f666c148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f666affd640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666c06dc50 0x7f666c148410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f666affd640 1 -- 172.21.15.94:0/1363431492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666c15c7e0 msgr2=0x7f666c1603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f666affd640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666c15c7e0 0x7f666c1603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f666affd640 1 -- 172.21.15.94:0/1363431492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c15fb80 con 0x7f666c0b0330 2024-04-25T16:51:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f666affd640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0b0330 0x7f666c0d19e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f666001ca10 tx=0x7f66600192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f6653fff640 1 -- 172.21.15.94:0/1363431492 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66600b1800 con 0x7f666c0b0330 2024-04-25T16:51:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f6671dfb640 1 -- 172.21.15.94:0/1363431492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666c06c7a0 con 0x7f666c0b0330 2024-04-25T16:51:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f6653fff640 1 -- 172.21.15.94:0/1363431492 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66600b19a0 con 0x7f666c0b0330 2024-04-25T16:51:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f6653fff640 1 -- 172.21.15.94:0/1363431492 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66600b1d70 con 0x7f666c0b0330 2024-04-25T16:51:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.827+0000 7f6671dfb640 1 -- 172.21.15.94:0/1363431492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666c157a50 con 0x7f666c0b0330 2024-04-25T16:51:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.831+0000 7f6671dfb640 1 -- 172.21.15.94:0/1363431492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f666c076830 con 0x7f666c0b0330 2024-04-25T16:51:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.831+0000 7f6653fff640 1 -- 172.21.15.94:0/1363431492 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f66600b93e0 con 0x7f666c0b0330 2024-04-25T16:51:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.831+0000 7f6653fff640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f665806f430 0x7f6658071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.831+0000 7f6653fff640 1 -- 172.21.15.94:0/1363431492 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6660129bd0 con 0x7f666c0b0330 2024-04-25T16:51:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.831+0000 7f666b7fe640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f665806f430 0x7f6658071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.831+0000 7f666b7fe640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f665806f430 0x7f6658071940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f665c0096f0 tx=0x7f665c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.835+0000 7f6653fff640 1 -- 172.21.15.94:0/1363431492 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f66600ad020 con 0x7f666c0b0330 2024-04-25T16:51:51.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.995+0000 7f6671dfb640 1 -- 172.21.15.94:0/1363431492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f666c1601d0 con 0x7f666c0b0330 2024-04-25T16:51:51.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.995+0000 7f6653fff640 1 -- 172.21.15.94:0/1363431492 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f6660016150 con 0x7f666c0b0330 2024-04-25T16:51:51.994 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:51:51.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.999+0000 7f6671dfb640 1 -- 172.21.15.94:0/1363431492 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f665806f430 msgr2=0x7f6658071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.999+0000 7f6671dfb640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f665806f430 0x7f6658071940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f665c0096f0 tx=0x7f665c008040 comp rx=0 tx=0).stop 2024-04-25T16:51:51.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.999+0000 7f6671dfb640 1 -- 172.21.15.94:0/1363431492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0b0330 msgr2=0x7f666c0d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:51.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.999+0000 7f6671dfb640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0b0330 0x7f666c0d19e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f666001ca10 tx=0x7f66600192d0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.999+0000 7f6671dfb640 1 -- 172.21.15.94:0/1363431492 shutdown_connections 2024-04-25T16:51:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.999+0000 7f6671dfb640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666c15c7e0 0x7f666c1603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.999+0000 7f6671dfb640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0b0330 0x7f666c0d19e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.999+0000 7f6671dfb640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f665806f430 0x7f6658071940 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.999+0000 7f6671dfb640 1 --2- 172.21.15.94:0/1363431492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666c06dc50 0x7f666c148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.999+0000 7f6671dfb640 1 -- 172.21.15.94:0/1363431492 >> 172.21.15.94:0/1363431492 conn(0x7f666c0b47f0 msgr2=0x7f666c1568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.999+0000 7f6671dfb640 1 -- 172.21.15.94:0/1363431492 shutdown_connections 2024-04-25T16:51:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:51.999+0000 7f6671dfb640 1 -- 172.21.15.94:0/1363431492 wait complete. 2024-04-25T16:51:52.015 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:51:52.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.163+0000 7f802684c640 1 Processor -- start 2024-04-25T16:51:52.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.167+0000 7f802684c640 1 -- start start 2024-04-25T16:51:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.167+0000 7f802684c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f802006d710 0x7f802006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.167+0000 7f802684c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8020074380 con 0x7f802006e7a0 2024-04-25T16:51:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.167+0000 7f802684c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f802006e060 con 0x7f802015c7e0 2024-04-25T16:51:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.167+0000 7f802684c640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f80200727e0 con 0x7f802006d710 2024-04-25T16:51:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.167+0000 7f801f7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f802006d710 0x7f802006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.167+0000 7f8024de9640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f802006e7a0 0x7f802006ebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49518/0 (socket says 172.21.15.94:49518) 2024-04-25T16:51:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.167+0000 7f8024de9640 1 -- 172.21.15.94:0/1655357127 learned_addr learned my addr 172.21.15.94:0/1655357127 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.167+0000 7f801f7fe640 1 -- 172.21.15.94:0/1655357127 >> v1:172.21.15.138:6789/0 conn(0x7f802015c7e0 legacy=0x7f80200afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.167+0000 7f801f7fe640 1 -- 172.21.15.94:0/1655357127 >> v1:172.21.15.94:6789/0 conn(0x7f802006e7a0 legacy=0x7f802006ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.167+0000 7f801f7fe640 1 -- 172.21.15.94:0/1655357127 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8020072ac0 con 0x7f802006d710 2024-04-25T16:51:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.167+0000 7f801effd640 1 -- 172.21.15.94:0/1655357127 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2306257286 0 0) 0x7f80180014a0 con 0x7f802006e7a0 2024-04-25T16:51:52.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.171+0000 7f801f7fe640 1 --2- 172.21.15.94:0/1655357127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f802006d710 0x7f802006db20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f80140097c0 tx=0x7f801401c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972ed21e2910bef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:52.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.171+0000 7f801effd640 1 -- 172.21.15.94:0/1655357127 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140b0070 con 0x7f802006d710 2024-04-25T16:51:52.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.171+0000 7f801effd640 1 -- 172.21.15.94:0/1655357127 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80140a4e50 con 0x7f802006d710 2024-04-25T16:51:52.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.171+0000 7f801effd640 1 -- 172.21.15.94:0/1655357127 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140ab820 con 0x7f802006d710 2024-04-25T16:51:52.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.171+0000 7f802684c640 1 -- 172.21.15.94:0/1655357127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f802006d710 msgr2=0x7f802006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:52.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.171+0000 7f802684c640 1 --2- 172.21.15.94:0/1655357127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f802006d710 0x7f802006db20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f80140097c0 tx=0x7f801401c1c0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.171+0000 7f802684c640 1 -- 172.21.15.94:0/1655357127 shutdown_connections 2024-04-25T16:51:52.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.171+0000 7f802684c640 1 --2- 172.21.15.94:0/1655357127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f802006d710 0x7f802006db20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.171+0000 7f802684c640 1 -- 172.21.15.94:0/1655357127 >> 172.21.15.94:0/1655357127 conn(0x7f80200b47f0 msgr2=0x7f80200b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:52.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.171+0000 7f802684c640 1 -- 172.21.15.94:0/1655357127 shutdown_connections 2024-04-25T16:51:52.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.171+0000 7f802684c640 1 -- 172.21.15.94:0/1655357127 wait complete. 2024-04-25T16:51:52.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.171+0000 7f802684c640 1 Processor -- start 2024-04-25T16:51:52.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.171+0000 7f802684c640 1 -- start start 2024-04-25T16:51:52.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f802684c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f802006d710 0x7f8020148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:52.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f802684c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802006e7a0 0x7f80201568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:52.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f802684c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802015c7e0 0x7f802015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:52.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f802684c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f802006f2c0 con 0x7f802006d710 2024-04-25T16:51:52.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f802684c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8020073ca0 con 0x7f802006e7a0 2024-04-25T16:51:52.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f802684c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f802015cbf0 con 0x7f802015c7e0 2024-04-25T16:51:52.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f801ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f802006d710 0x7f8020148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:52.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f801ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f802006d710 0x7f8020148ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59712/0 (socket says 172.21.15.94:59712) 2024-04-25T16:51:52.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f801ffff640 1 -- 172.21.15.94:0/1682486636 learned_addr learned my addr 172.21.15.94:0/1682486636 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:52.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f801ffff640 1 -- 172.21.15.94:0/1682486636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802015c7e0 msgr2=0x7f802015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:52.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f801ffff640 1 --2- 172.21.15.94:0/1682486636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802015c7e0 0x7f802015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f801ffff640 1 -- 172.21.15.94:0/1682486636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802006e7a0 msgr2=0x7f80201568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:52.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f801ffff640 1 --2- 172.21.15.94:0/1682486636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802006e7a0 0x7f80201568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f801ffff640 1 -- 172.21.15.94:0/1682486636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8020149400 con 0x7f802006d710 2024-04-25T16:51:52.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f801ffff640 1 --2- 172.21.15.94:0/1682486636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f802006d710 0x7f8020148ec0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f801000b440 tx=0x7f801000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:52.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f801cff9640 1 -- 172.21.15.94:0/1682486636 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010015020 con 0x7f802006d710 2024-04-25T16:51:52.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f801cff9640 1 -- 172.21.15.94:0/1682486636 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8010011830 con 0x7f802006d710 2024-04-25T16:51:52.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f801cff9640 1 -- 172.21.15.94:0/1682486636 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010011c20 con 0x7f802006d710 2024-04-25T16:51:52.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f802684c640 1 -- 172.21.15.94:0/1682486636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8020157320 con 0x7f802006d710 2024-04-25T16:51:52.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f802684c640 1 -- 172.21.15.94:0/1682486636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80201578e0 con 0x7f802006d710 2024-04-25T16:51:52.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.175+0000 7f802684c640 1 -- 172.21.15.94:0/1682486636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8020076830 con 0x7f802006d710 2024-04-25T16:51:52.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.179+0000 7f801cff9640 1 -- 172.21.15.94:0/1682486636 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8010018020 con 0x7f802006d710 2024-04-25T16:51:52.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.179+0000 7f801cff9640 1 --2- 172.21.15.94:0/1682486636 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f800c06f400 0x7f800c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:52.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.179+0000 7f801f7fe640 1 --2- 172.21.15.94:0/1682486636 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f800c06f400 0x7f800c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:52.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.179+0000 7f801cff9640 1 -- 172.21.15.94:0/1682486636 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f801008f2a0 con 0x7f802006d710 2024-04-25T16:51:52.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.179+0000 7f801f7fe640 1 --2- 172.21.15.94:0/1682486636 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f800c06f400 0x7f800c071910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f80140ad000 tx=0x7f8014008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:52.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.183+0000 7f801cff9640 1 -- 172.21.15.94:0/1682486636 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f801001f030 con 0x7f802006d710 2024-04-25T16:51:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.339+0000 7f802684c640 1 -- 172.21.15.94:0/1682486636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f80201601d0 con 0x7f802006d710 2024-04-25T16:51:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.339+0000 7f801cff9640 1 -- 172.21.15.94:0/1682486636 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f801005be20 con 0x7f802006d710 2024-04-25T16:51:52.339 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:52.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.343+0000 7f802684c640 1 -- 172.21.15.94:0/1682486636 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f800c06f400 msgr2=0x7f800c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:52.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.343+0000 7f802684c640 1 --2- 172.21.15.94:0/1682486636 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f800c06f400 0x7f800c071910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f80140ad000 tx=0x7f8014008040 comp rx=0 tx=0).stop 2024-04-25T16:51:52.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.343+0000 7f802684c640 1 -- 172.21.15.94:0/1682486636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f802006d710 msgr2=0x7f8020148ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.343+0000 7f802684c640 1 --2- 172.21.15.94:0/1682486636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f802006d710 0x7f8020148ec0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f801000b440 tx=0x7f801000b900 comp rx=0 tx=0).stop 2024-04-25T16:51:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.343+0000 7f802684c640 1 -- 172.21.15.94:0/1682486636 shutdown_connections 2024-04-25T16:51:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.343+0000 7f802684c640 1 --2- 172.21.15.94:0/1682486636 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f800c06f400 0x7f800c071910 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.343+0000 7f802684c640 1 --2- 172.21.15.94:0/1682486636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802015c7e0 0x7f802015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.343+0000 7f802684c640 1 --2- 172.21.15.94:0/1682486636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802006e7a0 0x7f80201568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.343+0000 7f802684c640 1 --2- 172.21.15.94:0/1682486636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f802006d710 0x7f8020148ec0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.343+0000 7f802684c640 1 -- 172.21.15.94:0/1682486636 >> 172.21.15.94:0/1682486636 conn(0x7f80200b47f0 msgr2=0x7f80200d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.343+0000 7f802684c640 1 -- 172.21.15.94:0/1682486636 shutdown_connections 2024-04-25T16:51:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.343+0000 7f802684c640 1 -- 172.21.15.94:0/1682486636 wait complete. 2024-04-25T16:51:52.360 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-04-25T16:51:52.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feacd9fa640 1 Processor -- start 2024-04-25T16:51:52.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feacd9fa640 1 -- start start 2024-04-25T16:51:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feacd9fa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feac806e960 0x7feac8072670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feacd9fa640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7feac8072bb0 0x7feac80ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feacd9fa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feac80ce880 0x7feac80cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feacd9fa640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7feac80d2720 con 0x7feac8072bb0 2024-04-25T16:51:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feacd9fa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feac8072fc0 con 0x7feac806e960 2024-04-25T16:51:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feacd9fa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feac80d3230 con 0x7feac80ce880 2024-04-25T16:51:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feabffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7feac8072bb0 0x7feac80ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feabffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7feac8072bb0 0x7feac80ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59722/0 (socket says 172.21.15.94:59722) 2024-04-25T16:51:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feabffff640 1 -- 172.21.15.94:0/3078204600 learned_addr learned my addr 172.21.15.94:0/3078204600 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feac6ffd640 1 --2- 172.21.15.94:0/3078204600 >> v2:172.21.15.138:3300/0 conn(0x7feac806e960 0x7feac8072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feac77fe640 1 --2- 172.21.15.94:0/3078204600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feac80ce880 0x7feac80cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feabffff640 1 -- 172.21.15.94:0/3078204600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feac80ce880 msgr2=0x7feac80cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feabffff640 1 --2- 172.21.15.94:0/3078204600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feac80ce880 0x7feac80cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feabffff640 1 -- 172.21.15.94:0/3078204600 >> v2:172.21.15.138:3300/0 conn(0x7feac806e960 msgr2=0x7feac8072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feabffff640 1 --2- 172.21.15.94:0/3078204600 >> v2:172.21.15.138:3300/0 conn(0x7feac806e960 0x7feac8072670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feabffff640 1 -- 172.21.15.94:0/3078204600 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac80d2990 con 0x7feac8072bb0 2024-04-25T16:51:52.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.511+0000 7feac6ffd640 1 --2- 172.21.15.94:0/3078204600 >> v2:172.21.15.138:3300/0 conn(0x7feac806e960 0x7feac8072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:52.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feabffff640 1 --2- 172.21.15.94:0/3078204600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac8072bb0 0x7feac80ce340 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feab0009870 tx=0x7feab001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf4dc27c1723505 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:52.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feac67fc640 1 -- 172.21.15.94:0/3078204600 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab00ab620 con 0x7feac8072bb0 2024-04-25T16:51:52.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feac67fc640 1 -- 172.21.15.94:0/3078204600 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab00b0070 con 0x7feac8072bb0 2024-04-25T16:51:52.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feac67fc640 1 -- 172.21.15.94:0/3078204600 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab00ab9e0 con 0x7feac8072bb0 2024-04-25T16:51:52.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feacd9fa640 1 -- 172.21.15.94:0/3078204600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac8072bb0 msgr2=0x7feac80ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:52.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feacd9fa640 1 --2- 172.21.15.94:0/3078204600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac8072bb0 0x7feac80ce340 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feab0009870 tx=0x7feab001c460 comp rx=0 tx=0).stop 2024-04-25T16:51:52.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feacd9fa640 1 -- 172.21.15.94:0/3078204600 shutdown_connections 2024-04-25T16:51:52.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feacd9fa640 1 --2- 172.21.15.94:0/3078204600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feac80ce880 0x7feac80cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feacd9fa640 1 --2- 172.21.15.94:0/3078204600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac8072bb0 0x7feac80ce340 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feacd9fa640 1 --2- 172.21.15.94:0/3078204600 >> v2:172.21.15.138:3300/0 conn(0x7feac806e960 0x7feac8072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feacd9fa640 1 -- 172.21.15.94:0/3078204600 >> 172.21.15.94:0/3078204600 conn(0x7feac80d3850 msgr2=0x7feac80d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:52.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feacd9fa640 1 -- 172.21.15.94:0/3078204600 shutdown_connections 2024-04-25T16:51:52.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feacd9fa640 1 -- 172.21.15.94:0/3078204600 wait complete. 2024-04-25T16:51:52.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feacd9fa640 1 Processor -- start 2024-04-25T16:51:52.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.515+0000 7feacd9fa640 1 -- start start 2024-04-25T16:51:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feacd9fa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac806e960 0x7feac80b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feacd9fa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feac8072bb0 0x7feac80c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feacd9fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac80ce880 0x7feac80ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feacd9fa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feac806ed70 con 0x7feac80ce880 2024-04-25T16:51:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feacd9fa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feac80d3010 con 0x7feac806e960 2024-04-25T16:51:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feacd9fa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feac806eef0 con 0x7feac8072bb0 2024-04-25T16:51:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac80ce880 0x7feac80ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac80ce880 0x7feac80ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59724/0 (socket says 172.21.15.94:59724) 2024-04-25T16:51:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac77fe640 1 -- 172.21.15.94:0/3216865035 learned_addr learned my addr 172.21.15.94:0/3216865035 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac6ffd640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac806e960 0x7feac80b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feabffff640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feac8072bb0 0x7feac80c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac77fe640 1 -- 172.21.15.94:0/3216865035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feac8072bb0 msgr2=0x7feac80c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac77fe640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feac8072bb0 0x7feac80c7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac77fe640 1 -- 172.21.15.94:0/3216865035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac806e960 msgr2=0x7feac80b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac77fe640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac806e960 0x7feac80b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac77fe640 1 -- 172.21.15.94:0/3216865035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac80b0e50 con 0x7feac80ce880 2024-04-25T16:51:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac6ffd640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac806e960 0x7feac80b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac77fe640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac80ce880 0x7feac80ca4c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7feac0007e90 tx=0x7feac000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac4ff9640 1 -- 172.21.15.94:0/3216865035 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0014420 con 0x7feac80ce880 2024-04-25T16:51:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac4ff9640 1 -- 172.21.15.94:0/3216865035 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feac0019070 con 0x7feac80ce880 2024-04-25T16:51:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feacd9fa640 1 -- 172.21.15.94:0/3216865035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac80b1130 con 0x7feac80ce880 2024-04-25T16:51:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac4ff9640 1 -- 172.21.15.94:0/3216865035 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0014800 con 0x7feac80ce880 2024-04-25T16:51:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feacd9fa640 1 -- 172.21.15.94:0/3216865035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac80b16f0 con 0x7feac80ce880 2024-04-25T16:51:52.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feacd9fa640 1 -- 172.21.15.94:0/3216865035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac806a730 con 0x7feac80ce880 2024-04-25T16:51:52.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.519+0000 7feac4ff9640 1 -- 172.21.15.94:0/3216865035 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feac00149a0 con 0x7feac80ce880 2024-04-25T16:51:52.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.523+0000 7feac4ff9640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feaac06f400 0x7feaac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:52.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.523+0000 7feac6ffd640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feaac06f400 0x7feaac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:52.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.523+0000 7feac4ff9640 1 -- 172.21.15.94:0/3216865035 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7feac008fe60 con 0x7feac80ce880 2024-04-25T16:51:52.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.523+0000 7feac6ffd640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feaac06f400 0x7feaac071910 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feac81612e0 tx=0x7feab8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:52.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.527+0000 7feac4ff9640 1 -- 172.21.15.94:0/3216865035 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feac005c9e0 con 0x7feac80ce880 2024-04-25T16:51:52.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.731+0000 7feacd9fa640 1 -- 172.21.15.94:0/3216865035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) -- 0x7feac80b3180 con 0x7feac80ce880 2024-04-25T16:51:52.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.947+0000 7feac4ff9640 1 -- 172.21.15.94:0/3216865035 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v74) ==== 147+0+0 (secure 0 0 0) 0x7feac005c2e0 con 0x7feac80ce880 2024-04-25T16:51:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.951+0000 7feacd9fa640 1 -- 172.21.15.94:0/3216865035 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feaac06f400 msgr2=0x7feaac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.951+0000 7feacd9fa640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feaac06f400 0x7feaac071910 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feac81612e0 tx=0x7feab8008040 comp rx=0 tx=0).stop 2024-04-25T16:51:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.951+0000 7feacd9fa640 1 -- 172.21.15.94:0/3216865035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac80ce880 msgr2=0x7feac80ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.951+0000 7feacd9fa640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac80ce880 0x7feac80ca4c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7feac0007e90 tx=0x7feac000e4b0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.951+0000 7feacd9fa640 1 -- 172.21.15.94:0/3216865035 shutdown_connections 2024-04-25T16:51:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.951+0000 7feacd9fa640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac80ce880 0x7feac80ca4c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.951+0000 7feacd9fa640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feaac06f400 0x7feaac071910 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.951+0000 7feacd9fa640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feac8072bb0 0x7feac80c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.951+0000 7feacd9fa640 1 --2- 172.21.15.94:0/3216865035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac806e960 0x7feac80b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.951+0000 7feacd9fa640 1 -- 172.21.15.94:0/3216865035 >> 172.21.15.94:0/3216865035 conn(0x7feac80d3850 msgr2=0x7feac815c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.951+0000 7feacd9fa640 1 -- 172.21.15.94:0/3216865035 shutdown_connections 2024-04-25T16:51:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:52.951+0000 7feacd9fa640 1 -- 172.21.15.94:0/3216865035 wait complete. 2024-04-25T16:51:52.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T16:51:53.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdff1587640 1 Processor -- start 2024-04-25T16:51:53.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdff1587640 1 -- start start 2024-04-25T16:51:53.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdff1587640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdfec0ce990 0x7fdfec15da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:53.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdff1587640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfec0d2000 0x7fdfec0d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:53.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdff1587640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfec0d2950 con 0x7fdfec0b7280 2024-04-25T16:51:53.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdff1587640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fdfec0b6270 con 0x7fdfec0ce990 2024-04-25T16:51:53.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdff1587640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdfec0b64c0 con 0x7fdfec0d2000 2024-04-25T16:51:53.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdfeaffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdfec0ce990 0x7fdfec15da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:53.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdfeaffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdfec0ce990 0x7fdfec15da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59732/0 (socket says 172.21.15.94:59732) 2024-04-25T16:51:53.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdfeaffd640 1 -- 172.21.15.94:0/3333857569 learned_addr learned my addr 172.21.15.94:0/3333857569 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:53.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdfea7fc640 1 --2- 172.21.15.94:0/3333857569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfec0d2000 0x7fdfec0d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:53.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdfeaffd640 1 -- 172.21.15.94:0/3333857569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfec0d2000 msgr2=0x7fdfec0d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:53.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdfeaffd640 1 --2- 172.21.15.94:0/3333857569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfec0d2000 0x7fdfec0d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdfeaffd640 1 -- 172.21.15.94:0/3333857569 >> v1:172.21.15.138:6789/0 conn(0x7fdfec0b7280 legacy=0x7fdfec0b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:53.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.131+0000 7fdfeaffd640 1 -- 172.21.15.94:0/3333857569 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec0b67a0 con 0x7fdfec0ce990 2024-04-25T16:51:53.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.135+0000 7fdfeaffd640 1 --2- 172.21.15.94:0/3333857569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfec0ce990 0x7fdfec15da50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009870 tx=0x7fdfdc01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cd9e755388b103 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:53.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.135+0000 7fdfe9ffb640 1 -- 172.21.15.94:0/3333857569 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0b5070 con 0x7fdfec0ce990 2024-04-25T16:51:53.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.135+0000 7fdfe9ffb640 1 -- 172.21.15.94:0/3333857569 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfdc0a8a50 con 0x7fdfec0ce990 2024-04-25T16:51:53.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.135+0000 7fdfe9ffb640 1 -- 172.21.15.94:0/3333857569 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0a8e20 con 0x7fdfec0ce990 2024-04-25T16:51:53.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.135+0000 7fdff1587640 1 -- 172.21.15.94:0/3333857569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfec0ce990 msgr2=0x7fdfec15da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:53.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.135+0000 7fdff1587640 1 --2- 172.21.15.94:0/3333857569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfec0ce990 0x7fdfec15da50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009870 tx=0x7fdfdc01ed50 comp rx=0 tx=0).stop 2024-04-25T16:51:53.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.135+0000 7fdff1587640 1 -- 172.21.15.94:0/3333857569 shutdown_connections 2024-04-25T16:51:53.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.135+0000 7fdff1587640 1 --2- 172.21.15.94:0/3333857569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfec0d2000 0x7fdfec0d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.135+0000 7fdff1587640 1 --2- 172.21.15.94:0/3333857569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfec0ce990 0x7fdfec15da50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.135+0000 7fdff1587640 1 -- 172.21.15.94:0/3333857569 >> 172.21.15.94:0/3333857569 conn(0x7fdfec0cfea0 msgr2=0x7fdfec0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:53.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.135+0000 7fdff1587640 1 -- 172.21.15.94:0/3333857569 shutdown_connections 2024-04-25T16:51:53.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.135+0000 7fdff1587640 1 -- 172.21.15.94:0/3333857569 wait complete. 2024-04-25T16:51:53.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdff1587640 1 Processor -- start 2024-04-25T16:51:53.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdff1587640 1 -- start start 2024-04-25T16:51:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdff1587640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfec0b7280 0x7fdfec1f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdff1587640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfec0ce990 0x7fdfec1f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdff1587640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfec0d2000 0x7fdfec1f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdff1587640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec0cf160 con 0x7fdfec0ce990 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdff1587640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec0b6d60 con 0x7fdfec0d2000 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdff1587640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdfec0cf2e0 con 0x7fdfec0b7280 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdfea7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfec0ce990 0x7fdfec1f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdfea7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfec0ce990 0x7fdfec1f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59740/0 (socket says 172.21.15.94:59740) 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdfea7fc640 1 -- 172.21.15.94:0/1141930116 learned_addr learned my addr 172.21.15.94:0/1141930116 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdfeaffd640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfec0b7280 0x7fdfec1f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdfeb7fe640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfec0d2000 0x7fdfec1f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdfea7fc640 1 -- 172.21.15.94:0/1141930116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfec0b7280 msgr2=0x7fdfec1f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdfea7fc640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfec0b7280 0x7fdfec1f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdfea7fc640 1 -- 172.21.15.94:0/1141930116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfec0d2000 msgr2=0x7fdfec1f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdfea7fc640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfec0d2000 0x7fdfec1f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdfea7fc640 1 -- 172.21.15.94:0/1141930116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec15e190 con 0x7fdfec0ce990 2024-04-25T16:51:53.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.139+0000 7fdfea7fc640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfec0ce990 0x7fdfec1f47a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0006e90 tx=0x7fdfe000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:53.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.143+0000 7fdfd37fe640 1 -- 172.21.15.94:0/1141930116 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000b6c0 con 0x7fdfec0ce990 2024-04-25T16:51:53.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.143+0000 7fdfd37fe640 1 -- 172.21.15.94:0/1141930116 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe0019070 con 0x7fdfec0ce990 2024-04-25T16:51:53.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.143+0000 7fdfd37fe640 1 -- 172.21.15.94:0/1141930116 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000bac0 con 0x7fdfec0ce990 2024-04-25T16:51:53.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.143+0000 7fdff1587640 1 -- 172.21.15.94:0/1141930116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfec14cc50 con 0x7fdfec0ce990 2024-04-25T16:51:53.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.143+0000 7fdff1587640 1 -- 172.21.15.94:0/1141930116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfec1f5000 con 0x7fdfec0ce990 2024-04-25T16:51:53.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.143+0000 7fdff1587640 1 -- 172.21.15.94:0/1141930116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfec06a730 con 0x7fdfec0ce990 2024-04-25T16:51:53.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.143+0000 7fdfd37fe640 1 -- 172.21.15.94:0/1141930116 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdfe000bc60 con 0x7fdfec0ce990 2024-04-25T16:51:53.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.143+0000 7fdfd37fe640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdfd806f430 0x7fdfd8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:53.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.143+0000 7fdfeaffd640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdfd806f430 0x7fdfd8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:53.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.143+0000 7fdfd37fe640 1 -- 172.21.15.94:0/1141930116 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdfe008f9f0 con 0x7fdfec0ce990 2024-04-25T16:51:53.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.143+0000 7fdfeaffd640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdfd806f430 0x7fdfd8071940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0b2000 tx=0x7fdfdc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:53.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.147+0000 7fdfd37fe640 1 -- 172.21.15.94:0/1141930116 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdfe005c570 con 0x7fdfec0ce990 2024-04-25T16:51:53.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.307+0000 7fdff1587640 1 -- 172.21.15.94:0/1141930116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdfec14c9e0 con 0x7fdfec0ce990 2024-04-25T16:51:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.315+0000 7fdfd37fe640 1 -- 172.21.15.94:0/1141930116 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fdfe005be70 con 0x7fdfec0ce990 2024-04-25T16:51:53.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.319+0000 7fdff1587640 1 -- 172.21.15.94:0/1141930116 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdfd806f430 msgr2=0x7fdfd8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:53.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.319+0000 7fdff1587640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdfd806f430 0x7fdfd8071940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0b2000 tx=0x7fdfdc009040 comp rx=0 tx=0).stop 2024-04-25T16:51:53.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.319+0000 7fdff1587640 1 -- 172.21.15.94:0/1141930116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfec0ce990 msgr2=0x7fdfec1f47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:53.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.319+0000 7fdff1587640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfec0ce990 0x7fdfec1f47a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0006e90 tx=0x7fdfe000dc60 comp rx=0 tx=0).stop 2024-04-25T16:51:53.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.319+0000 7fdff1587640 1 -- 172.21.15.94:0/1141930116 shutdown_connections 2024-04-25T16:51:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.319+0000 7fdff1587640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfec0d2000 0x7fdfec1f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.319+0000 7fdff1587640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfec0ce990 0x7fdfec1f47a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.319+0000 7fdff1587640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdfd806f430 0x7fdfd8071940 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.319+0000 7fdff1587640 1 --2- 172.21.15.94:0/1141930116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfec0b7280 0x7fdfec1f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.319+0000 7fdff1587640 1 -- 172.21.15.94:0/1141930116 >> 172.21.15.94:0/1141930116 conn(0x7fdfec0cfea0 msgr2=0x7fdfec15cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.319+0000 7fdff1587640 1 -- 172.21.15.94:0/1141930116 shutdown_connections 2024-04-25T16:51:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.319+0000 7fdff1587640 1 -- 172.21.15.94:0/1141930116 wait complete. 2024-04-25T16:51:53.336 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T16:51:53.336 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T16:51:53.369 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T16:51:53.431 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T16:51:53.432 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:51:53.432 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:51:53.437 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:51:53.437 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:51:53.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:53.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d925f4640 1 Processor -- start 2024-04-25T16:51:53.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d925f4640 1 -- start start 2024-04-25T16:51:53.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d925f4640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9d8c0ce480 0x7f9d8c0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:53.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d925f4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d8c0d2030 0x7f9d8c0d2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:53.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d925f4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9d8c0b6980 0x7f9d8c0b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d925f4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f9d8c0ca590 con 0x7f9d8c0b6980 2024-04-25T16:51:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d925f4640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9d8c0d2980 con 0x7f9d8c0ce480 2024-04-25T16:51:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d925f4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9d8c0ccf30 con 0x7f9d8c0d2030 2024-04-25T16:51:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d90b91640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9d8c0b6980 0x7f9d8c0b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d90b91640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9d8c0b6980 0x7f9d8c0b6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59752/0 (socket says 172.21.15.94:59752) 2024-04-25T16:51:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d90b91640 1 -- 172.21.15.94:0/1543895318 learned_addr learned my addr 172.21.15.94:0/1543895318 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d90b91640 1 -- 172.21.15.94:0/1543895318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d8c0d2030 msgr2=0x7f9d8c0d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d90b91640 1 --2- 172.21.15.94:0/1543895318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d8c0d2030 0x7f9d8c0d2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d8bfff640 1 --2- 172.21.15.94:0/1543895318 >> v2:172.21.15.138:3300/0 conn(0x7f9d8c0ce480 0x7f9d8c0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d90b91640 1 -- 172.21.15.94:0/1543895318 >> v2:172.21.15.138:3300/0 conn(0x7f9d8c0ce480 msgr2=0x7f9d8c0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d90b91640 1 --2- 172.21.15.94:0/1543895318 >> v2:172.21.15.138:3300/0 conn(0x7f9d8c0ce480 0x7f9d8c0ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.683+0000 7f9d90b91640 1 -- 172.21.15.94:0/1543895318 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c0cd210 con 0x7f9d8c0b6980 2024-04-25T16:51:53.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.687+0000 7f9d90b91640 1 --2- 172.21.15.94:0/1543895318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d8c0b6980 0x7f9d8c0b6d90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c074150 tx=0x7f9d8401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60044fadf637b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:53.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.687+0000 7f9d8affd640 1 -- 172.21.15.94:0/1543895318 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840a5670 con 0x7f9d8c0b6980 2024-04-25T16:51:53.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.687+0000 7f9d8affd640 1 -- 172.21.15.94:0/1543895318 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d840a5810 con 0x7f9d8c0b6980 2024-04-25T16:51:53.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.687+0000 7f9d8affd640 1 -- 172.21.15.94:0/1543895318 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840a5be0 con 0x7f9d8c0b6980 2024-04-25T16:51:53.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.687+0000 7f9d925f4640 1 -- 172.21.15.94:0/1543895318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d8c0b6980 msgr2=0x7f9d8c0b6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:53.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.687+0000 7f9d925f4640 1 --2- 172.21.15.94:0/1543895318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d8c0b6980 0x7f9d8c0b6d90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c074150 tx=0x7f9d8401c9f0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.687+0000 7f9d925f4640 1 -- 172.21.15.94:0/1543895318 shutdown_connections 2024-04-25T16:51:53.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.687+0000 7f9d925f4640 1 --2- 172.21.15.94:0/1543895318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d8c0b6980 0x7f9d8c0b6d90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.687+0000 7f9d925f4640 1 --2- 172.21.15.94:0/1543895318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d8c0d2030 0x7f9d8c0d2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.687+0000 7f9d925f4640 1 --2- 172.21.15.94:0/1543895318 >> v2:172.21.15.138:3300/0 conn(0x7f9d8c0ce480 0x7f9d8c0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.687+0000 7f9d925f4640 1 -- 172.21.15.94:0/1543895318 >> 172.21.15.94:0/1543895318 conn(0x7f9d8c0cfed0 msgr2=0x7f9d8c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:53.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.687+0000 7f9d925f4640 1 -- 172.21.15.94:0/1543895318 shutdown_connections 2024-04-25T16:51:53.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.687+0000 7f9d925f4640 1 -- 172.21.15.94:0/1543895318 wait complete. 2024-04-25T16:51:53.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d925f4640 1 Processor -- start 2024-04-25T16:51:53.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d925f4640 1 -- start start 2024-04-25T16:51:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d925f4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d8c0b6980 0x7f9d8c14c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d925f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d8c0ce480 0x7f9d8c157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d925f4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d8c0d2030 0x7f9d8c0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d925f4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c16b8e0 con 0x7f9d8c0ce480 2024-04-25T16:51:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d925f4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c16b760 con 0x7f9d8c0b6980 2024-04-25T16:51:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d925f4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9d8c16ba60 con 0x7f9d8c0d2030 2024-04-25T16:51:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d8b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d8c0ce480 0x7f9d8c157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d8b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d8c0ce480 0x7f9d8c157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59768/0 (socket says 172.21.15.94:59768) 2024-04-25T16:51:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d8b7fe640 1 -- 172.21.15.94:0/1988521145 learned_addr learned my addr 172.21.15.94:0/1988521145 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d90b91640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d8c0d2030 0x7f9d8c0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:53.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d8bfff640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d8c0b6980 0x7f9d8c14c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:53.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d8b7fe640 1 -- 172.21.15.94:0/1988521145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d8c0d2030 msgr2=0x7f9d8c0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:53.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d8b7fe640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d8c0d2030 0x7f9d8c0b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d8b7fe640 1 -- 172.21.15.94:0/1988521145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d8c0b6980 msgr2=0x7f9d8c14c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:53.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d8b7fe640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d8c0b6980 0x7f9d8c14c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d8b7fe640 1 -- 172.21.15.94:0/1988521145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c14c6a0 con 0x7f9d8c0ce480 2024-04-25T16:51:53.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d8b7fe640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d8c0ce480 0x7f9d8c157d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00b410 tx=0x7f9d7c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:53.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d88ff9640 1 -- 172.21.15.94:0/1988521145 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c015020 con 0x7f9d8c0ce480 2024-04-25T16:51:53.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d88ff9640 1 -- 172.21.15.94:0/1988521145 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c011830 con 0x7f9d8c0ce480 2024-04-25T16:51:53.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d925f4640 1 -- 172.21.15.94:0/1988521145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c1516a0 con 0x7f9d8c0ce480 2024-04-25T16:51:53.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d88ff9640 1 -- 172.21.15.94:0/1988521145 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c011c20 con 0x7f9d8c0ce480 2024-04-25T16:51:53.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d925f4640 1 -- 172.21.15.94:0/1988521145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c158b90 con 0x7f9d8c0ce480 2024-04-25T16:51:53.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.691+0000 7f9d925f4640 1 -- 172.21.15.94:0/1988521145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8c076830 con 0x7f9d8c0ce480 2024-04-25T16:51:53.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.695+0000 7f9d88ff9640 1 -- 172.21.15.94:0/1988521145 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9d7c018020 con 0x7f9d8c0ce480 2024-04-25T16:51:53.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.695+0000 7f9d88ff9640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9d6c06f400 0x7f9d6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:53.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.695+0000 7f9d88ff9640 1 -- 172.21.15.94:0/1988521145 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9d7c08fb30 con 0x7f9d8c0ce480 2024-04-25T16:51:53.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.695+0000 7f9d8bfff640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9d6c06f400 0x7f9d6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:53.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.695+0000 7f9d8bfff640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9d6c06f400 0x7f9d6c071910 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0cd9e0 tx=0x7f9d74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:53.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.699+0000 7f9d88ff9640 1 -- 172.21.15.94:0/1988521145 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9d7c01f020 con 0x7f9d8c0ce480 2024-04-25T16:51:53.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.859+0000 7f9d925f4640 1 -- 172.21.15.94:0/1988521145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d8c158640 con 0x7f9d8c0ce480 2024-04-25T16:51:53.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.863+0000 7f9d88ff9640 1 -- 172.21.15.94:0/1988521145 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7f9d8c14c6a0 con 0x7f9d8c0ce480 2024-04-25T16:51:53.861 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:53.862 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:53.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.867+0000 7f9d925f4640 1 -- 172.21.15.94:0/1988521145 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9d6c06f400 msgr2=0x7f9d6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:53.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.867+0000 7f9d925f4640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9d6c06f400 0x7f9d6c071910 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0cd9e0 tx=0x7f9d74008040 comp rx=0 tx=0).stop 2024-04-25T16:51:53.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.867+0000 7f9d925f4640 1 -- 172.21.15.94:0/1988521145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d8c0ce480 msgr2=0x7f9d8c157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:53.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.867+0000 7f9d925f4640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d8c0ce480 0x7f9d8c157d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00b410 tx=0x7f9d7c00b8d0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.867+0000 7f9d925f4640 1 -- 172.21.15.94:0/1988521145 shutdown_connections 2024-04-25T16:51:53.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.867+0000 7f9d925f4640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9d6c06f400 0x7f9d6c071910 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.867+0000 7f9d925f4640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d8c0d2030 0x7f9d8c0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.867+0000 7f9d925f4640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d8c0ce480 0x7f9d8c157d40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.867+0000 7f9d925f4640 1 --2- 172.21.15.94:0/1988521145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d8c0b6980 0x7f9d8c14c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:53.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.867+0000 7f9d925f4640 1 -- 172.21.15.94:0/1988521145 >> 172.21.15.94:0/1988521145 conn(0x7f9d8c0cfed0 msgr2=0x7f9d8c153700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:53.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.867+0000 7f9d925f4640 1 -- 172.21.15.94:0/1988521145 shutdown_connections 2024-04-25T16:51:53.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:53.867+0000 7f9d925f4640 1 -- 172.21.15.94:0/1988521145 wait complete. 2024-04-25T16:51:53.885 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:51:53.885 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:51:54.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.039+0000 7faa718bf640 1 Processor -- start 2024-04-25T16:51:54.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.039+0000 7faa718bf640 1 -- start start 2024-04-25T16:51:54.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.039+0000 7faa718bf640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6c06d710 0x7faa6c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.039+0000 7faa718bf640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faa6c06e7a0 0x7faa6c06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.039+0000 7faa718bf640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa6c074380 con 0x7faa6c15c7e0 2024-04-25T16:51:54.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.039+0000 7faa718bf640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7faa6c06e060 con 0x7faa6c06e7a0 2024-04-25T16:51:54.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.039+0000 7faa718bf640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c0727e0 con 0x7faa6c06d710 2024-04-25T16:51:54.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.039+0000 7faa6b7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faa6c06e7a0 0x7faa6c06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.039+0000 7faa6b7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faa6c06e7a0 0x7faa6c06ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59778/0 (socket says 172.21.15.94:59778) 2024-04-25T16:51:54.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.039+0000 7faa6b7fe640 1 -- 172.21.15.94:0/1950154791 learned_addr learned my addr 172.21.15.94:0/1950154791 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:54.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.039+0000 7faa6a7fc640 1 --2- 172.21.15.94:0/1950154791 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6c06d710 0x7faa6c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa6b7fe640 1 -- 172.21.15.94:0/1950154791 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6c06d710 msgr2=0x7faa6c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:54.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa6b7fe640 1 --2- 172.21.15.94:0/1950154791 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6c06d710 0x7faa6c06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa6b7fe640 1 -- 172.21.15.94:0/1950154791 >> v1:172.21.15.138:6789/0 conn(0x7faa6c15c7e0 legacy=0x7faa6c0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa6b7fe640 1 -- 172.21.15.94:0/1950154791 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c072ac0 con 0x7faa6c06e7a0 2024-04-25T16:51:54.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa6b7fe640 1 --2- 172.21.15.94:0/1950154791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c06e7a0 0x7faa6c06ebb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faa64009e50 tx=0x7faa6401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb2f611bda98b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:54.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa69ffb640 1 -- 172.21.15.94:0/1950154791 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640a5830 con 0x7faa6c06e7a0 2024-04-25T16:51:54.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa69ffb640 1 -- 172.21.15.94:0/1950154791 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa640a59d0 con 0x7faa6c06e7a0 2024-04-25T16:51:54.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa69ffb640 1 -- 172.21.15.94:0/1950154791 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640a5da0 con 0x7faa6c06e7a0 2024-04-25T16:51:54.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa718bf640 1 -- 172.21.15.94:0/1950154791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c06e7a0 msgr2=0x7faa6c06ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa718bf640 1 --2- 172.21.15.94:0/1950154791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c06e7a0 0x7faa6c06ebb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faa64009e50 tx=0x7faa6401c970 comp rx=0 tx=0).stop 2024-04-25T16:51:54.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa718bf640 1 -- 172.21.15.94:0/1950154791 shutdown_connections 2024-04-25T16:51:54.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa718bf640 1 --2- 172.21.15.94:0/1950154791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c06e7a0 0x7faa6c06ebb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa718bf640 1 --2- 172.21.15.94:0/1950154791 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6c06d710 0x7faa6c06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa718bf640 1 -- 172.21.15.94:0/1950154791 >> 172.21.15.94:0/1950154791 conn(0x7faa6c0b47f0 msgr2=0x7faa6c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:54.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa718bf640 1 -- 172.21.15.94:0/1950154791 shutdown_connections 2024-04-25T16:51:54.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.043+0000 7faa718bf640 1 -- 172.21.15.94:0/1950154791 wait complete. 2024-04-25T16:51:54.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa718bf640 1 Processor -- start 2024-04-25T16:51:54.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa718bf640 1 -- start start 2024-04-25T16:51:54.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa718bf640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6c06d710 0x7faa6c15f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa718bf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c06e7a0 0x7faa6c15d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa718bf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c15c7e0 0x7faa6c148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa718bf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c15cbf0 con 0x7faa6c15c7e0 2024-04-25T16:51:54.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa718bf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c073ca0 con 0x7faa6c06e7a0 2024-04-25T16:51:54.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa718bf640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c0b0500 con 0x7faa6c06d710 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c15c7e0 0x7faa6c148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c15c7e0 0x7faa6c148310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59788/0 (socket says 172.21.15.94:59788) 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa6b7fe640 1 -- 172.21.15.94:0/4068419431 learned_addr learned my addr 172.21.15.94:0/4068419431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa6a7fc640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c06e7a0 0x7faa6c15d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa6affd640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6c06d710 0x7faa6c15f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa6b7fe640 1 -- 172.21.15.94:0/4068419431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6c06d710 msgr2=0x7faa6c15f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa6b7fe640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6c06d710 0x7faa6c15f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa6b7fe640 1 -- 172.21.15.94:0/4068419431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c06e7a0 msgr2=0x7faa6c15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa6b7fe640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c06e7a0 0x7faa6c15d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa6b7fe640 1 -- 172.21.15.94:0/4068419431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c14d2c0 con 0x7faa6c15c7e0 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa6a7fc640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c06e7a0 0x7faa6c15d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa6b7fe640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c15c7e0 0x7faa6c148310 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faa64009e50 tx=0x7faa64019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa53fff640 1 -- 172.21.15.94:0/4068419431 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640bc020 con 0x7faa6c15c7e0 2024-04-25T16:51:54.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa53fff640 1 -- 172.21.15.94:0/4068419431 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa640b8670 con 0x7faa6c15c7e0 2024-04-25T16:51:54.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa53fff640 1 -- 172.21.15.94:0/4068419431 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640b8a60 con 0x7faa6c15c7e0 2024-04-25T16:51:54.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa718bf640 1 -- 172.21.15.94:0/4068419431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c147550 con 0x7faa6c15c7e0 2024-04-25T16:51:54.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.047+0000 7faa718bf640 1 -- 172.21.15.94:0/4068419431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c147b10 con 0x7faa6c15c7e0 2024-04-25T16:51:54.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.051+0000 7faa718bf640 1 -- 172.21.15.94:0/4068419431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6c076830 con 0x7faa6c15c7e0 2024-04-25T16:51:54.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.051+0000 7faa53fff640 1 -- 172.21.15.94:0/4068419431 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faa640bf020 con 0x7faa6c15c7e0 2024-04-25T16:51:54.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.051+0000 7faa53fff640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa5806f430 0x7faa58071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.051+0000 7faa6affd640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa5806f430 0x7faa58071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.051+0000 7faa53fff640 1 -- 172.21.15.94:0/4068419431 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7faa6412a7e0 con 0x7faa6c15c7e0 2024-04-25T16:51:54.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.051+0000 7faa6affd640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa5806f430 0x7faa58071940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa5c007fd0 tx=0x7faa5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:54.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.055+0000 7faa53fff640 1 -- 172.21.15.94:0/4068419431 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faa640c6030 con 0x7faa6c15c7e0 2024-04-25T16:51:54.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.203+0000 7faa718bf640 1 -- 172.21.15.94:0/4068419431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faa6c06fc20 con 0x7faa6c15c7e0 2024-04-25T16:51:54.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.203+0000 7faa53fff640 1 -- 172.21.15.94:0/4068419431 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7faa64016620 con 0x7faa6c15c7e0 2024-04-25T16:51:54.203 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:51:54.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.207+0000 7faa718bf640 1 -- 172.21.15.94:0/4068419431 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa5806f430 msgr2=0x7faa58071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.207+0000 7faa718bf640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa5806f430 0x7faa58071940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa5c007fd0 tx=0x7faa5c009040 comp rx=0 tx=0).stop 2024-04-25T16:51:54.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.207+0000 7faa718bf640 1 -- 172.21.15.94:0/4068419431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c15c7e0 msgr2=0x7faa6c148310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.207+0000 7faa718bf640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c15c7e0 0x7faa6c148310 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faa64009e50 tx=0x7faa64019710 comp rx=0 tx=0).stop 2024-04-25T16:51:54.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.207+0000 7faa718bf640 1 -- 172.21.15.94:0/4068419431 shutdown_connections 2024-04-25T16:51:54.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.207+0000 7faa718bf640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c15c7e0 0x7faa6c148310 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.207+0000 7faa718bf640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c06e7a0 0x7faa6c15d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.207+0000 7faa718bf640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa5806f430 0x7faa58071940 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.207+0000 7faa718bf640 1 --2- 172.21.15.94:0/4068419431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6c06d710 0x7faa6c15f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.207+0000 7faa718bf640 1 -- 172.21.15.94:0/4068419431 >> 172.21.15.94:0/4068419431 conn(0x7faa6c0b47f0 msgr2=0x7faa6c0b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:54.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.207+0000 7faa718bf640 1 -- 172.21.15.94:0/4068419431 shutdown_connections 2024-04-25T16:51:54.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.207+0000 7faa718bf640 1 -- 172.21.15.94:0/4068419431 wait complete. 2024-04-25T16:51:54.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:51:54.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d8faf640 1 Processor -- start 2024-04-25T16:51:54.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d8faf640 1 -- start start 2024-04-25T16:51:54.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d8faf640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f47d415c7e0 0x7f47d40afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d8faf640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f47d406de20 0x7f47d406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d8faf640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d4074380 con 0x7f47d40b0330 2024-04-25T16:51:54.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d8faf640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f47d40727e0 con 0x7f47d406de20 2024-04-25T16:51:54.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d8faf640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f47d4072ac0 con 0x7f47d415c7e0 2024-04-25T16:51:54.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d2d9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f47d406de20 0x7f47d406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d2d9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f47d406de20 0x7f47d406e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59804/0 (socket says 172.21.15.94:59804) 2024-04-25T16:51:54.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d2d9d640 1 -- 172.21.15.94:0/4271139657 learned_addr learned my addr 172.21.15.94:0/4271139657 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:54.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d259c640 1 --2- 172.21.15.94:0/4271139657 >> v2:172.21.15.138:3300/0 conn(0x7f47d415c7e0 0x7f47d40afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d2d9d640 1 -- 172.21.15.94:0/4271139657 >> v2:172.21.15.138:3300/0 conn(0x7f47d415c7e0 msgr2=0x7f47d40afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:54.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d2d9d640 1 --2- 172.21.15.94:0/4271139657 >> v2:172.21.15.138:3300/0 conn(0x7f47d415c7e0 0x7f47d40afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d2d9d640 1 -- 172.21.15.94:0/4271139657 >> v1:172.21.15.138:6789/0 conn(0x7f47d40b0330 legacy=0x7f47d406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:54.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.363+0000 7f47d2d9d640 1 -- 172.21.15.94:0/4271139657 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d4072da0 con 0x7f47d406de20 2024-04-25T16:51:54.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.367+0000 7f47d2d9d640 1 --2- 172.21.15.94:0/4271139657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d406de20 0x7f47d406e7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f47d4070940 tx=0x7f47cc01c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36b6bba35310d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:54.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.367+0000 7f47d159a640 1 -- 172.21.15.94:0/4271139657 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0a55d0 con 0x7f47d406de20 2024-04-25T16:51:54.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.367+0000 7f47d159a640 1 -- 172.21.15.94:0/4271139657 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47cc0a5770 con 0x7f47d406de20 2024-04-25T16:51:54.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.367+0000 7f47d159a640 1 -- 172.21.15.94:0/4271139657 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0a5b40 con 0x7f47d406de20 2024-04-25T16:51:54.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.367+0000 7f47d8faf640 1 -- 172.21.15.94:0/4271139657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d406de20 msgr2=0x7f47d406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.367+0000 7f47d8faf640 1 --2- 172.21.15.94:0/4271139657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d406de20 0x7f47d406e7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f47d4070940 tx=0x7f47cc01c710 comp rx=0 tx=0).stop 2024-04-25T16:51:54.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.367+0000 7f47d8faf640 1 -- 172.21.15.94:0/4271139657 shutdown_connections 2024-04-25T16:51:54.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.367+0000 7f47d8faf640 1 --2- 172.21.15.94:0/4271139657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d406de20 0x7f47d406e7a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.367+0000 7f47d8faf640 1 --2- 172.21.15.94:0/4271139657 >> v2:172.21.15.138:3300/0 conn(0x7f47d415c7e0 0x7f47d40afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.367+0000 7f47d8faf640 1 -- 172.21.15.94:0/4271139657 >> 172.21.15.94:0/4271139657 conn(0x7f47d40b47f0 msgr2=0x7f47d40b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:54.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.367+0000 7f47d8faf640 1 -- 172.21.15.94:0/4271139657 shutdown_connections 2024-04-25T16:51:54.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.367+0000 7f47d8faf640 1 -- 172.21.15.94:0/4271139657 wait complete. 2024-04-25T16:51:54.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d8faf640 1 Processor -- start 2024-04-25T16:51:54.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d8faf640 1 -- start start 2024-04-25T16:51:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d8faf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d40b0330 0x7f47d41fcd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d8faf640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47d415c7e0 0x7f47d406c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d8faf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d41494f0 0x7f47d406cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d8faf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47d41695d0 con 0x7f47d41494f0 2024-04-25T16:51:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d8faf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47d415cbf0 con 0x7f47d40b0330 2024-04-25T16:51:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d8faf640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f47d4169750 con 0x7f47d415c7e0 2024-04-25T16:51:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d41494f0 0x7f47d406cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d41494f0 0x7f47d406cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59806/0 (socket says 172.21.15.94:59806) 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d2d9d640 1 -- 172.21.15.94:0/764096515 learned_addr learned my addr 172.21.15.94:0/764096515 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d259c640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d40b0330 0x7f47d41fcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d1d9b640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47d415c7e0 0x7f47d406c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d2d9d640 1 -- 172.21.15.94:0/764096515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47d415c7e0 msgr2=0x7f47d406c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d2d9d640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47d415c7e0 0x7f47d406c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d2d9d640 1 -- 172.21.15.94:0/764096515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d40b0330 msgr2=0x7f47d41fcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d2d9d640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d40b0330 0x7f47d41fcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d2d9d640 1 -- 172.21.15.94:0/764096515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d40cf320 con 0x7f47d41494f0 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d1d9b640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47d415c7e0 0x7f47d406c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47d2d9d640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d41494f0 0x7f47d406cb40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0af040 tx=0x7f47cc0194b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.371+0000 7f47c2ffd640 1 -- 172.21.15.94:0/764096515 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc009510 con 0x7f47d41494f0 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.375+0000 7f47c2ffd640 1 -- 172.21.15.94:0/764096515 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47cc0b8470 con 0x7f47d41494f0 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.375+0000 7f47d8faf640 1 -- 172.21.15.94:0/764096515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d406d1d0 con 0x7f47d41494f0 2024-04-25T16:51:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.375+0000 7f47c2ffd640 1 -- 172.21.15.94:0/764096515 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0b8860 con 0x7f47d41494f0 2024-04-25T16:51:54.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.375+0000 7f47d8faf640 1 -- 172.21.15.94:0/764096515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d406d4b0 con 0x7f47d41494f0 2024-04-25T16:51:54.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.375+0000 7f47d8faf640 1 -- 172.21.15.94:0/764096515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d414ad20 con 0x7f47d41494f0 2024-04-25T16:51:54.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.375+0000 7f47c2ffd640 1 -- 172.21.15.94:0/764096515 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f47cc0c0020 con 0x7f47d41494f0 2024-04-25T16:51:54.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.375+0000 7f47c2ffd640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f47b406f400 0x7f47b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.375+0000 7f47c2ffd640 1 -- 172.21.15.94:0/764096515 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f47cc12a230 con 0x7f47d41494f0 2024-04-25T16:51:54.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.375+0000 7f47d259c640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f47b406f400 0x7f47b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.375+0000 7f47d259c640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f47b406f400 0x7f47b4071910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f47c400b930 tx=0x7f47c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:54.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.379+0000 7f47c2ffd640 1 -- 172.21.15.94:0/764096515 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f47d4074380 con 0x7f47d41494f0 2024-04-25T16:51:54.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.531+0000 7f47d8faf640 1 -- 172.21.15.94:0/764096515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f47d40d1110 con 0x7f47d41494f0 2024-04-25T16:51:54.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.531+0000 7f47c2ffd640 1 -- 172.21.15.94:0/764096515 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f47cc0f6db0 con 0x7f47d41494f0 2024-04-25T16:51:54.531 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.535+0000 7f47d8faf640 1 -- 172.21.15.94:0/764096515 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f47b406f400 msgr2=0x7f47b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.535+0000 7f47d8faf640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f47b406f400 0x7f47b4071910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f47c400b930 tx=0x7f47c4008040 comp rx=0 tx=0).stop 2024-04-25T16:51:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.535+0000 7f47d8faf640 1 -- 172.21.15.94:0/764096515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d41494f0 msgr2=0x7f47d406cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.535+0000 7f47d8faf640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d41494f0 0x7f47d406cb40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0af040 tx=0x7f47cc0194b0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.535+0000 7f47d8faf640 1 -- 172.21.15.94:0/764096515 shutdown_connections 2024-04-25T16:51:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.535+0000 7f47d8faf640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d41494f0 0x7f47d406cb40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.535+0000 7f47d8faf640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f47b406f400 0x7f47b4071910 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.535+0000 7f47d8faf640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47d415c7e0 0x7f47d406c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.535+0000 7f47d8faf640 1 --2- 172.21.15.94:0/764096515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d40b0330 0x7f47d41fcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.535+0000 7f47d8faf640 1 -- 172.21.15.94:0/764096515 >> 172.21.15.94:0/764096515 conn(0x7f47d40b47f0 msgr2=0x7f47d40b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.535+0000 7f47d8faf640 1 -- 172.21.15.94:0/764096515 shutdown_connections 2024-04-25T16:51:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.535+0000 7f47d8faf640 1 -- 172.21.15.94:0/764096515 wait complete. 2024-04-25T16:51:54.551 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:51:54.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d372a1640 1 Processor -- start 2024-04-25T16:51:54.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d372a1640 1 -- start start 2024-04-25T16:51:54.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d372a1640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1d30072b60 0x7f1d3015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d372a1640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1d300afa50 0x7f1d301601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d372a1640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d3006f360 con 0x7f1d3015cd60 2024-04-25T16:51:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d372a1640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f1d300703e0 con 0x7f1d30072b60 2024-04-25T16:51:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d372a1640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1d3006f620 con 0x7f1d300afa50 2024-04-25T16:51:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d3503d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1d30072b60 0x7f1d3015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d3503d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1d30072b60 0x7f1d3015c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59814/0 (socket says 172.21.15.94:59814) 2024-04-25T16:51:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d3503d640 1 -- 172.21.15.94:0/812554761 learned_addr learned my addr 172.21.15.94:0/812554761 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d3503d640 1 -- 172.21.15.94:0/812554761 >> v2:172.21.15.138:3300/0 conn(0x7f1d300afa50 msgr2=0x7f1d301601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d3503d640 1 --2- 172.21.15.94:0/812554761 >> v2:172.21.15.138:3300/0 conn(0x7f1d300afa50 0x7f1d301601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d3503d640 1 -- 172.21.15.94:0/812554761 >> v1:172.21.15.138:6789/0 conn(0x7f1d3015cd60 legacy=0x7f1d3015fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.691+0000 7f1d3503d640 1 -- 172.21.15.94:0/812554761 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3006f900 con 0x7f1d30072b60 2024-04-25T16:51:54.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.695+0000 7f1d3503d640 1 --2- 172.21.15.94:0/812554761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d30072b60 0x7f1d3015c820 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1d300726d0 tx=0x7f1d2001d530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9bc0e00ebbc217 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:54.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.695+0000 7f1d2ffff640 1 -- 172.21.15.94:0/812554761 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d200ab600 con 0x7f1d30072b60 2024-04-25T16:51:54.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.695+0000 7f1d2ffff640 1 -- 172.21.15.94:0/812554761 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d200b0070 con 0x7f1d30072b60 2024-04-25T16:51:54.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.695+0000 7f1d2ffff640 1 -- 172.21.15.94:0/812554761 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d200ab9c0 con 0x7f1d30072b60 2024-04-25T16:51:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.695+0000 7f1d372a1640 1 -- 172.21.15.94:0/812554761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d30072b60 msgr2=0x7f1d3015c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.695+0000 7f1d372a1640 1 --2- 172.21.15.94:0/812554761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d30072b60 0x7f1d3015c820 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1d300726d0 tx=0x7f1d2001d530 comp rx=0 tx=0).stop 2024-04-25T16:51:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.695+0000 7f1d372a1640 1 -- 172.21.15.94:0/812554761 shutdown_connections 2024-04-25T16:51:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.695+0000 7f1d372a1640 1 --2- 172.21.15.94:0/812554761 >> v2:172.21.15.138:3300/0 conn(0x7f1d300afa50 0x7f1d301601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.695+0000 7f1d372a1640 1 --2- 172.21.15.94:0/812554761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d30072b60 0x7f1d3015c820 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.695+0000 7f1d372a1640 1 -- 172.21.15.94:0/812554761 >> 172.21.15.94:0/812554761 conn(0x7f1d300b47f0 msgr2=0x7f1d300b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.695+0000 7f1d372a1640 1 -- 172.21.15.94:0/812554761 shutdown_connections 2024-04-25T16:51:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.695+0000 7f1d372a1640 1 -- 172.21.15.94:0/812554761 wait complete. 2024-04-25T16:51:54.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d372a1640 1 Processor -- start 2024-04-25T16:51:54.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d372a1640 1 -- start start 2024-04-25T16:51:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d372a1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d30072b60 0x7f1d300b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d372a1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d3015cd60 0x7f1d3009e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d372a1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d30145820 0x7f1d3009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d372a1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30159e40 con 0x7f1d30072b60 2024-04-25T16:51:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d372a1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30159cc0 con 0x7f1d3015cd60 2024-04-25T16:51:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d372a1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1d30160730 con 0x7f1d30145820 2024-04-25T16:51:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d3503d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d30072b60 0x7f1d300b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d3503d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d30072b60 0x7f1d300b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59828/0 (socket says 172.21.15.94:59828) 2024-04-25T16:51:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d3503d640 1 -- 172.21.15.94:0/3120038814 learned_addr learned my addr 172.21.15.94:0/3120038814 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d3483c640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d3015cd60 0x7f1d3009e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d3583e640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d30145820 0x7f1d3009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d3503d640 1 -- 172.21.15.94:0/3120038814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d30145820 msgr2=0x7f1d3009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d3503d640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d30145820 0x7f1d3009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d3503d640 1 -- 172.21.15.94:0/3120038814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d3015cd60 msgr2=0x7f1d3009e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d3503d640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d3015cd60 0x7f1d3009e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d3503d640 1 -- 172.21.15.94:0/3120038814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d300b11f0 con 0x7f1d30072b60 2024-04-25T16:51:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d3483c640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d3015cd60 0x7f1d3009e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:54.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d3503d640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d30072b60 0x7f1d300b0810 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1d200ad040 tx=0x7f1d2001bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:54.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d2dffb640 1 -- 172.21.15.94:0/3120038814 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d200be5f0 con 0x7f1d30072b60 2024-04-25T16:51:54.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d2dffb640 1 -- 172.21.15.94:0/3120038814 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d200be790 con 0x7f1d30072b60 2024-04-25T16:51:54.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d372a1640 1 -- 172.21.15.94:0/3120038814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d300b14d0 con 0x7f1d30072b60 2024-04-25T16:51:54.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d2dffb640 1 -- 172.21.15.94:0/3120038814 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d200beb80 con 0x7f1d30072b60 2024-04-25T16:51:54.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.699+0000 7f1d372a1640 1 -- 172.21.15.94:0/3120038814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3015a210 con 0x7f1d30072b60 2024-04-25T16:51:54.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.703+0000 7f1d372a1640 1 -- 172.21.15.94:0/3120038814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d30076830 con 0x7f1d30072b60 2024-04-25T16:51:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.703+0000 7f1d2dffb640 1 -- 172.21.15.94:0/3120038814 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1d200aa070 con 0x7f1d30072b60 2024-04-25T16:51:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.703+0000 7f1d2dffb640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d1c06f400 0x7f1d1c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.703+0000 7f1d2dffb640 1 -- 172.21.15.94:0/3120038814 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1d2012ac00 con 0x7f1d30072b60 2024-04-25T16:51:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.703+0000 7f1d3483c640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d1c06f400 0x7f1d1c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.703+0000 7f1d3483c640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d1c06f400 0x7f1d1c071910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d301f8820 tx=0x7f1d24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:54.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.707+0000 7f1d2dffb640 1 -- 172.21.15.94:0/3120038814 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1d20131080 con 0x7f1d30072b60 2024-04-25T16:51:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.855+0000 7f1d372a1640 1 -- 172.21.15.94:0/3120038814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1d30076830 con 0x7f1d30072b60 2024-04-25T16:51:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.855+0000 7f1d2dffb640 1 -- 172.21.15.94:0/3120038814 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f1d200c4060 con 0x7f1d30072b60 2024-04-25T16:51:54.853 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:51:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.855+0000 7f1d372a1640 1 -- 172.21.15.94:0/3120038814 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d1c06f400 msgr2=0x7f1d1c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.855+0000 7f1d372a1640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d1c06f400 0x7f1d1c071910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d301f8820 tx=0x7f1d24008040 comp rx=0 tx=0).stop 2024-04-25T16:51:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.855+0000 7f1d372a1640 1 -- 172.21.15.94:0/3120038814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d30072b60 msgr2=0x7f1d300b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.855+0000 7f1d372a1640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d30072b60 0x7f1d300b0810 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1d200ad040 tx=0x7f1d2001bd70 comp rx=0 tx=0).stop 2024-04-25T16:51:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.855+0000 7f1d372a1640 1 -- 172.21.15.94:0/3120038814 shutdown_connections 2024-04-25T16:51:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.859+0000 7f1d372a1640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d1c06f400 0x7f1d1c071910 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.859+0000 7f1d372a1640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d30145820 0x7f1d3009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.859+0000 7f1d372a1640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d3015cd60 0x7f1d3009e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.859+0000 7f1d372a1640 1 --2- 172.21.15.94:0/3120038814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d30072b60 0x7f1d300b0810 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.859+0000 7f1d372a1640 1 -- 172.21.15.94:0/3120038814 >> 172.21.15.94:0/3120038814 conn(0x7f1d300b47f0 msgr2=0x7f1d300d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.859+0000 7f1d372a1640 1 -- 172.21.15.94:0/3120038814 shutdown_connections 2024-04-25T16:51:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:54.859+0000 7f1d372a1640 1 -- 172.21.15.94:0/3120038814 wait complete. 2024-04-25T16:51:54.874 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:51:55.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.015+0000 7f6e1b9f3640 1 Processor -- start 2024-04-25T16:51:55.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.015+0000 7f6e1b9f3640 1 -- start start 2024-04-25T16:51:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e1b9f3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e14145cb0 0x7f6e140d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e1b9f3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e140b6950 con 0x7f6e14142e20 2024-04-25T16:51:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e1b9f3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e140b6c30 con 0x7f6e141460c0 2024-04-25T16:51:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e1b9f3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6e140b6f10 con 0x7f6e14145cb0 2024-04-25T16:51:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e1978f640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f6e14142e20 0x7f6e1406fd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49534/0 (socket says 172.21.15.94:49534) 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e1978f640 1 -- 172.21.15.94:0/3732536659 learned_addr learned my addr 172.21.15.94:0/3732536659 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510279961 0 0) 0x7f6e140b6950 con 0x7f6e14142e20 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e18f8e640 1 --2- 172.21.15.94:0/3732536659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e14145cb0 0x7f6e140d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e00003530 con 0x7f6e14142e20 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2092267796 0 0) 0x7f6e00003530 con 0x7f6e14142e20 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e14145cb0 msgr2=0x7f6e140d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 --2- 172.21.15.94:0/3732536659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e14145cb0 0x7f6e140d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 >> v1:172.21.15.138:6789/0 conn(0x7f6e141460c0 legacy=0x7f6e141434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1406f560 con 0x7f6e14142e20 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f6e10001680 con 0x7f6e14142e20 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 >> v1:172.21.15.94:6789/0 conn(0x7f6e14142e20 legacy=0x7f6e1406fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 --2- 172.21.15.94:0/3732536659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e14145cb0 0x7f6e000039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 --2- 172.21.15.94:0/3732536659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e00003f70 0x7f6e00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 --2- 172.21.15.94:0/3732536659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00004d50 0x7f6e00005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e140b6f10 con 0x7f6e00003f70 2024-04-25T16:51:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0000bc90 con 0x7f6e00004d50 2024-04-25T16:51:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0000e0a0 con 0x7f6e14145cb0 2024-04-25T16:51:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e18f8e640 1 --2- 172.21.15.94:0/3732536659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e00003f70 0x7f6e00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e1978f640 1 --2- 172.21.15.94:0/3732536659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e14145cb0 0x7f6e000039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e18f8e640 1 -- 172.21.15.94:0/3732536659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e14145cb0 msgr2=0x7f6e000039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e18f8e640 1 --2- 172.21.15.94:0/3732536659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e14145cb0 0x7f6e000039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e18f8e640 1 -- 172.21.15.94:0/3732536659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00004d50 msgr2=0x7f6e00005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e19f90640 1 --2- 172.21.15.94:0/3732536659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00004d50 0x7f6e00005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e18f8e640 1 --2- 172.21.15.94:0/3732536659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00004d50 0x7f6e00005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.019+0000 7f6e18f8e640 1 -- 172.21.15.94:0/3732536659 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e00003530 con 0x7f6e00003f70 2024-04-25T16:51:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.023+0000 7f6e18f8e640 1 --2- 172.21.15.94:0/3732536659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e00003f70 0x7f6e00004810 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009ae0 tx=0x7f6e0401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba7396930b8036d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.023+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e040b2070 con 0x7f6e00003f70 2024-04-25T16:51:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.023+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e040a5570 con 0x7f6e00003f70 2024-04-25T16:51:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.023+0000 7f6e0bfff640 1 -- 172.21.15.94:0/3732536659 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e040096b0 con 0x7f6e00003f70 2024-04-25T16:51:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.023+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/3732536659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e00003f70 msgr2=0x7f6e00004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.023+0000 7f6e1b9f3640 1 --2- 172.21.15.94:0/3732536659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e00003f70 0x7f6e00004810 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009ae0 tx=0x7f6e0401ca90 comp rx=0 tx=0).stop 2024-04-25T16:51:55.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.023+0000 7f6e19f90640 1 -- 172.21.15.94:0/3732536659 reap_dead start 2024-04-25T16:51:55.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.023+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/3732536659 shutdown_connections 2024-04-25T16:51:55.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.023+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/3732536659 >> 172.21.15.94:0/3732536659 conn(0x7f6e14071f00 msgr2=0x7f6e14073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:55.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.023+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/3732536659 shutdown_connections 2024-04-25T16:51:55.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.023+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/3732536659 wait complete. 2024-04-25T16:51:55.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.023+0000 7f6e1b9f3640 1 Processor -- start 2024-04-25T16:51:55.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.023+0000 7f6e1b9f3640 1 -- start start 2024-04-25T16:51:55.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e1b9f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00003f70 0x7f6e141583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e1b9f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e141401b0 0x7f6e14161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e1b9f3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e1415b530 0x7f6e1415b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e1b9f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1406ce90 con 0x7f6e141401b0 2024-04-25T16:51:55.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e1b9f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e140d1e30 con 0x7f6e00003f70 2024-04-25T16:51:55.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e1b9f3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6e1406d010 con 0x7f6e1415b530 2024-04-25T16:51:55.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e18f8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e141401b0 0x7f6e14161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e18f8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e141401b0 0x7f6e14161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59850/0 (socket says 172.21.15.94:59850) 2024-04-25T16:51:55.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e18f8e640 1 -- 172.21.15.94:0/1061690236 learned_addr learned my addr 172.21.15.94:0/1061690236 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:55.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e1978f640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00003f70 0x7f6e141583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e19f90640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e1415b530 0x7f6e1415b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e18f8e640 1 -- 172.21.15.94:0/1061690236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e1415b530 msgr2=0x7f6e1415b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e18f8e640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e1415b530 0x7f6e1415b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e18f8e640 1 -- 172.21.15.94:0/1061690236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00003f70 msgr2=0x7f6e141583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e18f8e640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00003f70 0x7f6e141583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e18f8e640 1 -- 172.21.15.94:0/1061690236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e141405c0 con 0x7f6e141401b0 2024-04-25T16:51:55.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e18f8e640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e141401b0 0x7f6e14161060 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009ae0 tx=0x7f6e00005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e09ffb640 1 -- 172.21.15.94:0/1061690236 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e040bd070 con 0x7f6e141401b0 2024-04-25T16:51:55.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e09ffb640 1 -- 172.21.15.94:0/1061690236 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e040a5d20 con 0x7f6e141401b0 2024-04-25T16:51:55.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/1061690236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1415a490 con 0x7f6e141401b0 2024-04-25T16:51:55.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e09ffb640 1 -- 172.21.15.94:0/1061690236 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e040b2040 con 0x7f6e141401b0 2024-04-25T16:51:55.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/1061690236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e1415aa50 con 0x7f6e141401b0 2024-04-25T16:51:55.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/1061690236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e1406a730 con 0x7f6e141401b0 2024-04-25T16:51:55.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e09ffb640 1 -- 172.21.15.94:0/1061690236 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6e040c0070 con 0x7f6e141401b0 2024-04-25T16:51:55.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e09ffb640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6df806f430 0x7f6df8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e09ffb640 1 -- 172.21.15.94:0/1061690236 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6e041290d0 con 0x7f6e141401b0 2024-04-25T16:51:55.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.027+0000 7f6e1978f640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6df806f430 0x7f6df8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.031+0000 7f6e1978f640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6df806f430 0x7f6df8071940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6e10001ec0 tx=0x7f6e10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.035+0000 7f6e09ffb640 1 -- 172.21.15.94:0/1061690236 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6e040167a0 con 0x7f6e141401b0 2024-04-25T16:51:55.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.191+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/1061690236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6e1406ea50 con 0x7f6e141401b0 2024-04-25T16:51:55.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.191+0000 7f6e09ffb640 1 -- 172.21.15.94:0/1061690236 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f6e040f5770 con 0x7f6e141401b0 2024-04-25T16:51:55.190 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.195+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/1061690236 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6df806f430 msgr2=0x7f6df8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.195+0000 7f6e1b9f3640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6df806f430 0x7f6df8071940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6e10001ec0 tx=0x7f6e10009040 comp rx=0 tx=0).stop 2024-04-25T16:51:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.195+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/1061690236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e141401b0 msgr2=0x7f6e14161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.195+0000 7f6e1b9f3640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e141401b0 0x7f6e14161060 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009ae0 tx=0x7f6e00005840 comp rx=0 tx=0).stop 2024-04-25T16:51:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.195+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/1061690236 shutdown_connections 2024-04-25T16:51:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.195+0000 7f6e1b9f3640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f6df806f430 0x7f6df8071940 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.195+0000 7f6e1b9f3640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e1415b530 0x7f6e1415b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.195+0000 7f6e1b9f3640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e141401b0 0x7f6e14161060 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.195+0000 7f6e1b9f3640 1 --2- 172.21.15.94:0/1061690236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00003f70 0x7f6e141583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.195+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/1061690236 >> 172.21.15.94:0/1061690236 conn(0x7f6e14071f00 msgr2=0x7f6e14142070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.195+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/1061690236 shutdown_connections 2024-04-25T16:51:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.195+0000 7f6e1b9f3640 1 -- 172.21.15.94:0/1061690236 wait complete. 2024-04-25T16:51:55.210 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:51:55.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c630b3640 1 Processor -- start 2024-04-25T16:51:55.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c630b3640 1 -- start start 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c630b3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9c5c0ce010 0x7f9c5c1681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c630b3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5c168b30 con 0x7f9c5c164170 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c630b3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5c0ce9f0 con 0x7f9c5c168720 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c630b3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9c5c0d10a0 con 0x7f9c5c0ce010 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c60e4f640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f9c5c164170 0x7f9c5c0b6e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49550/0 (socket says 172.21.15.94:49550) 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c60e4f640 1 -- 172.21.15.94:0/3189310675 learned_addr learned my addr 172.21.15.94:0/3189310675 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2125804068 0 0) 0x7f9c5c168b30 con 0x7f9c5c164170 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c5bfff640 1 --2- 172.21.15.94:0/3189310675 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c0ce010 0x7f9c5c1681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c48003530 con 0x7f9c5c164170 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2072373364 0 0) 0x7f9c48003530 con 0x7f9c5c164170 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c0ce010 msgr2=0x7f9c5c1681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c5b7fe640 1 --2- 172.21.15.94:0/3189310675 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c0ce010 0x7f9c5c1681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 >> v1:172.21.15.138:6789/0 conn(0x7f9c5c168720 legacy=0x7f9c5c073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.347+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c0b5390 con 0x7f9c5c164170 2024-04-25T16:51:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f9c54001a00 con 0x7f9c5c164170 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 >> v1:172.21.15.94:6789/0 conn(0x7f9c5c164170 legacy=0x7f9c5c0b6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5b7fe640 1 --2- 172.21.15.94:0/3189310675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c480039c0 0x7f9c48003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5b7fe640 1 --2- 172.21.15.94:0/3189310675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c48004810 0x7f9c48004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5b7fe640 1 --2- 172.21.15.94:0/3189310675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c48005310 0x7f9c4800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c0d10a0 con 0x7f9c48004810 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4800c2b0 con 0x7f9c48005310 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9c4800c4b0 con 0x7f9c480039c0 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5bfff640 1 --2- 172.21.15.94:0/3189310675 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c0ce010 0x7f9c5c1681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5bfff640 1 --2- 172.21.15.94:0/3189310675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c48004810 0x7f9c48004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c60e4f640 1 --2- 172.21.15.94:0/3189310675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c480039c0 0x7f9c48003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c61650640 1 --2- 172.21.15.94:0/3189310675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c48005310 0x7f9c4800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5bfff640 1 -- 172.21.15.94:0/3189310675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c480039c0 msgr2=0x7f9c48003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5bfff640 1 --2- 172.21.15.94:0/3189310675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c480039c0 0x7f9c48003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5bfff640 1 -- 172.21.15.94:0/3189310675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c48005310 msgr2=0x7f9c4800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5bfff640 1 --2- 172.21.15.94:0/3189310675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c48005310 0x7f9c4800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c61650640 1 -- 172.21.15.94:0/3189310675 reap_dead start 2024-04-25T16:51:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5bfff640 1 -- 172.21.15.94:0/3189310675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c48003530 con 0x7f9c48004810 2024-04-25T16:51:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5bfff640 1 --2- 172.21.15.94:0/3189310675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c48004810 0x7f9c48004c20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9c440098b0 tx=0x7f9c4401c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a667a7e9264d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4400f040 con 0x7f9c48004810 2024-04-25T16:51:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c440a8050 con 0x7f9c48004810 2024-04-25T16:51:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c5b7fe640 1 -- 172.21.15.94:0/3189310675 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44004440 con 0x7f9c48004810 2024-04-25T16:51:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c630b3640 1 -- 172.21.15.94:0/3189310675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c48004810 msgr2=0x7f9c48004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c630b3640 1 --2- 172.21.15.94:0/3189310675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c48004810 0x7f9c48004c20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9c440098b0 tx=0x7f9c4401c5b0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c630b3640 1 -- 172.21.15.94:0/3189310675 shutdown_connections 2024-04-25T16:51:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c630b3640 1 --2- 172.21.15.94:0/3189310675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c48004810 0x7f9c48004c20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c630b3640 1 -- 172.21.15.94:0/3189310675 >> 172.21.15.94:0/3189310675 conn(0x7f9c5c0d2690 msgr2=0x7f9c5c0d2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c630b3640 1 -- 172.21.15.94:0/3189310675 shutdown_connections 2024-04-25T16:51:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.351+0000 7f9c630b3640 1 -- 172.21.15.94:0/3189310675 wait complete. 2024-04-25T16:51:55.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c630b3640 1 Processor -- start 2024-04-25T16:51:55.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c630b3640 1 -- start start 2024-04-25T16:51:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c630b3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c48004810 0x7f9c5c161db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c630b3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c5c0c8220 0x7f9c5c161420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c630b3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c5c162b60 0x7f9c5c162f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c630b3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c073b90 con 0x7f9c5c162b60 2024-04-25T16:51:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c630b3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c073a10 con 0x7f9c5c0c8220 2024-04-25T16:51:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c630b3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5c073d10 con 0x7f9c48004810 2024-04-25T16:51:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c61650640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c5c162b60 0x7f9c5c162f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c61650640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c5c162b60 0x7f9c5c162f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59866/0 (socket says 172.21.15.94:59866) 2024-04-25T16:51:55.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c61650640 1 -- 172.21.15.94:0/4270205625 learned_addr learned my addr 172.21.15.94:0/4270205625 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:55.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c60e4f640 1 --2- 172.21.15.94:0/4270205625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c48004810 0x7f9c5c161db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c61650640 1 -- 172.21.15.94:0/4270205625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c48004810 msgr2=0x7f9c5c161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c61650640 1 --2- 172.21.15.94:0/4270205625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c48004810 0x7f9c5c161db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c61650640 1 -- 172.21.15.94:0/4270205625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c5c0c8220 msgr2=0x7f9c5c161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c61650640 1 --2- 172.21.15.94:0/4270205625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c5c0c8220 0x7f9c5c161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c61650640 1 -- 172.21.15.94:0/4270205625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c1634b0 con 0x7f9c5c162b60 2024-04-25T16:51:55.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c61650640 1 --2- 172.21.15.94:0/4270205625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c5c162b60 0x7f9c5c162f70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c073790 tx=0x7f9c4c00e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c597fa640 1 -- 172.21.15.94:0/4270205625 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c019070 con 0x7f9c5c162b60 2024-04-25T16:51:55.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c597fa640 1 -- 172.21.15.94:0/4270205625 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c4c005070 con 0x7f9c5c162b60 2024-04-25T16:51:55.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c630b3640 1 -- 172.21.15.94:0/4270205625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5c0c8a00 con 0x7f9c5c162b60 2024-04-25T16:51:55.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c597fa640 1 -- 172.21.15.94:0/4270205625 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c005440 con 0x7f9c5c162b60 2024-04-25T16:51:55.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.355+0000 7f9c630b3640 1 -- 172.21.15.94:0/4270205625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5c0c8fc0 con 0x7f9c5c162b60 2024-04-25T16:51:55.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.359+0000 7f9c630b3640 1 -- 172.21.15.94:0/4270205625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c5c0c8750 con 0x7f9c5c162b60 2024-04-25T16:51:55.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.359+0000 7f9c597fa640 1 -- 172.21.15.94:0/4270205625 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9c4c007030 con 0x7f9c5c162b60 2024-04-25T16:51:55.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.359+0000 7f9c597fa640 1 --2- 172.21.15.94:0/4270205625 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9c3c06f430 0x7f9c3c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.359+0000 7f9c597fa640 1 -- 172.21.15.94:0/4270205625 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9c4c08e880 con 0x7f9c5c162b60 2024-04-25T16:51:55.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.359+0000 7f9c60e4f640 1 --2- 172.21.15.94:0/4270205625 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9c3c06f430 0x7f9c3c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.359+0000 7f9c60e4f640 1 --2- 172.21.15.94:0/4270205625 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9c3c06f430 0x7f9c3c071940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9c54002240 tx=0x7f9c54002110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.363+0000 7f9c597fa640 1 -- 172.21.15.94:0/4270205625 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9c5c1634b0 con 0x7f9c5c162b60 2024-04-25T16:51:55.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.543+0000 7f9c630b3640 1 -- 172.21.15.94:0/4270205625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9c5c06a730 con 0x7f9c5c162b60 2024-04-25T16:51:55.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.543+0000 7f9c597fa640 1 -- 172.21.15.94:0/4270205625 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f9c4c05b370 con 0x7f9c5c162b60 2024-04-25T16:51:55.542 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:51:55.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.547+0000 7f9c630b3640 1 -- 172.21.15.94:0/4270205625 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9c3c06f430 msgr2=0x7f9c3c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.547+0000 7f9c630b3640 1 --2- 172.21.15.94:0/4270205625 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9c3c06f430 0x7f9c3c071940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9c54002240 tx=0x7f9c54002110 comp rx=0 tx=0).stop 2024-04-25T16:51:55.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.547+0000 7f9c630b3640 1 -- 172.21.15.94:0/4270205625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c5c162b60 msgr2=0x7f9c5c162f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.547+0000 7f9c630b3640 1 --2- 172.21.15.94:0/4270205625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c5c162b60 0x7f9c5c162f70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c073790 tx=0x7f9c4c00e820 comp rx=0 tx=0).stop 2024-04-25T16:51:55.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.547+0000 7f9c630b3640 1 -- 172.21.15.94:0/4270205625 shutdown_connections 2024-04-25T16:51:55.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.547+0000 7f9c630b3640 1 --2- 172.21.15.94:0/4270205625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c5c162b60 0x7f9c5c162f70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.547+0000 7f9c630b3640 1 --2- 172.21.15.94:0/4270205625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c5c0c8220 0x7f9c5c161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.547+0000 7f9c630b3640 1 --2- 172.21.15.94:0/4270205625 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9c3c06f430 0x7f9c3c071940 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.547+0000 7f9c630b3640 1 --2- 172.21.15.94:0/4270205625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c48004810 0x7f9c5c161db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.547+0000 7f9c630b3640 1 -- 172.21.15.94:0/4270205625 >> 172.21.15.94:0/4270205625 conn(0x7f9c5c0d2690 msgr2=0x7f9c5c166080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:55.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.547+0000 7f9c630b3640 1 -- 172.21.15.94:0/4270205625 shutdown_connections 2024-04-25T16:51:55.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.547+0000 7f9c630b3640 1 -- 172.21.15.94:0/4270205625 wait complete. 2024-04-25T16:51:55.563 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:51:55.564 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:51:55.569 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:55.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c46cc640 1 Processor -- start 2024-04-25T16:51:55.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c46cc640 1 -- start start 2024-04-25T16:51:55.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c46cc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f24bc15ff50 0x7f24bc0d5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c46cc640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24bc0d1d40 con 0x7f24bc06f6c0 2024-04-25T16:51:55.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c46cc640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24bc0b2240 con 0x7f24bc0d3be0 2024-04-25T16:51:55.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c46cc640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f24bc0d2fd0 con 0x7f24bc15ff50 2024-04-25T16:51:55.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c2468640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f24bc06f6c0 0x7f24bc0d34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49566/0 (socket says 172.21.15.94:49566) 2024-04-25T16:51:55.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c2468640 1 -- 172.21.15.94:0/3790415588 learned_addr learned my addr 172.21.15.94:0/3790415588 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:55.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c2c69640 1 --2- 172.21.15.94:0/3790415588 >> v2:172.21.15.94:3300/0 conn(0x7f24bc15ff50 0x7f24bc0d5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c1466640 1 -- 172.21.15.94:0/3790415588 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3948277793 0 0) 0x7f24bc0d1d40 con 0x7f24bc06f6c0 2024-04-25T16:51:55.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c1466640 1 -- 172.21.15.94:0/3790415588 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24a8003530 con 0x7f24bc06f6c0 2024-04-25T16:51:55.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c2c69640 1 -- 172.21.15.94:0/3790415588 >> v1:172.21.15.138:6789/0 conn(0x7f24bc0d3be0 legacy=0x7f24bc0d4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c2c69640 1 -- 172.21.15.94:0/3790415588 >> v1:172.21.15.94:6789/0 conn(0x7f24bc06f6c0 legacy=0x7f24bc0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.751+0000 7f24c2c69640 1 -- 172.21.15.94:0/3790415588 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc0d2cf0 con 0x7f24bc15ff50 2024-04-25T16:51:55.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.755+0000 7f24c2c69640 1 --2- 172.21.15.94:0/3790415588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc15ff50 0x7f24bc0d5500 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009d90 tx=0x7f24b401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c6d8bae0641a29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.755+0000 7f24c1466640 1 -- 172.21.15.94:0/3790415588 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b40b2070 con 0x7f24bc15ff50 2024-04-25T16:51:55.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.755+0000 7f24c1466640 1 -- 172.21.15.94:0/3790415588 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b40a5a50 con 0x7f24bc15ff50 2024-04-25T16:51:55.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.755+0000 7f24c1466640 1 -- 172.21.15.94:0/3790415588 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b40a5e00 con 0x7f24bc15ff50 2024-04-25T16:51:55.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.755+0000 7f24c46cc640 1 -- 172.21.15.94:0/3790415588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc15ff50 msgr2=0x7f24bc0d5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.755+0000 7f24c46cc640 1 --2- 172.21.15.94:0/3790415588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc15ff50 0x7f24bc0d5500 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009d90 tx=0x7f24b401c970 comp rx=0 tx=0).stop 2024-04-25T16:51:55.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.755+0000 7f24c46cc640 1 -- 172.21.15.94:0/3790415588 shutdown_connections 2024-04-25T16:51:55.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.755+0000 7f24c46cc640 1 --2- 172.21.15.94:0/3790415588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc15ff50 0x7f24bc0d5500 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.755+0000 7f24c46cc640 1 -- 172.21.15.94:0/3790415588 >> 172.21.15.94:0/3790415588 conn(0x7f24bc09e950 msgr2=0x7f24bc08d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:55.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.755+0000 7f24c46cc640 1 -- 172.21.15.94:0/3790415588 shutdown_connections 2024-04-25T16:51:55.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.755+0000 7f24c46cc640 1 -- 172.21.15.94:0/3790415588 wait complete. 2024-04-25T16:51:55.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c46cc640 1 Processor -- start 2024-04-25T16:51:55.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c46cc640 1 -- start start 2024-04-25T16:51:55.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c46cc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc06f6c0 0x7f24bc0cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c46cc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24bc15ff50 0x7f24bc150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c46cc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24bc14e320 0x7f24bc153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c46cc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc0d2aa0 con 0x7f24bc06f6c0 2024-04-25T16:51:55.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c46cc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc06fe20 con 0x7f24bc15ff50 2024-04-25T16:51:55.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c46cc640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f24bc16e810 con 0x7f24bc14e320 2024-04-25T16:51:55.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c2468640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc06f6c0 0x7f24bc0cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c2468640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc06f6c0 0x7f24bc0cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59888/0 (socket says 172.21.15.94:59888) 2024-04-25T16:51:55.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c2468640 1 -- 172.21.15.94:0/2817837382 learned_addr learned my addr 172.21.15.94:0/2817837382 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:55.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c1c67640 1 --2- 172.21.15.94:0/2817837382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24bc15ff50 0x7f24bc150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c2468640 1 -- 172.21.15.94:0/2817837382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24bc14e320 msgr2=0x7f24bc153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c2468640 1 --2- 172.21.15.94:0/2817837382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24bc14e320 0x7f24bc153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c2468640 1 -- 172.21.15.94:0/2817837382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24bc15ff50 msgr2=0x7f24bc150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c2468640 1 --2- 172.21.15.94:0/2817837382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24bc15ff50 0x7f24bc150f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c2468640 1 -- 172.21.15.94:0/2817837382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc1534e0 con 0x7f24bc06f6c0 2024-04-25T16:51:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c2468640 1 --2- 172.21.15.94:0/2817837382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc06f6c0 0x7f24bc0cb340 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c460 tx=0x7f24b800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24b2ffd640 1 -- 172.21.15.94:0/2817837382 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8017070 con 0x7f24bc06f6c0 2024-04-25T16:51:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24b2ffd640 1 -- 172.21.15.94:0/2817837382 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b8007480 con 0x7f24bc06f6c0 2024-04-25T16:51:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c46cc640 1 -- 172.21.15.94:0/2817837382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24bc14d840 con 0x7f24bc06f6c0 2024-04-25T16:51:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24b2ffd640 1 -- 172.21.15.94:0/2817837382 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8007630 con 0x7f24bc06f6c0 2024-04-25T16:51:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c46cc640 1 -- 172.21.15.94:0/2817837382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24bc0cb880 con 0x7f24bc06f6c0 2024-04-25T16:51:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.759+0000 7f24c46cc640 1 -- 172.21.15.94:0/2817837382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24bc14d630 con 0x7f24bc06f6c0 2024-04-25T16:51:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.763+0000 7f24b2ffd640 1 -- 172.21.15.94:0/2817837382 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f24b801a070 con 0x7f24bc06f6c0 2024-04-25T16:51:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.763+0000 7f24b2ffd640 1 --2- 172.21.15.94:0/2817837382 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f24a006f430 0x7f24a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.763+0000 7f24b2ffd640 1 -- 172.21.15.94:0/2817837382 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f24b808ef60 con 0x7f24bc06f6c0 2024-04-25T16:51:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.763+0000 7f24c1c67640 1 --2- 172.21.15.94:0/2817837382 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f24a006f430 0x7f24a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.763+0000 7f24c1c67640 1 --2- 172.21.15.94:0/2817837382 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f24a006f430 0x7f24a0071940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f24bc16b9a0 tx=0x7f24ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.767+0000 7f24b2ffd640 1 -- 172.21.15.94:0/2817837382 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f24b805b600 con 0x7f24bc06f6c0 2024-04-25T16:51:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.923+0000 7f24c46cc640 1 -- 172.21.15.94:0/2817837382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24bc076830 con 0x7f24bc06f6c0 2024-04-25T16:51:55.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.927+0000 7f24b2ffd640 1 -- 172.21.15.94:0/2817837382 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7f24b805f900 con 0x7f24bc06f6c0 2024-04-25T16:51:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:55.925 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:55.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.927+0000 7f24c46cc640 1 -- 172.21.15.94:0/2817837382 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f24a006f430 msgr2=0x7f24a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.927+0000 7f24c46cc640 1 --2- 172.21.15.94:0/2817837382 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f24a006f430 0x7f24a0071940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f24bc16b9a0 tx=0x7f24ac009040 comp rx=0 tx=0).stop 2024-04-25T16:51:55.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.927+0000 7f24c46cc640 1 -- 172.21.15.94:0/2817837382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc06f6c0 msgr2=0x7f24bc0cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.927+0000 7f24c46cc640 1 --2- 172.21.15.94:0/2817837382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc06f6c0 0x7f24bc0cb340 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c460 tx=0x7f24b800c920 comp rx=0 tx=0).stop 2024-04-25T16:51:55.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.927+0000 7f24c46cc640 1 -- 172.21.15.94:0/2817837382 shutdown_connections 2024-04-25T16:51:55.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.927+0000 7f24c46cc640 1 --2- 172.21.15.94:0/2817837382 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f24a006f430 0x7f24a0071940 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.927+0000 7f24c46cc640 1 --2- 172.21.15.94:0/2817837382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24bc14e320 0x7f24bc153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.927+0000 7f24c46cc640 1 --2- 172.21.15.94:0/2817837382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24bc15ff50 0x7f24bc150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.927+0000 7f24c46cc640 1 --2- 172.21.15.94:0/2817837382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc06f6c0 0x7f24bc0cb340 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.927+0000 7f24c46cc640 1 -- 172.21.15.94:0/2817837382 >> 172.21.15.94:0/2817837382 conn(0x7f24bc09e950 msgr2=0x7f24bc08d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:55.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.927+0000 7f24c46cc640 1 -- 172.21.15.94:0/2817837382 shutdown_connections 2024-04-25T16:51:55.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:55.927+0000 7f24c46cc640 1 -- 172.21.15.94:0/2817837382 wait complete. 2024-04-25T16:51:55.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:51:55.945 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:51:56.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f824a9bb640 1 Processor -- start 2024-04-25T16:51:56.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f824a9bb640 1 -- start start 2024-04-25T16:51:56.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f824a9bb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82440d3450 0x7f82440d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f824a9bb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f82440d2440 0x7f82440b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f824a9bb640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8244164510 con 0x7f82440d3da0 2024-04-25T16:51:56.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f824a9bb640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f8244164a90 con 0x7f82440d2440 2024-04-25T16:51:56.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f824a9bb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f82440d51b0 con 0x7f82440d3450 2024-04-25T16:51:56.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f823bfff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f82440d3da0 0x7f82440b6fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49572/0 (socket says 172.21.15.94:49572) 2024-04-25T16:51:56.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f823bfff640 1 -- 172.21.15.94:0/1624752247 learned_addr learned my addr 172.21.15.94:0/1624752247 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:56.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f8248f58640 1 --2- 172.21.15.94:0/1624752247 >> v2:172.21.15.94:3300/0 conn(0x7f82440d2440 0x7f82440b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f82437fe640 1 -- 172.21.15.94:0/1624752247 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1867030340 0 0) 0x7f8244164510 con 0x7f82440d3da0 2024-04-25T16:51:56.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f8243fff640 1 --2- 172.21.15.94:0/1624752247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82440d3450 0x7f82440d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f82437fe640 1 -- 172.21.15.94:0/1624752247 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8230003530 con 0x7f82440d3da0 2024-04-25T16:51:56.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f8248f58640 1 -- 172.21.15.94:0/1624752247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82440d3450 msgr2=0x7f82440d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:56.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f8248f58640 1 --2- 172.21.15.94:0/1624752247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82440d3450 0x7f82440d3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f8248f58640 1 -- 172.21.15.94:0/1624752247 >> v1:172.21.15.94:6789/0 conn(0x7f82440d3da0 legacy=0x7f82440b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.091+0000 7f8248f58640 1 -- 172.21.15.94:0/1624752247 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82441647b0 con 0x7f82440d2440 2024-04-25T16:51:56.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.095+0000 7f8248f58640 1 --2- 172.21.15.94:0/1624752247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82440d2440 0x7f82440b76e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8244164c10 tx=0x7f823c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad69eca154be338 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:56.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.095+0000 7f82437fe640 1 -- 172.21.15.94:0/1624752247 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c0b2070 con 0x7f82440d2440 2024-04-25T16:51:56.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.095+0000 7f82437fe640 1 -- 172.21.15.94:0/1624752247 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f823c0a5950 con 0x7f82440d2440 2024-04-25T16:51:56.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.095+0000 7f82437fe640 1 -- 172.21.15.94:0/1624752247 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c0a5d20 con 0x7f82440d2440 2024-04-25T16:51:56.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.095+0000 7f824a9bb640 1 -- 172.21.15.94:0/1624752247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82440d2440 msgr2=0x7f82440b76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.095+0000 7f824a9bb640 1 --2- 172.21.15.94:0/1624752247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82440d2440 0x7f82440b76e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8244164c10 tx=0x7f823c01c890 comp rx=0 tx=0).stop 2024-04-25T16:51:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.095+0000 7f824a9bb640 1 -- 172.21.15.94:0/1624752247 shutdown_connections 2024-04-25T16:51:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.095+0000 7f824a9bb640 1 --2- 172.21.15.94:0/1624752247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82440d2440 0x7f82440b76e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.095+0000 7f824a9bb640 1 --2- 172.21.15.94:0/1624752247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82440d3450 0x7f82440d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.095+0000 7f824a9bb640 1 -- 172.21.15.94:0/1624752247 >> 172.21.15.94:0/1624752247 conn(0x7f824409e8d0 msgr2=0x7f82440b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.095+0000 7f824a9bb640 1 -- 172.21.15.94:0/1624752247 shutdown_connections 2024-04-25T16:51:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.095+0000 7f824a9bb640 1 -- 172.21.15.94:0/1624752247 wait complete. 2024-04-25T16:51:56.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f824a9bb640 1 Processor -- start 2024-04-25T16:51:56.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f824a9bb640 1 -- start start 2024-04-25T16:51:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f824a9bb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82440d2440 0x7f82440c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f824a9bb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82440d3450 0x7f8244152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f824a9bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82440d3da0 0x7f82440cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f824a9bb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82440b8cd0 con 0x7f82440d3da0 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f824a9bb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82440d41b0 con 0x7f82440d2440 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f824a9bb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f82440b8e50 con 0x7f82440d3450 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f8248f58640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82440d3da0 0x7f82440cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f8248f58640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82440d3da0 0x7f82440cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59908/0 (socket says 172.21.15.94:59908) 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f8248f58640 1 -- 172.21.15.94:0/824706193 learned_addr learned my addr 172.21.15.94:0/824706193 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f8248f58640 1 -- 172.21.15.94:0/824706193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82440d3450 msgr2=0x7f8244152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f8248f58640 1 --2- 172.21.15.94:0/824706193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82440d3450 0x7f8244152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f8248f58640 1 -- 172.21.15.94:0/824706193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82440d2440 msgr2=0x7f82440c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f8248f58640 1 --2- 172.21.15.94:0/824706193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82440d2440 0x7f82440c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f8248f58640 1 -- 172.21.15.94:0/824706193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824415df60 con 0x7f82440d3da0 2024-04-25T16:51:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f8248f58640 1 --2- 172.21.15.94:0/824706193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82440d3da0 0x7f82440cb440 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f823c01a4c0 tx=0x7f823c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f82417fa640 1 -- 172.21.15.94:0/824706193 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c0b2050 con 0x7f82440d3da0 2024-04-25T16:51:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f82417fa640 1 -- 172.21.15.94:0/824706193 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f823c0be070 con 0x7f82440d3da0 2024-04-25T16:51:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f82417fa640 1 -- 172.21.15.94:0/824706193 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c0ada90 con 0x7f82440d3da0 2024-04-25T16:51:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f824a9bb640 1 -- 172.21.15.94:0/824706193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824415e240 con 0x7f82440d3da0 2024-04-25T16:51:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f824a9bb640 1 -- 172.21.15.94:0/824706193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824415e800 con 0x7f82440d3da0 2024-04-25T16:51:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.099+0000 7f824a9bb640 1 -- 172.21.15.94:0/824706193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8244076830 con 0x7f82440d3da0 2024-04-25T16:51:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.103+0000 7f82417fa640 1 -- 172.21.15.94:0/824706193 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f823c0adc30 con 0x7f82440d3da0 2024-04-25T16:51:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.103+0000 7f82417fa640 1 --2- 172.21.15.94:0/824706193 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f822406f430 0x7f8224071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.103+0000 7f82417fa640 1 -- 172.21.15.94:0/824706193 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f823c12a310 con 0x7f82440d3da0 2024-04-25T16:51:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.103+0000 7f8243fff640 1 --2- 172.21.15.94:0/824706193 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f822406f430 0x7f8224071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.103+0000 7f8243fff640 1 --2- 172.21.15.94:0/824706193 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f822406f430 0x7f8224071940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f82440b8c60 tx=0x7f8234008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:56.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.107+0000 7f82417fa640 1 -- 172.21.15.94:0/824706193 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f823c0c7020 con 0x7f82440d3da0 2024-04-25T16:51:56.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.255+0000 7f824a9bb640 1 -- 172.21.15.94:0/824706193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f82440d2c40 con 0x7f82440d3da0 2024-04-25T16:51:56.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.255+0000 7f82417fa640 1 -- 172.21.15.94:0/824706193 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f823c016540 con 0x7f82440d3da0 2024-04-25T16:51:56.254 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:51:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.255+0000 7f824a9bb640 1 -- 172.21.15.94:0/824706193 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f822406f430 msgr2=0x7f8224071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.255+0000 7f824a9bb640 1 --2- 172.21.15.94:0/824706193 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f822406f430 0x7f8224071940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f82440b8c60 tx=0x7f8234008040 comp rx=0 tx=0).stop 2024-04-25T16:51:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.255+0000 7f824a9bb640 1 -- 172.21.15.94:0/824706193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82440d3da0 msgr2=0x7f82440cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.255+0000 7f824a9bb640 1 --2- 172.21.15.94:0/824706193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82440d3da0 0x7f82440cb440 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f823c01a4c0 tx=0x7f823c019630 comp rx=0 tx=0).stop 2024-04-25T16:51:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.259+0000 7f824a9bb640 1 -- 172.21.15.94:0/824706193 shutdown_connections 2024-04-25T16:51:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.259+0000 7f824a9bb640 1 --2- 172.21.15.94:0/824706193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82440d3da0 0x7f82440cb440 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.259+0000 7f824a9bb640 1 --2- 172.21.15.94:0/824706193 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f822406f430 0x7f8224071940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.259+0000 7f824a9bb640 1 --2- 172.21.15.94:0/824706193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82440d3450 0x7f8244152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.259+0000 7f824a9bb640 1 --2- 172.21.15.94:0/824706193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82440d2440 0x7f82440c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.259+0000 7f824a9bb640 1 -- 172.21.15.94:0/824706193 >> 172.21.15.94:0/824706193 conn(0x7f824409e8d0 msgr2=0x7f82440b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.259+0000 7f824a9bb640 1 -- 172.21.15.94:0/824706193 shutdown_connections 2024-04-25T16:51:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.259+0000 7f824a9bb640 1 -- 172.21.15.94:0/824706193 wait complete. 2024-04-25T16:51:56.273 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:51:56.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.427+0000 7fb3d6296640 1 Processor -- start 2024-04-25T16:51:56.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.427+0000 7fb3d6296640 1 -- start start 2024-04-25T16:51:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3d6296640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb3d006d710 0x7fb3d006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3d6296640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3d006e7a0 0x7fb3d006ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3d6296640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3d0074380 con 0x7fb3d015c7e0 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3d6296640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb3d006e060 con 0x7fb3d006d710 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3d6296640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d00727e0 con 0x7fb3d006e7a0 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3cffff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fb3d015c7e0 0x7fb3d00afdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49580/0 (socket says 172.21.15.94:49580) 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3cffff640 1 -- 172.21.15.94:0/637230550 learned_addr learned my addr 172.21.15.94:0/637230550 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3cf7fe640 1 --2- 172.21.15.94:0/637230550 >> v2:172.21.15.138:3300/0 conn(0x7fb3d006d710 0x7fb3d006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3d4833640 1 --2- 172.21.15.94:0/637230550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3d006e7a0 0x7fb3d006ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2347958963 0 0) 0x7fb3c0001120 con 0x7fb3d015c7e0 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3b8003530 con 0x7fb3d015c7e0 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1471286303 0 0) 0x7fb3b8003530 con 0x7fb3d015c7e0 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3d006e7a0 msgr2=0x7fb3d006ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 --2- 172.21.15.94:0/637230550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3d006e7a0 0x7fb3d006ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 >> v2:172.21.15.138:3300/0 conn(0x7fb3d006d710 msgr2=0x7fb3d006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 --2- 172.21.15.94:0/637230550 >> v2:172.21.15.138:3300/0 conn(0x7fb3d006d710 0x7fb3d006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d0072ac0 con 0x7fb3d015c7e0 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fb3c0001850 con 0x7fb3d015c7e0 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 >> v1:172.21.15.94:6789/0 conn(0x7fb3d015c7e0 legacy=0x7fb3d00afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 --2- 172.21.15.94:0/637230550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b80039c0 0x7fb3b8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 --2- 172.21.15.94:0/637230550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b8004810 0x7fb3b8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 --2- 172.21.15.94:0/637230550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3d006e7a0 0x7fb3b80051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d006e060 con 0x7fb3b8004810 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d00727e0 con 0x7fb3b80039c0 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b800dd40 con 0x7fb3d006e7a0 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3cf7fe640 1 --2- 172.21.15.94:0/637230550 >> v2:172.21.15.138:3300/0 conn(0x7fb3d006d710 0x7fb3d006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3cf7fe640 1 --2- 172.21.15.94:0/637230550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b8004810 0x7fb3b8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3cffff640 1 --2- 172.21.15.94:0/637230550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b80039c0 0x7fb3b8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3d4833640 1 --2- 172.21.15.94:0/637230550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3d006e7a0 0x7fb3b80051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3cf7fe640 1 -- 172.21.15.94:0/637230550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3d006e7a0 msgr2=0x7fb3b80051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3cf7fe640 1 --2- 172.21.15.94:0/637230550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3d006e7a0 0x7fb3b80051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3cf7fe640 1 -- 172.21.15.94:0/637230550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b80039c0 msgr2=0x7fb3b8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3cf7fe640 1 --2- 172.21.15.94:0/637230550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b80039c0 0x7fb3b8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.431+0000 7fb3cf7fe640 1 -- 172.21.15.94:0/637230550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b8003530 con 0x7fb3b8004810 2024-04-25T16:51:56.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.435+0000 7fb3cf7fe640 1 --2- 172.21.15.94:0/637230550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b8004810 0x7fb3b8004c20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009880 tx=0x7fb3c401c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1d3b4981cdb35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:56.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.435+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400f040 con 0x7fb3b8004810 2024-04-25T16:51:56.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.435+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c4004220 con 0x7fb3b8004810 2024-04-25T16:51:56.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.435+0000 7fb3ceffd640 1 -- 172.21.15.94:0/637230550 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4003620 con 0x7fb3b8004810 2024-04-25T16:51:56.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.435+0000 7fb3d6296640 1 -- 172.21.15.94:0/637230550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b8004810 msgr2=0x7fb3b8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.435+0000 7fb3d6296640 1 --2- 172.21.15.94:0/637230550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b8004810 0x7fb3b8004c20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009880 tx=0x7fb3c401c5c0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.435+0000 7fb3d4833640 1 -- 172.21.15.94:0/637230550 reap_dead start 2024-04-25T16:51:56.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.435+0000 7fb3d6296640 1 -- 172.21.15.94:0/637230550 shutdown_connections 2024-04-25T16:51:56.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.435+0000 7fb3d6296640 1 -- 172.21.15.94:0/637230550 >> 172.21.15.94:0/637230550 conn(0x7fb3d00b47f0 msgr2=0x7fb3d00b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:56.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.435+0000 7fb3d6296640 1 -- 172.21.15.94:0/637230550 shutdown_connections 2024-04-25T16:51:56.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.435+0000 7fb3d6296640 1 -- 172.21.15.94:0/637230550 wait complete. 2024-04-25T16:51:56.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.435+0000 7fb3d6296640 1 Processor -- start 2024-04-25T16:51:56.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.435+0000 7fb3d6296640 1 -- start start 2024-04-25T16:51:56.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3d6296640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b8004810 0x7fb3d015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3d6296640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3d0156cb0 0x7fb3d009e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3d6296640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d009ee10 0x7fb3d0161df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3d6296640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d015cbf0 con 0x7fb3b8004810 2024-04-25T16:51:56.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3d6296640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d0073ca0 con 0x7fb3d009ee10 2024-04-25T16:51:56.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3d6296640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d00b0500 con 0x7fb3d0156cb0 2024-04-25T16:51:56.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b8004810 0x7fb3d015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b8004810 0x7fb3d015a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59926/0 (socket says 172.21.15.94:59926) 2024-04-25T16:51:56.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3cffff640 1 -- 172.21.15.94:0/3294363587 learned_addr learned my addr 172.21.15.94:0/3294363587 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:56.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3cf7fe640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3d0156cb0 0x7fb3d009e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3d4833640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d009ee10 0x7fb3d0161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3cffff640 1 -- 172.21.15.94:0/3294363587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3d0156cb0 msgr2=0x7fb3d009e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3cffff640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3d0156cb0 0x7fb3d009e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3cffff640 1 -- 172.21.15.94:0/3294363587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d009ee10 msgr2=0x7fb3d0161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3cffff640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d009ee10 0x7fb3d0161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3cffff640 1 -- 172.21.15.94:0/3294363587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d015ac00 con 0x7fb3b8004810 2024-04-25T16:51:56.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3d4833640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d009ee10 0x7fb3d0161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:56.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3cffff640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b8004810 0x7fb3d015a3d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0000fc0 tx=0x7fb3c000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:56.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3ccff9640 1 -- 172.21.15.94:0/3294363587 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c000c8a0 con 0x7fb3b8004810 2024-04-25T16:51:56.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3ccff9640 1 -- 172.21.15.94:0/3294363587 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c0018070 con 0x7fb3b8004810 2024-04-25T16:51:56.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3d6296640 1 -- 172.21.15.94:0/3294363587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d015aee0 con 0x7fb3b8004810 2024-04-25T16:51:56.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3ccff9640 1 -- 172.21.15.94:0/3294363587 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c000cca0 con 0x7fb3b8004810 2024-04-25T16:51:56.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3d6296640 1 -- 172.21.15.94:0/3294363587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d015b4a0 con 0x7fb3b8004810 2024-04-25T16:51:56.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3d6296640 1 -- 172.21.15.94:0/3294363587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d0076830 con 0x7fb3b8004810 2024-04-25T16:51:56.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.439+0000 7fb3ccff9640 1 -- 172.21.15.94:0/3294363587 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb3c001d460 con 0x7fb3b8004810 2024-04-25T16:51:56.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.443+0000 7fb3ccff9640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb3bc06f400 0x7fb3bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.443+0000 7fb3cf7fe640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb3bc06f400 0x7fb3bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.443+0000 7fb3ccff9640 1 -- 172.21.15.94:0/3294363587 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb3c008f600 con 0x7fb3b8004810 2024-04-25T16:51:56.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.443+0000 7fb3cf7fe640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb3bc06f400 0x7fb3bc071910 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40ae000 tx=0x7fb3c40b2040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:56.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.447+0000 7fb3ccff9640 1 -- 172.21.15.94:0/3294363587 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb3d0074380 con 0x7fb3b8004810 2024-04-25T16:51:56.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.595+0000 7fb3d6296640 1 -- 172.21.15.94:0/3294363587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb3d01601d0 con 0x7fb3b8004810 2024-04-25T16:51:56.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.595+0000 7fb3ccff9640 1 -- 172.21.15.94:0/3294363587 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fb3c005c0f0 con 0x7fb3b8004810 2024-04-25T16:51:56.595 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:56.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.599+0000 7fb3d6296640 1 -- 172.21.15.94:0/3294363587 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb3bc06f400 msgr2=0x7fb3bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.599+0000 7fb3d6296640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb3bc06f400 0x7fb3bc071910 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40ae000 tx=0x7fb3c40b2040 comp rx=0 tx=0).stop 2024-04-25T16:51:56.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.599+0000 7fb3d6296640 1 -- 172.21.15.94:0/3294363587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b8004810 msgr2=0x7fb3d015a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.599+0000 7fb3d6296640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b8004810 0x7fb3d015a3d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0000fc0 tx=0x7fb3c000ec30 comp rx=0 tx=0).stop 2024-04-25T16:51:56.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.599+0000 7fb3d6296640 1 -- 172.21.15.94:0/3294363587 shutdown_connections 2024-04-25T16:51:56.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.599+0000 7fb3d6296640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d009ee10 0x7fb3d0161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.599+0000 7fb3d6296640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb3bc06f400 0x7fb3bc071910 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.599+0000 7fb3d6296640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3d0156cb0 0x7fb3d009e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.599+0000 7fb3d6296640 1 --2- 172.21.15.94:0/3294363587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3b8004810 0x7fb3d015a3d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.599+0000 7fb3d6296640 1 -- 172.21.15.94:0/3294363587 >> 172.21.15.94:0/3294363587 conn(0x7fb3d00b47f0 msgr2=0x7fb3d00b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:56.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.599+0000 7fb3d6296640 1 -- 172.21.15.94:0/3294363587 shutdown_connections 2024-04-25T16:51:56.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.599+0000 7fb3d6296640 1 -- 172.21.15.94:0/3294363587 wait complete. 2024-04-25T16:51:56.615 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:51:56.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4fd11640 1 Processor -- start 2024-04-25T16:51:56.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4fd11640 1 -- start start 2024-04-25T16:51:56.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4fd11640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efd480ce8c0 0x7efd480cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4fd11640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd48153d10 con 0x7efd480cfe80 2024-04-25T16:51:56.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4fd11640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd480ccb10 con 0x7efd48153900 2024-04-25T16:51:56.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4fd11640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efd4815d3f0 con 0x7efd480ce8c0 2024-04-25T16:51:56.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4daad640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7efd480cfe80 0x7efd4806e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49588/0 (socket says 172.21.15.94:49588) 2024-04-25T16:51:56.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4daad640 1 -- 172.21.15.94:0/199457531 learned_addr learned my addr 172.21.15.94:0/199457531 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:56.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3386851980 0 0) 0x7efd48153d10 con 0x7efd480cfe80 2024-04-25T16:51:56.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd34005740 con 0x7efd480cfe80 2024-04-25T16:51:56.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4e2ae640 1 --2- 172.21.15.94:0/199457531 >> v2:172.21.15.138:3300/0 conn(0x7efd480ce8c0 0x7efd480cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 303878383 0 0) 0x7efd34005740 con 0x7efd480cfe80 2024-04-25T16:51:56.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 >> v2:172.21.15.138:3300/0 conn(0x7efd480ce8c0 msgr2=0x7efd480cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:56.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 --2- 172.21.15.94:0/199457531 >> v2:172.21.15.138:3300/0 conn(0x7efd480ce8c0 0x7efd480cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 >> v1:172.21.15.138:6789/0 conn(0x7efd48153900 legacy=0x7efd480ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4815d6d0 con 0x7efd480cfe80 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7efd440016a0 con 0x7efd480cfe80 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4e2ae640 1 --2- 172.21.15.94:0/199457531 >> v2:172.21.15.138:3300/0 conn(0x7efd480ce8c0 0x7efd480cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 >> v1:172.21.15.94:6789/0 conn(0x7efd480cfe80 legacy=0x7efd4806e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 --2- 172.21.15.94:0/199457531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd34005bd0 0x7efd34006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 --2- 172.21.15.94:0/199457531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd34006a20 0x7efd34006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 --2- 172.21.15.94:0/199457531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd34007490 0x7efd3400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd4815d3f0 con 0x7efd34005bd0 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd3400e420 con 0x7efd34006a20 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efd3400e620 con 0x7efd34007490 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4daad640 1 --2- 172.21.15.94:0/199457531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd34005bd0 0x7efd34006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4d2ac640 1 --2- 172.21.15.94:0/199457531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd34006a20 0x7efd34006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4daad640 1 -- 172.21.15.94:0/199457531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd34007490 msgr2=0x7efd3400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4e2ae640 1 --2- 172.21.15.94:0/199457531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd34007490 0x7efd3400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4daad640 1 --2- 172.21.15.94:0/199457531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd34007490 0x7efd3400dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4daad640 1 -- 172.21.15.94:0/199457531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd34006a20 msgr2=0x7efd34006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4daad640 1 --2- 172.21.15.94:0/199457531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd34006a20 0x7efd34006e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4e2ae640 1 -- 172.21.15.94:0/199457531 reap_dead start 2024-04-25T16:51:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.755+0000 7efd4daad640 1 -- 172.21.15.94:0/199457531 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd34005740 con 0x7efd34005bd0 2024-04-25T16:51:56.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.759+0000 7efd4daad640 1 --2- 172.21.15.94:0/199457531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd34005bd0 0x7efd34006040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efd4400c210 tx=0x7efd4401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e21f280b4e88d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:56.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.759+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44007d40 con 0x7efd34005bd0 2024-04-25T16:51:56.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.759+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd440a9070 con 0x7efd34005bd0 2024-04-25T16:51:56.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.759+0000 7efd4caab640 1 -- 172.21.15.94:0/199457531 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44005d40 con 0x7efd34005bd0 2024-04-25T16:51:56.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.759+0000 7efd4fd11640 1 -- 172.21.15.94:0/199457531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd34005bd0 msgr2=0x7efd34006040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.759+0000 7efd4fd11640 1 --2- 172.21.15.94:0/199457531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd34005bd0 0x7efd34006040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efd4400c210 tx=0x7efd4401cc00 comp rx=0 tx=0).stop 2024-04-25T16:51:56.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.759+0000 7efd4fd11640 1 -- 172.21.15.94:0/199457531 shutdown_connections 2024-04-25T16:51:56.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.759+0000 7efd4fd11640 1 --2- 172.21.15.94:0/199457531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd34005bd0 0x7efd34006040 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.759+0000 7efd4fd11640 1 -- 172.21.15.94:0/199457531 >> 172.21.15.94:0/199457531 conn(0x7efd480ced30 msgr2=0x7efd480cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:56.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.759+0000 7efd4fd11640 1 -- 172.21.15.94:0/199457531 shutdown_connections 2024-04-25T16:51:56.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.759+0000 7efd4fd11640 1 -- 172.21.15.94:0/199457531 wait complete. 2024-04-25T16:51:56.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4fd11640 1 Processor -- start 2024-04-25T16:51:56.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4fd11640 1 -- start start 2024-04-25T16:51:56.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4fd11640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd34005bd0 0x7efd4815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4fd11640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd4813fd60 0x7efd4813f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4fd11640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd4806ce70 0x7efd4806d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4fd11640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd4806e190 con 0x7efd4806ce70 2024-04-25T16:51:56.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4fd11640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd4806e010 con 0x7efd34005bd0 2024-04-25T16:51:56.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4fd11640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efd4806e310 con 0x7efd4813fd60 2024-04-25T16:51:56.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4e2ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd4806ce70 0x7efd4806d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4e2ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd4806ce70 0x7efd4806d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59950/0 (socket says 172.21.15.94:59950) 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4e2ae640 1 -- 172.21.15.94:0/375880365 learned_addr learned my addr 172.21.15.94:0/375880365 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4daad640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd34005bd0 0x7efd4815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4d2ac640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd4813fd60 0x7efd4813f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4e2ae640 1 -- 172.21.15.94:0/375880365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd4813fd60 msgr2=0x7efd4813f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4e2ae640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd4813fd60 0x7efd4813f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4e2ae640 1 -- 172.21.15.94:0/375880365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd34005bd0 msgr2=0x7efd4815c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4e2ae640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd34005bd0 0x7efd4815c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4e2ae640 1 -- 172.21.15.94:0/375880365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4806d7c0 con 0x7efd4806ce70 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4e2ae640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd4806ce70 0x7efd4806d280 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efd480b1d40 tx=0x7efd40004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd3e7fc640 1 -- 172.21.15.94:0/375880365 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40019070 con 0x7efd4806ce70 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd3e7fc640 1 -- 172.21.15.94:0/375880365 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd40007cb0 con 0x7efd4806ce70 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd3e7fc640 1 -- 172.21.15.94:0/375880365 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40007e60 con 0x7efd4806ce70 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4fd11640 1 -- 172.21.15.94:0/375880365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd481423d0 con 0x7efd4806ce70 2024-04-25T16:51:56.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4fd11640 1 -- 172.21.15.94:0/375880365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd48142990 con 0x7efd4806ce70 2024-04-25T16:51:56.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd4fd11640 1 -- 172.21.15.94:0/375880365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd48151160 con 0x7efd4806ce70 2024-04-25T16:51:56.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.763+0000 7efd3e7fc640 1 -- 172.21.15.94:0/375880365 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efd4001c070 con 0x7efd4806ce70 2024-04-25T16:51:56.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.767+0000 7efd3e7fc640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7efd2c06f400 0x7efd2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.767+0000 7efd4daad640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7efd2c06f400 0x7efd2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.767+0000 7efd3e7fc640 1 -- 172.21.15.94:0/375880365 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7efd4008f220 con 0x7efd4806ce70 2024-04-25T16:51:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.767+0000 7efd4daad640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7efd2c06f400 0x7efd2c071910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efd440ab000 tx=0x7efd440b0040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:56.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.771+0000 7efd3e7fc640 1 -- 172.21.15.94:0/375880365 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efd4806d7c0 con 0x7efd4806ce70 2024-04-25T16:51:56.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd4fd11640 1 -- 172.21.15.94:0/375880365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd48142160 con 0x7efd4806ce70 2024-04-25T16:51:56.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd3e7fc640 1 -- 172.21.15.94:0/375880365 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7efd4005bd10 con 0x7efd4806ce70 2024-04-25T16:51:56.917 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:51:56.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd4fd11640 1 -- 172.21.15.94:0/375880365 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7efd2c06f400 msgr2=0x7efd2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd4fd11640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7efd2c06f400 0x7efd2c071910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efd440ab000 tx=0x7efd440b0040 comp rx=0 tx=0).stop 2024-04-25T16:51:56.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd4fd11640 1 -- 172.21.15.94:0/375880365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd4806ce70 msgr2=0x7efd4806d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:56.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd4fd11640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd4806ce70 0x7efd4806d280 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efd480b1d40 tx=0x7efd40004840 comp rx=0 tx=0).stop 2024-04-25T16:51:56.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd4fd11640 1 -- 172.21.15.94:0/375880365 shutdown_connections 2024-04-25T16:51:56.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd4fd11640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd4806ce70 0x7efd4806d280 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd4fd11640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7efd2c06f400 0x7efd2c071910 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd4fd11640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd4813fd60 0x7efd4813f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd4fd11640 1 --2- 172.21.15.94:0/375880365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd34005bd0 0x7efd4815c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:56.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd4fd11640 1 -- 172.21.15.94:0/375880365 >> 172.21.15.94:0/375880365 conn(0x7efd480ced30 msgr2=0x7efd4815e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:56.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd4fd11640 1 -- 172.21.15.94:0/375880365 shutdown_connections 2024-04-25T16:51:56.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:56.919+0000 7efd4fd11640 1 -- 172.21.15.94:0/375880365 wait complete. 2024-04-25T16:51:56.938 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:51:57.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f958d856640 1 Processor -- start 2024-04-25T16:51:57.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f958d856640 1 -- start start 2024-04-25T16:51:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f958d856640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9588072b60 0x7f958815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f958d856640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f958815cd60 0x7f958815fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f958d856640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f958806f360 con 0x7f9588160020 2024-04-25T16:51:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f958d856640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f95880703e0 con 0x7f9588072b60 2024-04-25T16:51:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f958d856640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f958806f620 con 0x7f958815cd60 2024-04-25T16:51:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f9586ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9588072b60 0x7f958815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f9586ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9588072b60 0x7f958815c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59954/0 (socket says 172.21.15.94:59954) 2024-04-25T16:51:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f9586ffd640 1 -- 172.21.15.94:0/3249224160 learned_addr learned my addr 172.21.15.94:0/3249224160 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f957ffff640 1 --2- 172.21.15.94:0/3249224160 >> v2:172.21.15.138:3300/0 conn(0x7f958815cd60 0x7f958815fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f9586ffd640 1 -- 172.21.15.94:0/3249224160 >> v2:172.21.15.138:3300/0 conn(0x7f958815cd60 msgr2=0x7f958815fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f9586ffd640 1 --2- 172.21.15.94:0/3249224160 >> v2:172.21.15.138:3300/0 conn(0x7f958815cd60 0x7f958815fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f9586ffd640 1 -- 172.21.15.94:0/3249224160 >> v1:172.21.15.138:6789/0 conn(0x7f9588160020 legacy=0x7f95880aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.079+0000 7f9586ffd640 1 -- 172.21.15.94:0/3249224160 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958806f900 con 0x7f9588072b60 2024-04-25T16:51:57.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f9586ffd640 1 --2- 172.21.15.94:0/3249224160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9588072b60 0x7f958815c820 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f958806e190 tx=0x7f957801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ade456a61c6e3ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:57.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f95867fc640 1 -- 172.21.15.94:0/3249224160 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95780b0070 con 0x7f9588072b60 2024-04-25T16:51:57.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f95867fc640 1 -- 172.21.15.94:0/3249224160 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95780ab600 con 0x7f9588072b60 2024-04-25T16:51:57.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f95867fc640 1 -- 172.21.15.94:0/3249224160 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95780ab9d0 con 0x7f9588072b60 2024-04-25T16:51:57.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f958d856640 1 -- 172.21.15.94:0/3249224160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9588072b60 msgr2=0x7f958815c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f958d856640 1 --2- 172.21.15.94:0/3249224160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9588072b60 0x7f958815c820 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f958806e190 tx=0x7f957801c420 comp rx=0 tx=0).stop 2024-04-25T16:51:57.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f958d856640 1 -- 172.21.15.94:0/3249224160 shutdown_connections 2024-04-25T16:51:57.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f958d856640 1 --2- 172.21.15.94:0/3249224160 >> v2:172.21.15.138:3300/0 conn(0x7f958815cd60 0x7f958815fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f958d856640 1 --2- 172.21.15.94:0/3249224160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9588072b60 0x7f958815c820 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f958d856640 1 -- 172.21.15.94:0/3249224160 >> 172.21.15.94:0/3249224160 conn(0x7f95880b47f0 msgr2=0x7f95880b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:57.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f958d856640 1 -- 172.21.15.94:0/3249224160 shutdown_connections 2024-04-25T16:51:57.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f958d856640 1 -- 172.21.15.94:0/3249224160 wait complete. 2024-04-25T16:51:57.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f958d856640 1 Processor -- start 2024-04-25T16:51:57.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.083+0000 7f958d856640 1 -- start start 2024-04-25T16:51:57.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f958d856640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9588072b60 0x7f958809b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f958d856640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958815cd60 0x7f95881455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f958d856640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9588160020 0x7f95880b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f958d856640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95880730f0 con 0x7f958815cd60 2024-04-25T16:51:57.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f958d856640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9588072f70 con 0x7f9588160020 2024-04-25T16:51:57.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f958d856640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9588159cc0 con 0x7f9588072b60 2024-04-25T16:51:57.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f957ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958815cd60 0x7f95881455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f957ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958815cd60 0x7f95881455e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59962/0 (socket says 172.21.15.94:59962) 2024-04-25T16:51:57.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f957ffff640 1 -- 172.21.15.94:0/453121281 learned_addr learned my addr 172.21.15.94:0/453121281 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:57.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f9586ffd640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9588072b60 0x7f958809b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f957ffff640 1 -- 172.21.15.94:0/453121281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9588072b60 msgr2=0x7f958809b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f95877fe640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9588160020 0x7f95880b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f957ffff640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9588072b60 0x7f958809b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f957ffff640 1 -- 172.21.15.94:0/453121281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9588160020 msgr2=0x7f95880b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f957ffff640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9588160020 0x7f95880b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f957ffff640 1 -- 172.21.15.94:0/453121281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958809b720 con 0x7f958815cd60 2024-04-25T16:51:57.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f957ffff640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958815cd60 0x7f95881455e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f957000d620 tx=0x7f957000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:57.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f9584ff9640 1 -- 172.21.15.94:0/453121281 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957000b620 con 0x7f958815cd60 2024-04-25T16:51:57.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f9584ff9640 1 -- 172.21.15.94:0/453121281 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9570019070 con 0x7f958815cd60 2024-04-25T16:51:57.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f958d856640 1 -- 172.21.15.94:0/453121281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9588145b20 con 0x7f958815cd60 2024-04-25T16:51:57.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f9584ff9640 1 -- 172.21.15.94:0/453121281 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957000b9e0 con 0x7f958815cd60 2024-04-25T16:51:57.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f958d856640 1 -- 172.21.15.94:0/453121281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9588099180 con 0x7f958815cd60 2024-04-25T16:51:57.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f958d856640 1 -- 172.21.15.94:0/453121281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9588076830 con 0x7f958815cd60 2024-04-25T16:51:57.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f9584ff9640 1 -- 172.21.15.94:0/453121281 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f957000bb80 con 0x7f958815cd60 2024-04-25T16:51:57.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f9584ff9640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f956806f430 0x7f9568071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f9586ffd640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f956806f430 0x7f9568071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.087+0000 7f9584ff9640 1 -- 172.21.15.94:0/453121281 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f957008fa00 con 0x7f958815cd60 2024-04-25T16:51:57.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.091+0000 7f9586ffd640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f956806f430 0x7f9568071940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f95780ad000 tx=0x7f9578008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:57.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.095+0000 7f9584ff9640 1 -- 172.21.15.94:0/453121281 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f957005c580 con 0x7f958815cd60 2024-04-25T16:51:57.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.247+0000 7f958d856640 1 -- 172.21.15.94:0/453121281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f958815e4c0 con 0x7f958815cd60 2024-04-25T16:51:57.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.251+0000 7f9584ff9640 1 -- 172.21.15.94:0/453121281 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f957005be80 con 0x7f958815cd60 2024-04-25T16:51:57.248 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:57.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.251+0000 7f958d856640 1 -- 172.21.15.94:0/453121281 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f956806f430 msgr2=0x7f9568071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.251+0000 7f958d856640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f956806f430 0x7f9568071940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f95780ad000 tx=0x7f9578008040 comp rx=0 tx=0).stop 2024-04-25T16:51:57.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.251+0000 7f958d856640 1 -- 172.21.15.94:0/453121281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958815cd60 msgr2=0x7f95881455e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.251+0000 7f958d856640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958815cd60 0x7f95881455e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f957000d620 tx=0x7f957000dae0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.251+0000 7f958d856640 1 -- 172.21.15.94:0/453121281 shutdown_connections 2024-04-25T16:51:57.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.251+0000 7f958d856640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9588160020 0x7f95880b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.251+0000 7f958d856640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958815cd60 0x7f95881455e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.251+0000 7f958d856640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f956806f430 0x7f9568071940 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.251+0000 7f958d856640 1 --2- 172.21.15.94:0/453121281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9588072b60 0x7f958809b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.251+0000 7f958d856640 1 -- 172.21.15.94:0/453121281 >> 172.21.15.94:0/453121281 conn(0x7f95880b47f0 msgr2=0x7f9588151ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:57.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.251+0000 7f958d856640 1 -- 172.21.15.94:0/453121281 shutdown_connections 2024-04-25T16:51:57.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.251+0000 7f958d856640 1 -- 172.21.15.94:0/453121281 wait complete. 2024-04-25T16:51:57.269 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:51:57.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d52cc6640 1 Processor -- start 2024-04-25T16:51:57.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d52cc6640 1 -- start start 2024-04-25T16:51:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d52cc6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4c06f470 0x7f1d4c15dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d52cc6640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d4c070630 con 0x7f1d4c15e160 2024-04-25T16:51:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d52cc6640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d4c0ae170 con 0x7f1d4c0b0420 2024-04-25T16:51:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d52cc6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4c0aeaa0 con 0x7f1d4c06f470 2024-04-25T16:51:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d51263640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f1d4c15e160 0x7f1d4c15e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:49594/0 (socket says 172.21.15.94:49594) 2024-04-25T16:51:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d51263640 1 -- 172.21.15.94:0/1698072247 learned_addr learned my addr 172.21.15.94:0/1698072247 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d4bfff640 1 --2- 172.21.15.94:0/1698072247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4c06f470 0x7f1d4c15dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 118280644 0 0) 0x7f1d4c070630 con 0x7f1d4c15e160 2024-04-25T16:51:57.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d34003530 con 0x7f1d4c15e160 2024-04-25T16:51:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4262503724 0 0) 0x7f1d34003530 con 0x7f1d4c15e160 2024-04-25T16:51:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4c06f470 msgr2=0x7f1d4c15dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d4b7fe640 1 --2- 172.21.15.94:0/1698072247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4c06f470 0x7f1d4c15dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 >> v1:172.21.15.138:6789/0 conn(0x7f1d4c0b0420 legacy=0x7f1d4c06ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c0ae450 con 0x7f1d4c15e160 2024-04-25T16:51:57.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.415+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f1d44001a00 con 0x7f1d4c15e160 2024-04-25T16:51:57.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 >> v1:172.21.15.94:6789/0 conn(0x7f1d4c15e160 legacy=0x7f1d4c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d4b7fe640 1 --2- 172.21.15.94:0/1698072247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d340039f0 0x7f1d34003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d4b7fe640 1 --2- 172.21.15.94:0/1698072247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d34004810 0x7f1d34004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d4b7fe640 1 --2- 172.21.15.94:0/1698072247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d34005160 0x7f1d3400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c0aeaa0 con 0x7f1d34005160 2024-04-25T16:51:57.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3400bf60 con 0x7f1d340039f0 2024-04-25T16:51:57.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3400c160 con 0x7f1d34004810 2024-04-25T16:51:57.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d51263640 1 --2- 172.21.15.94:0/1698072247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d34005160 0x7f1d3400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d4bfff640 1 --2- 172.21.15.94:0/1698072247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d34004810 0x7f1d34004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d50a62640 1 --2- 172.21.15.94:0/1698072247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d340039f0 0x7f1d34003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d51263640 1 -- 172.21.15.94:0/1698072247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d34004810 msgr2=0x7f1d34004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:57.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d51263640 1 --2- 172.21.15.94:0/1698072247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d34004810 0x7f1d34004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d51263640 1 -- 172.21.15.94:0/1698072247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d340039f0 msgr2=0x7f1d34003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:57.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d51263640 1 --2- 172.21.15.94:0/1698072247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d340039f0 0x7f1d34003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d51263640 1 -- 172.21.15.94:0/1698072247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d34003530 con 0x7f1d34005160 2024-04-25T16:51:57.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d50a62640 1 --2- 172.21.15.94:0/1698072247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d340039f0 0x7f1d34003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:57.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d51263640 1 --2- 172.21.15.94:0/1698072247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d34005160 0x7f1d3400b9d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400a760 tx=0x7f1d4401d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c05ffefabb429c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:57.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400a3b0 con 0x7f1d34005160 2024-04-25T16:51:57.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d440ad650 con 0x7f1d34005160 2024-04-25T16:51:57.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d4b7fe640 1 -- 172.21.15.94:0/1698072247 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d440a5e60 con 0x7f1d34005160 2024-04-25T16:51:57.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d52cc6640 1 -- 172.21.15.94:0/1698072247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d34005160 msgr2=0x7f1d3400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d52cc6640 1 --2- 172.21.15.94:0/1698072247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d34005160 0x7f1d3400b9d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400a760 tx=0x7f1d4401d280 comp rx=0 tx=0).stop 2024-04-25T16:51:57.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.419+0000 7f1d51263640 1 -- 172.21.15.94:0/1698072247 reap_dead start 2024-04-25T16:51:57.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d52cc6640 1 -- 172.21.15.94:0/1698072247 shutdown_connections 2024-04-25T16:51:57.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d52cc6640 1 -- 172.21.15.94:0/1698072247 >> 172.21.15.94:0/1698072247 conn(0x7f1d4c0b47f0 msgr2=0x7f1d4c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:57.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d52cc6640 1 -- 172.21.15.94:0/1698072247 shutdown_connections 2024-04-25T16:51:57.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d52cc6640 1 -- 172.21.15.94:0/1698072247 wait complete. 2024-04-25T16:51:57.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d52cc6640 1 Processor -- start 2024-04-25T16:51:57.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d52cc6640 1 -- start start 2024-04-25T16:51:57.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d52cc6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4c147b90 0x7f1d4c144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d52cc6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d4c0cfbc0 0x7f1d4c09a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d52cc6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4c1593b0 0x7f1d4c1597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d52cc6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c0aef30 con 0x7f1d4c147b90 2024-04-25T16:51:57.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d52cc6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c0703e0 con 0x7f1d4c0cfbc0 2024-04-25T16:51:57.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d52cc6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4c0b0830 con 0x7f1d4c1593b0 2024-04-25T16:51:57.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d50a62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4c147b90 0x7f1d4c144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d50a62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4c147b90 0x7f1d4c144fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59980/0 (socket says 172.21.15.94:59980) 2024-04-25T16:51:57.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d50a62640 1 -- 172.21.15.94:0/3803724222 learned_addr learned my addr 172.21.15.94:0/3803724222 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:57.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d50a62640 1 -- 172.21.15.94:0/3803724222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4c1593b0 msgr2=0x7f1d4c1597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d50a62640 1 --2- 172.21.15.94:0/3803724222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4c1593b0 0x7f1d4c1597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d50a62640 1 -- 172.21.15.94:0/3803724222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d4c0cfbc0 msgr2=0x7f1d4c09a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d4bfff640 1 --2- 172.21.15.94:0/3803724222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d4c0cfbc0 0x7f1d4c09a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d50a62640 1 --2- 172.21.15.94:0/3803724222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d4c0cfbc0 0x7f1d4c09a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d50a62640 1 -- 172.21.15.94:0/3803724222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c159d00 con 0x7f1d4c147b90 2024-04-25T16:51:57.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.423+0000 7f1d50a62640 1 --2- 172.21.15.94:0/3803724222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4c147b90 0x7f1d4c144fe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00e530 tx=0x7f1d3c00e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:57.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.427+0000 7f1d497fa640 1 -- 172.21.15.94:0/3803724222 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0052e0 con 0x7f1d4c147b90 2024-04-25T16:51:57.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.427+0000 7f1d497fa640 1 -- 172.21.15.94:0/3803724222 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d3c018070 con 0x7f1d4c147b90 2024-04-25T16:51:57.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.427+0000 7f1d52cc6640 1 -- 172.21.15.94:0/3803724222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4c09aef0 con 0x7f1d4c147b90 2024-04-25T16:51:57.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.427+0000 7f1d497fa640 1 -- 172.21.15.94:0/3803724222 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c005480 con 0x7f1d4c147b90 2024-04-25T16:51:57.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.427+0000 7f1d52cc6640 1 -- 172.21.15.94:0/3803724222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4c151700 con 0x7f1d4c147b90 2024-04-25T16:51:57.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.427+0000 7f1d52cc6640 1 -- 172.21.15.94:0/3803724222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4c076830 con 0x7f1d4c147b90 2024-04-25T16:51:57.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.427+0000 7f1d497fa640 1 -- 172.21.15.94:0/3803724222 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1d3c003820 con 0x7f1d4c147b90 2024-04-25T16:51:57.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.427+0000 7f1d497fa640 1 --2- 172.21.15.94:0/3803724222 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d3806f430 0x7f1d38071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.427+0000 7f1d4bfff640 1 --2- 172.21.15.94:0/3803724222 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d3806f430 0x7f1d38071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.427+0000 7f1d497fa640 1 -- 172.21.15.94:0/3803724222 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1d3c08fd90 con 0x7f1d4c147b90 2024-04-25T16:51:57.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.427+0000 7f1d4bfff640 1 --2- 172.21.15.94:0/3803724222 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d3806f430 0x7f1d38071940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c148190 tx=0x7f1d40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:57.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.431+0000 7f1d497fa640 1 -- 172.21.15.94:0/3803724222 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1d4c159d00 con 0x7f1d4c147b90 2024-04-25T16:51:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.591+0000 7f1d52cc6640 1 -- 172.21.15.94:0/3803724222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1d4c06fb20 con 0x7f1d4c147b90 2024-04-25T16:51:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.591+0000 7f1d497fa640 1 -- 172.21.15.94:0/3803724222 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f1d3c05c9a0 con 0x7f1d4c147b90 2024-04-25T16:51:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:51:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.595+0000 7f1d52cc6640 1 -- 172.21.15.94:0/3803724222 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d3806f430 msgr2=0x7f1d38071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.595+0000 7f1d52cc6640 1 --2- 172.21.15.94:0/3803724222 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d3806f430 0x7f1d38071940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c148190 tx=0x7f1d40008040 comp rx=0 tx=0).stop 2024-04-25T16:51:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.595+0000 7f1d52cc6640 1 -- 172.21.15.94:0/3803724222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4c147b90 msgr2=0x7f1d4c144fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.595+0000 7f1d52cc6640 1 --2- 172.21.15.94:0/3803724222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4c147b90 0x7f1d4c144fe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00e530 tx=0x7f1d3c00e9f0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.595+0000 7f1d52cc6640 1 -- 172.21.15.94:0/3803724222 shutdown_connections 2024-04-25T16:51:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.595+0000 7f1d52cc6640 1 --2- 172.21.15.94:0/3803724222 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d3806f430 0x7f1d38071940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.595+0000 7f1d52cc6640 1 --2- 172.21.15.94:0/3803724222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d4c1593b0 0x7f1d4c1597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.595+0000 7f1d52cc6640 1 --2- 172.21.15.94:0/3803724222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d4c0cfbc0 0x7f1d4c09a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.595+0000 7f1d52cc6640 1 --2- 172.21.15.94:0/3803724222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4c147b90 0x7f1d4c144fe0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.595+0000 7f1d52cc6640 1 -- 172.21.15.94:0/3803724222 >> 172.21.15.94:0/3803724222 conn(0x7f1d4c0b47f0 msgr2=0x7f1d4c15a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.595+0000 7f1d52cc6640 1 -- 172.21.15.94:0/3803724222 shutdown_connections 2024-04-25T16:51:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.595+0000 7f1d52cc6640 1 -- 172.21.15.94:0/3803724222 wait complete. 2024-04-25T16:51:57.610 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-04-25T16:51:57.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:57.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T16:51:57.622 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:51:57.622 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:51:57.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:51:57.622 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T16:51:57.668 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:51:57.668 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:51:57.675 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:51:57.675 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:51:57.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:57.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f8ec8640 1 Processor -- start 2024-04-25T16:51:57.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f8ec8640 1 -- start start 2024-04-25T16:51:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f8ec8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92f4070080 0x7f92f415e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f8ec8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f92f415ebd0 0x7f92f406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f8ec8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f406fc60 con 0x7f92f415e7c0 2024-04-25T16:51:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f8ec8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f92f4158550 con 0x7f92f415ebd0 2024-04-25T16:51:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f8ec8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f92f406f7d0 con 0x7f92f4070080 2024-04-25T16:51:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f2d9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f92f415ebd0 0x7f92f406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f2d9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f92f415ebd0 0x7f92f406cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59984/0 (socket says 172.21.15.94:59984) 2024-04-25T16:51:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f2d9d640 1 -- 172.21.15.94:0/2487714784 learned_addr learned my addr 172.21.15.94:0/2487714784 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f259c640 1 --2- 172.21.15.94:0/2487714784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92f4070080 0x7f92f415e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f2d9d640 1 -- 172.21.15.94:0/2487714784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92f4070080 msgr2=0x7f92f415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f2d9d640 1 --2- 172.21.15.94:0/2487714784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92f4070080 0x7f92f415e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f2d9d640 1 -- 172.21.15.94:0/2487714784 >> v1:172.21.15.138:6789/0 conn(0x7f92f415e7c0 legacy=0x7f92f406c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f2d9d640 1 -- 172.21.15.94:0/2487714784 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f40aef70 con 0x7f92f415ebd0 2024-04-25T16:51:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.875+0000 7f92f259c640 1 --2- 172.21.15.94:0/2487714784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92f4070080 0x7f92f415e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:57.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f2d9d640 1 --2- 172.21.15.94:0/2487714784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92f415ebd0 0x7f92f406cd10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009e50 tx=0x7f92ec01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6babde101451b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:57.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f159a640 1 -- 172.21.15.94:0/2487714784 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec0a57e0 con 0x7f92f415ebd0 2024-04-25T16:51:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f159a640 1 -- 172.21.15.94:0/2487714784 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ec0a5980 con 0x7f92f415ebd0 2024-04-25T16:51:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f159a640 1 -- 172.21.15.94:0/2487714784 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec0a5d50 con 0x7f92f415ebd0 2024-04-25T16:51:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2487714784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92f415ebd0 msgr2=0x7f92f406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f8ec8640 1 --2- 172.21.15.94:0/2487714784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92f415ebd0 0x7f92f406cd10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009e50 tx=0x7f92ec01ca00 comp rx=0 tx=0).stop 2024-04-25T16:51:57.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2487714784 shutdown_connections 2024-04-25T16:51:57.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f8ec8640 1 --2- 172.21.15.94:0/2487714784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92f415ebd0 0x7f92f406cd10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f8ec8640 1 --2- 172.21.15.94:0/2487714784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92f4070080 0x7f92f415e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2487714784 >> 172.21.15.94:0/2487714784 conn(0x7f92f40b47f0 msgr2=0x7f92f40b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:57.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2487714784 shutdown_connections 2024-04-25T16:51:57.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2487714784 wait complete. 2024-04-25T16:51:57.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f8ec8640 1 Processor -- start 2024-04-25T16:51:57.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.879+0000 7f92f8ec8640 1 -- start start 2024-04-25T16:51:57.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f8ec8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92f4070080 0x7f92f4150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f8ec8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92f415e7c0 0x7f92f409a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f8ec8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92f415ebd0 0x7f92f409aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f8ec8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4070490 con 0x7f92f415e7c0 2024-04-25T16:51:57.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f8ec8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92f40ae6c0 con 0x7f92f415ebd0 2024-04-25T16:51:57.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f8ec8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f92f416ae90 con 0x7f92f4070080 2024-04-25T16:51:57.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92f415e7c0 0x7f92f409a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92f415e7c0 0x7f92f409a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59998/0 (socket says 172.21.15.94:59998) 2024-04-25T16:51:57.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f1d9b640 1 -- 172.21.15.94:0/2968803472 learned_addr learned my addr 172.21.15.94:0/2968803472 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f2d9d640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92f415ebd0 0x7f92f409aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f259c640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92f4070080 0x7f92f4150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f1d9b640 1 -- 172.21.15.94:0/2968803472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92f4070080 msgr2=0x7f92f4150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f1d9b640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92f4070080 0x7f92f4150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f1d9b640 1 -- 172.21.15.94:0/2968803472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92f415ebd0 msgr2=0x7f92f409aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f1d9b640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92f415ebd0 0x7f92f409aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f1d9b640 1 -- 172.21.15.94:0/2968803472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f41f7480 con 0x7f92f415e7c0 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f259c640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92f4070080 0x7f92f4150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f1d9b640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92f415e7c0 0x7f92f409a570 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00c3a0 tx=0x7f92dc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92eaffd640 1 -- 172.21.15.94:0/2968803472 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0052d0 con 0x7f92f415e7c0 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92eaffd640 1 -- 172.21.15.94:0/2968803472 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92dc005470 con 0x7f92f415e7c0 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2968803472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f41f7760 con 0x7f92f415e7c0 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92eaffd640 1 -- 172.21.15.94:0/2968803472 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc007260 con 0x7f92f415e7c0 2024-04-25T16:51:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2968803472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f41f7d20 con 0x7f92f415e7c0 2024-04-25T16:51:57.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2968803472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f4076830 con 0x7f92f415e7c0 2024-04-25T16:51:57.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.883+0000 7f92eaffd640 1 -- 172.21.15.94:0/2968803472 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f92dc007400 con 0x7f92f415e7c0 2024-04-25T16:51:57.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.887+0000 7f92eaffd640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f92d406f430 0x7f92d4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:57.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.887+0000 7f92eaffd640 1 -- 172.21.15.94:0/2968803472 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f92dc08ea70 con 0x7f92f415e7c0 2024-04-25T16:51:57.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.887+0000 7f92f259c640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f92d406f430 0x7f92d4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:57.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.887+0000 7f92f259c640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f92d406f430 0x7f92d4071940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f92f40b50c0 tx=0x7f92e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:57.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:57.891+0000 7f92eaffd640 1 -- 172.21.15.94:0/2968803472 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92dc05b680 con 0x7f92f415e7c0 2024-04-25T16:51:58.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.055+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2968803472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92f40b0fe0 con 0x7f92f415e7c0 2024-04-25T16:51:58.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.059+0000 7f92eaffd640 1 -- 172.21.15.94:0/2968803472 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7f92dc05af80 con 0x7f92f415e7c0 2024-04-25T16:51:58.056 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:51:58.056 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:51:58.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.059+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2968803472 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f92d406f430 msgr2=0x7f92d4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.059+0000 7f92f8ec8640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f92d406f430 0x7f92d4071940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f92f40b50c0 tx=0x7f92e4008040 comp rx=0 tx=0).stop 2024-04-25T16:51:58.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.059+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2968803472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92f415e7c0 msgr2=0x7f92f409a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.059+0000 7f92f8ec8640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92f415e7c0 0x7f92f409a570 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00c3a0 tx=0x7f92dc00c860 comp rx=0 tx=0).stop 2024-04-25T16:51:58.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.059+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2968803472 shutdown_connections 2024-04-25T16:51:58.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.059+0000 7f92f8ec8640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92f415ebd0 0x7f92f409aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.059+0000 7f92f8ec8640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92f415e7c0 0x7f92f409a570 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.059+0000 7f92f8ec8640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f92d406f430 0x7f92d4071940 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.059+0000 7f92f8ec8640 1 --2- 172.21.15.94:0/2968803472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92f4070080 0x7f92f4150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.059+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2968803472 >> 172.21.15.94:0/2968803472 conn(0x7f92f40b47f0 msgr2=0x7f92f4068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:58.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.059+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2968803472 shutdown_connections 2024-04-25T16:51:58.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.059+0000 7f92f8ec8640 1 -- 172.21.15.94:0/2968803472 wait complete. 2024-04-25T16:51:58.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:51:58.078 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:51:58.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f77a4a68640 1 Processor -- start 2024-04-25T16:51:58.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f77a4a68640 1 -- start start 2024-04-25T16:51:58.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f77a4a68640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f77a006c600 0x7f77a006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f77a4a68640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f77a006ed70 0x7f77a006f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f77a4a68640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77a006d330 con 0x7f77a0072f30 2024-04-25T16:51:58.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f77a4a68640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f77a006d610 con 0x7f77a006ed70 2024-04-25T16:51:58.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f77a4a68640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f77a006d8f0 con 0x7f77a006c600 2024-04-25T16:51:58.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f779ed9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f77a006ed70 0x7f77a006f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f779e59c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f77a0072f30 0x7f77a0073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38748/0 (socket says 172.21.15.94:38748) 2024-04-25T16:51:58.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f779e59c640 1 -- 172.21.15.94:0/2246954300 learned_addr learned my addr 172.21.15.94:0/2246954300 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:58.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f779dd9b640 1 --2- 172.21.15.94:0/2246954300 >> v2:172.21.15.138:3300/0 conn(0x7f77a006c600 0x7f77a006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f779ed9d640 1 -- 172.21.15.94:0/2246954300 >> v2:172.21.15.138:3300/0 conn(0x7f77a006c600 msgr2=0x7f77a006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:58.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f779ed9d640 1 --2- 172.21.15.94:0/2246954300 >> v2:172.21.15.138:3300/0 conn(0x7f77a006c600 0x7f77a006ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f779ed9d640 1 -- 172.21.15.94:0/2246954300 >> v1:172.21.15.94:6789/0 conn(0x7f77a0072f30 legacy=0x7f77a0073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f779ed9d640 1 -- 172.21.15.94:0/2246954300 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a006dbd0 con 0x7f77a006ed70 2024-04-25T16:51:58.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.231+0000 7f779dd9b640 1 --2- 172.21.15.94:0/2246954300 >> v2:172.21.15.138:3300/0 conn(0x7f77a006c600 0x7f77a006ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.235+0000 7f779ed9d640 1 --2- 172.21.15.94:0/2246954300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a006ed70 0x7f77a006f180 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f77a0070b90 tx=0x7f779801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f93ccf3429c481e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.235+0000 7f779d59a640 1 -- 172.21.15.94:0/2246954300 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77980b0070 con 0x7f77a006ed70 2024-04-25T16:51:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.235+0000 7f779d59a640 1 -- 172.21.15.94:0/2246954300 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77980ab600 con 0x7f77a006ed70 2024-04-25T16:51:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.235+0000 7f779d59a640 1 -- 172.21.15.94:0/2246954300 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77980ab9d0 con 0x7f77a006ed70 2024-04-25T16:51:58.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.235+0000 7f77a4a68640 1 -- 172.21.15.94:0/2246954300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a006ed70 msgr2=0x7f77a006f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.235+0000 7f77a4a68640 1 --2- 172.21.15.94:0/2246954300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a006ed70 0x7f77a006f180 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f77a0070b90 tx=0x7f779801c480 comp rx=0 tx=0).stop 2024-04-25T16:51:58.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.235+0000 7f77a4a68640 1 -- 172.21.15.94:0/2246954300 shutdown_connections 2024-04-25T16:51:58.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.235+0000 7f77a4a68640 1 --2- 172.21.15.94:0/2246954300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a006ed70 0x7f77a006f180 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.235+0000 7f77a4a68640 1 --2- 172.21.15.94:0/2246954300 >> v2:172.21.15.138:3300/0 conn(0x7f77a006c600 0x7f77a006ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.235+0000 7f77a4a68640 1 -- 172.21.15.94:0/2246954300 >> 172.21.15.94:0/2246954300 conn(0x7f77a00b47f0 msgr2=0x7f77a00b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:58.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.235+0000 7f77a4a68640 1 -- 172.21.15.94:0/2246954300 shutdown_connections 2024-04-25T16:51:58.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.235+0000 7f77a4a68640 1 -- 172.21.15.94:0/2246954300 wait complete. 2024-04-25T16:51:58.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f77a4a68640 1 Processor -- start 2024-04-25T16:51:58.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f77a4a68640 1 -- start start 2024-04-25T16:51:58.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f77a4a68640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77a006c600 0x7f77a0161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f77a4a68640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77a006ed70 0x7f77a01463e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f77a4a68640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a0072f30 0x7f77a0165d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f77a4a68640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77a0171a30 con 0x7f77a0072f30 2024-04-25T16:51:58.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f77a4a68640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77a01718b0 con 0x7f77a006c600 2024-04-25T16:51:58.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f77a4a68640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f77a0171bb0 con 0x7f77a006ed70 2024-04-25T16:51:58.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f779ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a0072f30 0x7f77a0165d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f779ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a0072f30 0x7f77a0165d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54564/0 (socket says 172.21.15.94:54564) 2024-04-25T16:51:58.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f779ed9d640 1 -- 172.21.15.94:0/639530734 learned_addr learned my addr 172.21.15.94:0/639530734 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:58.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f779e59c640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77a006c600 0x7f77a0161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f779dd9b640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77a006ed70 0x7f77a01463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f779ed9d640 1 -- 172.21.15.94:0/639530734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77a006ed70 msgr2=0x7f77a01463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f779ed9d640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77a006ed70 0x7f77a01463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f779ed9d640 1 -- 172.21.15.94:0/639530734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77a006c600 msgr2=0x7f77a0161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f779ed9d640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77a006c600 0x7f77a0161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f779ed9d640 1 -- 172.21.15.94:0/639530734 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a01674d0 con 0x7f77a0072f30 2024-04-25T16:51:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f779ed9d640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a0072f30 0x7f77a0165d80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f77980ae040 tx=0x7f7798019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f7796ffd640 1 -- 172.21.15.94:0/639530734 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77980b0040 con 0x7f77a0072f30 2024-04-25T16:51:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f7796ffd640 1 -- 172.21.15.94:0/639530734 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77980ba070 con 0x7f77a0072f30 2024-04-25T16:51:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f77a4a68640 1 -- 172.21.15.94:0/639530734 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a015c860 con 0x7f77a0072f30 2024-04-25T16:51:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f7796ffd640 1 -- 172.21.15.94:0/639530734 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77980b4910 con 0x7f77a0072f30 2024-04-25T16:51:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f77a4a68640 1 -- 172.21.15.94:0/639530734 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a015ce20 con 0x7f77a0072f30 2024-04-25T16:51:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.239+0000 7f77a4a68640 1 -- 172.21.15.94:0/639530734 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a0167220 con 0x7f77a0072f30 2024-04-25T16:51:58.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.243+0000 7f7796ffd640 1 -- 172.21.15.94:0/639530734 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f77980b4ab0 con 0x7f77a0072f30 2024-04-25T16:51:58.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.243+0000 7f7796ffd640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f778006f400 0x7f7780071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.243+0000 7f7796ffd640 1 -- 172.21.15.94:0/639530734 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7798129390 con 0x7f77a0072f30 2024-04-25T16:51:58.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.243+0000 7f779e59c640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f778006f400 0x7f7780071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.243+0000 7f779e59c640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f778006f400 0x7f7780071910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f77900012f0 tx=0x7f7790009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:58.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.247+0000 7f7796ffd640 1 -- 172.21.15.94:0/639530734 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7798016070 con 0x7f77a0072f30 2024-04-25T16:51:58.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.407+0000 7f77a4a68640 1 -- 172.21.15.94:0/639530734 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f77a006e8e0 con 0x7f77a0072f30 2024-04-25T16:51:58.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.407+0000 7f7796ffd640 1 -- 172.21.15.94:0/639530734 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f77980f5f10 con 0x7f77a0072f30 2024-04-25T16:51:58.407 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:51:58.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.411+0000 7f77a4a68640 1 -- 172.21.15.94:0/639530734 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f778006f400 msgr2=0x7f7780071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.411+0000 7f77a4a68640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f778006f400 0x7f7780071910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f77900012f0 tx=0x7f7790009040 comp rx=0 tx=0).stop 2024-04-25T16:51:58.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.411+0000 7f77a4a68640 1 -- 172.21.15.94:0/639530734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a0072f30 msgr2=0x7f77a0165d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.411+0000 7f77a4a68640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a0072f30 0x7f77a0165d80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f77980ae040 tx=0x7f7798019220 comp rx=0 tx=0).stop 2024-04-25T16:51:58.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.411+0000 7f77a4a68640 1 -- 172.21.15.94:0/639530734 shutdown_connections 2024-04-25T16:51:58.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.411+0000 7f77a4a68640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a0072f30 0x7f77a0165d80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.411+0000 7f77a4a68640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f778006f400 0x7f7780071910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.411+0000 7f77a4a68640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77a006ed70 0x7f77a01463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.411+0000 7f77a4a68640 1 --2- 172.21.15.94:0/639530734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77a006c600 0x7f77a0161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.411+0000 7f77a4a68640 1 -- 172.21.15.94:0/639530734 >> 172.21.15.94:0/639530734 conn(0x7f77a00b47f0 msgr2=0x7f77a00b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:58.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.411+0000 7f77a4a68640 1 -- 172.21.15.94:0/639530734 shutdown_connections 2024-04-25T16:51:58.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.411+0000 7f77a4a68640 1 -- 172.21.15.94:0/639530734 wait complete. 2024-04-25T16:51:58.429 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:51:58.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc531c3640 1 Processor -- start 2024-04-25T16:51:58.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc531c3640 1 -- start start 2024-04-25T16:51:58.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc531c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc4c0b0420 0x7fdc4c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc531c3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc4c070630 con 0x7fdc4c15ec30 2024-04-25T16:51:58.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc531c3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc4c06f6b0 con 0x7fdc4c06f2a0 2024-04-25T16:51:58.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc531c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4c0aeaa0 con 0x7fdc4c0b0420 2024-04-25T16:51:58.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc51760640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fdc4c15ec30 0x7fdc4c06cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38760/0 (socket says 172.21.15.94:38760) 2024-04-25T16:51:58.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc51760640 1 -- 172.21.15.94:0/2789526845 learned_addr learned my addr 172.21.15.94:0/2789526845 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:58.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc50f5f640 1 --2- 172.21.15.94:0/2789526845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc4c0b0420 0x7fdc4c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3644025949 0 0) 0x7fdc4c070630 con 0x7fdc4c15ec30 2024-04-25T16:51:58.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc34003530 con 0x7fdc4c15ec30 2024-04-25T16:51:58.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4123523841 0 0) 0x7fdc34003530 con 0x7fdc4c15ec30 2024-04-25T16:51:58.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc4c0b0420 msgr2=0x7fdc4c06ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:58.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 --2- 172.21.15.94:0/2789526845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc4c0b0420 0x7fdc4c06ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 >> v1:172.21.15.138:6789/0 conn(0x7fdc4c06f2a0 legacy=0x7fdc4c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4c06d420 con 0x7fdc4c15ec30 2024-04-25T16:51:58.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fdc44001a00 con 0x7fdc4c15ec30 2024-04-25T16:51:58.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 >> v1:172.21.15.94:6789/0 conn(0x7fdc4c15ec30 legacy=0x7fdc4c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 --2- 172.21.15.94:0/2789526845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc340039f0 0x7fdc34003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 --2- 172.21.15.94:0/2789526845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc4c0b0420 0x7fdc34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 --2- 172.21.15.94:0/2789526845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc34004d50 0x7fdc34005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c0aeaa0 con 0x7fdc34004d50 2024-04-25T16:51:58.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3400bb00 con 0x7fdc340039f0 2024-04-25T16:51:58.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdc3400bd00 con 0x7fdc4c0b0420 2024-04-25T16:51:58.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc51760640 1 --2- 172.21.15.94:0/2789526845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc34004d50 0x7fdc34005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc50f5f640 1 --2- 172.21.15.94:0/2789526845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc340039f0 0x7fdc34003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc4bfff640 1 --2- 172.21.15.94:0/2789526845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc4c0b0420 0x7fdc34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc51760640 1 -- 172.21.15.94:0/2789526845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc4c0b0420 msgr2=0x7fdc34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:58.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc51760640 1 --2- 172.21.15.94:0/2789526845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc4c0b0420 0x7fdc34004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc51760640 1 -- 172.21.15.94:0/2789526845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc340039f0 msgr2=0x7fdc34003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:58.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc51760640 1 --2- 172.21.15.94:0/2789526845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc340039f0 0x7fdc34003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc51760640 1 -- 172.21.15.94:0/2789526845 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc34003530 con 0x7fdc34004d50 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.571+0000 7fdc50f5f640 1 --2- 172.21.15.94:0/2789526845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc340039f0 0x7fdc34003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.575+0000 7fdc51760640 1 --2- 172.21.15.94:0/2789526845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc34004d50 0x7fdc34005160 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400a810 tx=0x7fdc4401d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb2042c799b5bcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.575+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4400a460 con 0x7fdc34004d50 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.575+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc440ad650 con 0x7fdc34004d50 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.575+0000 7fdc4b7fe640 1 -- 172.21.15.94:0/2789526845 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc440ada40 con 0x7fdc34004d50 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.575+0000 7fdc531c3640 1 -- 172.21.15.94:0/2789526845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc34004d50 msgr2=0x7fdc34005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.575+0000 7fdc531c3640 1 --2- 172.21.15.94:0/2789526845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc34004d50 0x7fdc34005160 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400a810 tx=0x7fdc4401d340 comp rx=0 tx=0).stop 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.575+0000 7fdc51760640 1 -- 172.21.15.94:0/2789526845 reap_dead start 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.575+0000 7fdc531c3640 1 -- 172.21.15.94:0/2789526845 shutdown_connections 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.575+0000 7fdc531c3640 1 -- 172.21.15.94:0/2789526845 >> 172.21.15.94:0/2789526845 conn(0x7fdc4c0b47f0 msgr2=0x7fdc4c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.575+0000 7fdc531c3640 1 -- 172.21.15.94:0/2789526845 shutdown_connections 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.575+0000 7fdc531c3640 1 -- 172.21.15.94:0/2789526845 wait complete. 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc531c3640 1 Processor -- start 2024-04-25T16:51:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc531c3640 1 -- start start 2024-04-25T16:51:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc531c3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc34004d50 0x7fdc4c1f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc531c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc4c160d60 0x7fdc4c0cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc531c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc4c0cdcd0 0x7fdc4c152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc531c3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c0ae450 con 0x7fdc34004d50 2024-04-25T16:51:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc531c3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c0703e0 con 0x7fdc4c160d60 2024-04-25T16:51:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc531c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4c0ae730 con 0x7fdc4c0cdcd0 2024-04-25T16:51:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc50f5f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc34004d50 0x7fdc4c1f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc50f5f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc34004d50 0x7fdc4c1f3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54590/0 (socket says 172.21.15.94:54590) 2024-04-25T16:51:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc50f5f640 1 -- 172.21.15.94:0/699093009 learned_addr learned my addr 172.21.15.94:0/699093009 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc51760640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc4c0cdcd0 0x7fdc4c152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc4bfff640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc4c160d60 0x7fdc4c0cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc50f5f640 1 -- 172.21.15.94:0/699093009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc4c0cdcd0 msgr2=0x7fdc4c152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc50f5f640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc4c0cdcd0 0x7fdc4c152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc50f5f640 1 -- 172.21.15.94:0/699093009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc4c160d60 msgr2=0x7fdc4c0cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc50f5f640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc4c160d60 0x7fdc4c0cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc50f5f640 1 -- 172.21.15.94:0/699093009 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4c152170 con 0x7fdc34004d50 2024-04-25T16:51:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc50f5f640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc34004d50 0x7fdc4c1f3a80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdc400108d0 tx=0x7fdc40010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc497fa640 1 -- 172.21.15.94:0/699093009 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000f040 con 0x7fdc34004d50 2024-04-25T16:51:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc497fa640 1 -- 172.21.15.94:0/699093009 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc4000eb00 con 0x7fdc34004d50 2024-04-25T16:51:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc531c3640 1 -- 172.21.15.94:0/699093009 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4c0ce0e0 con 0x7fdc34004d50 2024-04-25T16:51:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc497fa640 1 -- 172.21.15.94:0/699093009 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4001c400 con 0x7fdc34004d50 2024-04-25T16:51:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc531c3640 1 -- 172.21.15.94:0/699093009 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc4c0cfb30 con 0x7fdc34004d50 2024-04-25T16:51:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.579+0000 7fdc531c3640 1 -- 172.21.15.94:0/699093009 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc4c076830 con 0x7fdc34004d50 2024-04-25T16:51:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.583+0000 7fdc497fa640 1 -- 172.21.15.94:0/699093009 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdc400253f0 con 0x7fdc34004d50 2024-04-25T16:51:58.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.583+0000 7fdc497fa640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdc3806f400 0x7fdc38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.583+0000 7fdc497fa640 1 -- 172.21.15.94:0/699093009 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdc4008fec0 con 0x7fdc34004d50 2024-04-25T16:51:58.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.583+0000 7fdc4bfff640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdc3806f400 0x7fdc38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.583+0000 7fdc4bfff640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdc3806f400 0x7fdc38071910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c007fd0 tx=0x7fdc3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:58.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.587+0000 7fdc497fa640 1 -- 172.21.15.94:0/699093009 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdc4c0ce0e0 con 0x7fdc34004d50 2024-04-25T16:51:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.743+0000 7fdc531c3640 1 -- 172.21.15.94:0/699093009 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdc4c0cd0d0 con 0x7fdc34004d50 2024-04-25T16:51:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.743+0000 7fdc497fa640 1 -- 172.21.15.94:0/699093009 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fdc4c152170 con 0x7fdc34004d50 2024-04-25T16:51:58.743 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:58.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.747+0000 7fdc531c3640 1 -- 172.21.15.94:0/699093009 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdc3806f400 msgr2=0x7fdc38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.747+0000 7fdc531c3640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdc3806f400 0x7fdc38071910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c007fd0 tx=0x7fdc3c009040 comp rx=0 tx=0).stop 2024-04-25T16:51:58.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.747+0000 7fdc531c3640 1 -- 172.21.15.94:0/699093009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc34004d50 msgr2=0x7fdc4c1f3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.747+0000 7fdc531c3640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc34004d50 0x7fdc4c1f3a80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdc400108d0 tx=0x7fdc40010d90 comp rx=0 tx=0).stop 2024-04-25T16:51:58.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.747+0000 7fdc531c3640 1 -- 172.21.15.94:0/699093009 shutdown_connections 2024-04-25T16:51:58.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.747+0000 7fdc531c3640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fdc3806f400 0x7fdc38071910 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.747+0000 7fdc531c3640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdc4c0cdcd0 0x7fdc4c152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.747+0000 7fdc531c3640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc4c160d60 0x7fdc4c0cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.747+0000 7fdc531c3640 1 --2- 172.21.15.94:0/699093009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc34004d50 0x7fdc4c1f3a80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.747+0000 7fdc531c3640 1 -- 172.21.15.94:0/699093009 >> 172.21.15.94:0/699093009 conn(0x7fdc4c0b47f0 msgr2=0x7fdc4c0b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:58.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.747+0000 7fdc531c3640 1 -- 172.21.15.94:0/699093009 shutdown_connections 2024-04-25T16:51:58.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.747+0000 7fdc531c3640 1 -- 172.21.15.94:0/699093009 wait complete. 2024-04-25T16:51:58.763 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:51:58.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f0864c40640 1 Processor -- start 2024-04-25T16:51:58.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f0864c40640 1 -- start start 2024-04-25T16:51:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f0864c40640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f086015c7e0 0x7f08600afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f0864c40640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f086006de20 0x7f086006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f0864c40640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0860074380 con 0x7f08600b0330 2024-04-25T16:51:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f0864c40640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f08600727e0 con 0x7f086015c7e0 2024-04-25T16:51:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f0864c40640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0860072ac0 con 0x7f086006de20 2024-04-25T16:51:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f085e59c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f086015c7e0 0x7f08600afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f085ed9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f086006de20 0x7f086006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f085e59c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f086015c7e0 0x7f08600afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:42262/0 (socket says 172.21.15.94:42262) 2024-04-25T16:51:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f085e59c640 1 -- 172.21.15.94:0/1785412097 learned_addr learned my addr 172.21.15.94:0/1785412097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f085dd9b640 1 --1- 172.21.15.94:0/1785412097 >> v1:172.21.15.138:6789/0 conn(0x7f08600b0330 0x7f086006d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:52840/0 (socket says 172.21.15.94:52840) 2024-04-25T16:51:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f085e59c640 1 -- 172.21.15.94:0/1785412097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f086006de20 msgr2=0x7f086006e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f085e59c640 1 --2- 172.21.15.94:0/1785412097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f086006de20 0x7f086006e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f085e59c640 1 -- 172.21.15.94:0/1785412097 >> v1:172.21.15.138:6789/0 conn(0x7f08600b0330 legacy=0x7f086006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.911+0000 7f085e59c640 1 -- 172.21.15.94:0/1785412097 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0860072da0 con 0x7f086015c7e0 2024-04-25T16:51:58.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.915+0000 7f085e59c640 1 --2- 172.21.15.94:0/1785412097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086015c7e0 0x7f08600afdf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0850009870 tx=0x7f085001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f985d1ea1b1d7f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:58.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.915+0000 7f085d59a640 1 -- 172.21.15.94:0/1785412097 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08500aa600 con 0x7f086015c7e0 2024-04-25T16:51:58.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.915+0000 7f085d59a640 1 -- 172.21.15.94:0/1785412097 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08500af070 con 0x7f086015c7e0 2024-04-25T16:51:58.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.915+0000 7f085d59a640 1 -- 172.21.15.94:0/1785412097 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08500aa9c0 con 0x7f086015c7e0 2024-04-25T16:51:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.915+0000 7f0864c40640 1 -- 172.21.15.94:0/1785412097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086015c7e0 msgr2=0x7f08600afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.915+0000 7f0864c40640 1 --2- 172.21.15.94:0/1785412097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086015c7e0 0x7f08600afdf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0850009870 tx=0x7f085001c4a0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.915+0000 7f0864c40640 1 -- 172.21.15.94:0/1785412097 shutdown_connections 2024-04-25T16:51:58.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.915+0000 7f0864c40640 1 --2- 172.21.15.94:0/1785412097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f086006de20 0x7f086006e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.915+0000 7f0864c40640 1 --2- 172.21.15.94:0/1785412097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086015c7e0 0x7f08600afdf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.915+0000 7f0864c40640 1 -- 172.21.15.94:0/1785412097 >> 172.21.15.94:0/1785412097 conn(0x7f08600b47f0 msgr2=0x7f08600b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:58.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.915+0000 7f0864c40640 1 -- 172.21.15.94:0/1785412097 shutdown_connections 2024-04-25T16:51:58.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.915+0000 7f0864c40640 1 -- 172.21.15.94:0/1785412097 wait complete. 2024-04-25T16:51:58.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f0864c40640 1 Processor -- start 2024-04-25T16:51:58.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f0864c40640 1 -- start start 2024-04-25T16:51:58.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f0864c40640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08600b0330 0x7f086014c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f0864c40640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086015c7e0 0x7f086006c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f0864c40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f086015b9b0 0x7f086006cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f0864c40640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f08601695d0 con 0x7f086015b9b0 2024-04-25T16:51:58.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f0864c40640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f086015cbf0 con 0x7f086015c7e0 2024-04-25T16:51:58.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f0864c40640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0860169750 con 0x7f08600b0330 2024-04-25T16:51:58.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f085ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f086015b9b0 0x7f086006cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f085ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f086015b9b0 0x7f086006cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54602/0 (socket says 172.21.15.94:54602) 2024-04-25T16:51:58.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f085ed9d640 1 -- 172.21.15.94:0/581888582 learned_addr learned my addr 172.21.15.94:0/581888582 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:58.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f085e59c640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08600b0330 0x7f086014c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f085dd9b640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086015c7e0 0x7f086006c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f085ed9d640 1 -- 172.21.15.94:0/581888582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08600b0330 msgr2=0x7f086014c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f085ed9d640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08600b0330 0x7f086014c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f085ed9d640 1 -- 172.21.15.94:0/581888582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086015c7e0 msgr2=0x7f086006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:58.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f085ed9d640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086015c7e0 0x7f086006c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:58.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f085ed9d640 1 -- 172.21.15.94:0/581888582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086014ce70 con 0x7f086015b9b0 2024-04-25T16:51:58.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f085dd9b640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086015c7e0 0x7f086006c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:58.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f085e59c640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08600b0330 0x7f086014c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:58.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f085ed9d640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f086015b9b0 0x7f086006cb40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0858009d30 tx=0x7f085800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:58.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f084effd640 1 -- 172.21.15.94:0/581888582 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858018020 con 0x7f086015b9b0 2024-04-25T16:51:58.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f084effd640 1 -- 172.21.15.94:0/581888582 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0858014480 con 0x7f086015b9b0 2024-04-25T16:51:58.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f0864c40640 1 -- 172.21.15.94:0/581888582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086014e3e0 con 0x7f086015b9b0 2024-04-25T16:51:58.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f084effd640 1 -- 172.21.15.94:0/581888582 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858014870 con 0x7f086015b9b0 2024-04-25T16:51:58.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f0864c40640 1 -- 172.21.15.94:0/581888582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086014e9a0 con 0x7f086015b9b0 2024-04-25T16:51:58.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.919+0000 7f0864c40640 1 -- 172.21.15.94:0/581888582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0860076830 con 0x7f086015b9b0 2024-04-25T16:51:58.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.923+0000 7f084effd640 1 -- 172.21.15.94:0/581888582 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f085801b020 con 0x7f086015b9b0 2024-04-25T16:51:58.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.923+0000 7f084effd640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f084006f430 0x7f0840071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:58.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.923+0000 7f084effd640 1 -- 172.21.15.94:0/581888582 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f08580902a0 con 0x7f086015b9b0 2024-04-25T16:51:58.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.923+0000 7f085e59c640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f084006f430 0x7f0840071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:58.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.923+0000 7f085e59c640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f084006f430 0x7f0840071940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f08500ae000 tx=0x7f0850008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:58.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:58.927+0000 7f084effd640 1 -- 172.21.15.94:0/581888582 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0858022030 con 0x7f086015b9b0 2024-04-25T16:51:59.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f0864c40640 1 -- 172.21.15.94:0/581888582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f08601601d0 con 0x7f086015b9b0 2024-04-25T16:51:59.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f084effd640 1 -- 172.21.15.94:0/581888582 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f0858010040 con 0x7f086015b9b0 2024-04-25T16:51:59.089 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:51:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f0864c40640 1 -- 172.21.15.94:0/581888582 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f084006f430 msgr2=0x7f0840071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f0864c40640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f084006f430 0x7f0840071940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f08500ae000 tx=0x7f0850008040 comp rx=0 tx=0).stop 2024-04-25T16:51:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f0864c40640 1 -- 172.21.15.94:0/581888582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f086015b9b0 msgr2=0x7f086006cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f0864c40640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f086015b9b0 0x7f086006cb40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0858009d30 tx=0x7f085800c490 comp rx=0 tx=0).stop 2024-04-25T16:51:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f0864c40640 1 -- 172.21.15.94:0/581888582 shutdown_connections 2024-04-25T16:51:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f0864c40640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f086015b9b0 0x7f086006cb40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f0864c40640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f086015c7e0 0x7f086006c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f0864c40640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f084006f430 0x7f0840071940 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f0864c40640 1 --2- 172.21.15.94:0/581888582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08600b0330 0x7f086014c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f0864c40640 1 -- 172.21.15.94:0/581888582 >> 172.21.15.94:0/581888582 conn(0x7f08600b47f0 msgr2=0x7f08600b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f0864c40640 1 -- 172.21.15.94:0/581888582 shutdown_connections 2024-04-25T16:51:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.091+0000 7f0864c40640 1 -- 172.21.15.94:0/581888582 wait complete. 2024-04-25T16:51:59.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:51:59.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f4a638640 1 Processor -- start 2024-04-25T16:51:59.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f4a638640 1 -- start start 2024-04-25T16:51:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f4a638640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f440d1fb0 0x7f2f44163e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f4a638640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f441611c0 0x7f2f441615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f4a638640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f440d2800 con 0x7f2f440b75e0 2024-04-25T16:51:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f4a638640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f2f441601a0 con 0x7f2f440d1fb0 2024-04-25T16:51:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f4a638640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f44160480 con 0x7f2f441611c0 2024-04-25T16:51:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f43fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f440d1fb0 0x7f2f44163e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f437fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f2f440b75e0 0x7f2f440b79f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38764/0 (socket says 172.21.15.94:38764) 2024-04-25T16:51:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f437fe640 1 -- 172.21.15.94:0/781831809 learned_addr learned my addr 172.21.15.94:0/781831809 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f48bd5640 1 --2- 172.21.15.94:0/781831809 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f441611c0 0x7f2f441615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f43fff640 1 -- 172.21.15.94:0/781831809 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f441611c0 msgr2=0x7f2f441615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f43fff640 1 --2- 172.21.15.94:0/781831809 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f441611c0 0x7f2f441615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f42ffd640 1 -- 172.21.15.94:0/781831809 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595074615 0 0) 0x7f2f440d2800 con 0x7f2f440b75e0 2024-04-25T16:51:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f43fff640 1 -- 172.21.15.94:0/781831809 >> v1:172.21.15.94:6789/0 conn(0x7f2f440b75e0 legacy=0x7f2f440b79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f43fff640 1 -- 172.21.15.94:0/781831809 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f44160760 con 0x7f2f440d1fb0 2024-04-25T16:51:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.247+0000 7f2f48bd5640 1 --2- 172.21.15.94:0/781831809 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f441611c0 0x7f2f441615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:59.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.251+0000 7f2f43fff640 1 --2- 172.21.15.94:0/781831809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f440d1fb0 0x7f2f44163e40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2f340097c0 tx=0x7f2f3401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be26ccfcd23fd82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:59.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.251+0000 7f2f42ffd640 1 -- 172.21.15.94:0/781831809 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f340b0070 con 0x7f2f440d1fb0 2024-04-25T16:51:59.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.251+0000 7f2f42ffd640 1 -- 172.21.15.94:0/781831809 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f340ab600 con 0x7f2f440d1fb0 2024-04-25T16:51:59.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.251+0000 7f2f42ffd640 1 -- 172.21.15.94:0/781831809 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f340ab9d0 con 0x7f2f440d1fb0 2024-04-25T16:51:59.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 -- 172.21.15.94:0/781831809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f440d1fb0 msgr2=0x7f2f44163e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 --2- 172.21.15.94:0/781831809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f440d1fb0 0x7f2f44163e40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2f340097c0 tx=0x7f2f3401c4a0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 -- 172.21.15.94:0/781831809 shutdown_connections 2024-04-25T16:51:59.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 --2- 172.21.15.94:0/781831809 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f441611c0 0x7f2f441615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 --2- 172.21.15.94:0/781831809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f440d1fb0 0x7f2f44163e40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 -- 172.21.15.94:0/781831809 >> 172.21.15.94:0/781831809 conn(0x7f2f440cfe50 msgr2=0x7f2f440d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:59.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 -- 172.21.15.94:0/781831809 shutdown_connections 2024-04-25T16:51:59.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 -- 172.21.15.94:0/781831809 wait complete. 2024-04-25T16:51:59.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 Processor -- start 2024-04-25T16:51:59.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 -- start start 2024-04-25T16:51:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f440b75e0 0x7f2f4414eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f440d1fb0 0x7f2f440b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f441611c0 0x7f2f440ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4416cf90 con 0x7f2f440d1fb0 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4416d110 con 0x7f2f441611c0 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f4a638640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f4416d290 con 0x7f2f440b75e0 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f440d1fb0 0x7f2f440b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f440d1fb0 0x7f2f440b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54616/0 (socket says 172.21.15.94:54616) 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f437fe640 1 -- 172.21.15.94:0/351637714 learned_addr learned my addr 172.21.15.94:0/351637714 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f43fff640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f440b75e0 0x7f2f4414eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f48bd5640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f441611c0 0x7f2f440ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f437fe640 1 -- 172.21.15.94:0/351637714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f440b75e0 msgr2=0x7f2f4414eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f437fe640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f440b75e0 0x7f2f4414eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f437fe640 1 -- 172.21.15.94:0/351637714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f441611c0 msgr2=0x7f2f440ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f437fe640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f441611c0 0x7f2f440ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f437fe640 1 -- 172.21.15.94:0/351637714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f440cdaa0 con 0x7f2f440d1fb0 2024-04-25T16:51:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.255+0000 7f2f43fff640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f440b75e0 0x7f2f4414eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:51:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.259+0000 7f2f437fe640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f440d1fb0 0x7f2f440b2f30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800c410 tx=0x7f2f3800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.259+0000 7f2f40ff9640 1 -- 172.21.15.94:0/351637714 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f380074e0 con 0x7f2f440d1fb0 2024-04-25T16:51:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.259+0000 7f2f40ff9640 1 -- 172.21.15.94:0/351637714 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f380047f0 con 0x7f2f440d1fb0 2024-04-25T16:51:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.259+0000 7f2f4a638640 1 -- 172.21.15.94:0/351637714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f440cdd80 con 0x7f2f440d1fb0 2024-04-25T16:51:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.259+0000 7f2f40ff9640 1 -- 172.21.15.94:0/351637714 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f38007680 con 0x7f2f440d1fb0 2024-04-25T16:51:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.259+0000 7f2f4a638640 1 -- 172.21.15.94:0/351637714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f440ce340 con 0x7f2f440d1fb0 2024-04-25T16:51:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.259+0000 7f2f4a638640 1 -- 172.21.15.94:0/351637714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f440b39d0 con 0x7f2f440d1fb0 2024-04-25T16:51:59.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.259+0000 7f2f40ff9640 1 -- 172.21.15.94:0/351637714 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2f38011070 con 0x7f2f440d1fb0 2024-04-25T16:51:59.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.259+0000 7f2f40ff9640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f3006f430 0x7f2f30071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.259+0000 7f2f40ff9640 1 -- 172.21.15.94:0/351637714 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2f3808f210 con 0x7f2f440d1fb0 2024-04-25T16:51:59.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.259+0000 7f2f43fff640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f3006f430 0x7f2f30071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.259+0000 7f2f43fff640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f3006f430 0x7f2f30071940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2f340ad000 tx=0x7f2f34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:59.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.263+0000 7f2f40ff9640 1 -- 172.21.15.94:0/351637714 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2f3805bd90 con 0x7f2f440d1fb0 2024-04-25T16:51:59.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.411+0000 7f2f4a638640 1 -- 172.21.15.94:0/351637714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2f4406a730 con 0x7f2f440d1fb0 2024-04-25T16:51:59.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.415+0000 7f2f40ff9640 1 -- 172.21.15.94:0/351637714 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f2f3805b690 con 0x7f2f440d1fb0 2024-04-25T16:51:59.413 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:51:59.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.415+0000 7f2f4a638640 1 -- 172.21.15.94:0/351637714 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f3006f430 msgr2=0x7f2f30071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.415+0000 7f2f4a638640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f3006f430 0x7f2f30071940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2f340ad000 tx=0x7f2f34008040 comp rx=0 tx=0).stop 2024-04-25T16:51:59.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.415+0000 7f2f4a638640 1 -- 172.21.15.94:0/351637714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f440d1fb0 msgr2=0x7f2f440b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.415+0000 7f2f4a638640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f440d1fb0 0x7f2f440b2f30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800c410 tx=0x7f2f3800c8d0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.415+0000 7f2f4a638640 1 -- 172.21.15.94:0/351637714 shutdown_connections 2024-04-25T16:51:59.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.415+0000 7f2f4a638640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f441611c0 0x7f2f440ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.415+0000 7f2f4a638640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f440d1fb0 0x7f2f440b2f30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.415+0000 7f2f4a638640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2f3006f430 0x7f2f30071940 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.415+0000 7f2f4a638640 1 --2- 172.21.15.94:0/351637714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f440b75e0 0x7f2f4414eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.415+0000 7f2f4a638640 1 -- 172.21.15.94:0/351637714 >> 172.21.15.94:0/351637714 conn(0x7f2f440cfe50 msgr2=0x7f2f440b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:59.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.415+0000 7f2f4a638640 1 -- 172.21.15.94:0/351637714 shutdown_connections 2024-04-25T16:51:59.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.415+0000 7f2f4a638640 1 -- 172.21.15.94:0/351637714 wait complete. 2024-04-25T16:51:59.434 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:51:59.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.571+0000 7feedab17640 1 Processor -- start 2024-04-25T16:51:59.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.571+0000 7feedab17640 1 -- start start 2024-04-25T16:51:59.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.571+0000 7feedab17640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed406fe90 0x7feed40702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.571+0000 7feedab17640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed406f0d0 con 0x7feed4163720 2024-04-25T16:51:59.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.571+0000 7feedab17640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed406f3b0 con 0x7feed40cca50 2024-04-25T16:51:59.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.571+0000 7feedab17640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feed40d03a0 con 0x7feed406fe90 2024-04-25T16:51:59.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.571+0000 7feed90b4640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7feed4163720 0x7feed4163b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38780/0 (socket says 172.21.15.94:38780) 2024-04-25T16:51:59.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.571+0000 7feed90b4640 1 -- 172.21.15.94:0/3668445625 learned_addr learned my addr 172.21.15.94:0/3668445625 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:59.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.571+0000 7feed3fff640 1 --2- 172.21.15.94:0/3668445625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed406fe90 0x7feed40702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.571+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4187527592 0 0) 0x7feed406f0d0 con 0x7feed4163720 2024-04-25T16:51:59.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.571+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feec0003530 con 0x7feed4163720 2024-04-25T16:51:59.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 614827282 0 0) 0x7feec0003530 con 0x7feed4163720 2024-04-25T16:51:59.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed406fe90 msgr2=0x7feed40702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:59.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 --2- 172.21.15.94:0/3668445625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed406fe90 0x7feed40702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 >> v1:172.21.15.138:6789/0 conn(0x7feed40cca50 legacy=0x7feed40cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed406f690 con 0x7feed4163720 2024-04-25T16:51:59.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7feec40016a0 con 0x7feed4163720 2024-04-25T16:51:59.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 >> v1:172.21.15.94:6789/0 conn(0x7feed4163720 legacy=0x7feed4163b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 --2- 172.21.15.94:0/3668445625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed406fe90 0x7feec00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 --2- 172.21.15.94:0/3668445625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feec0003f70 0x7feec0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 --2- 172.21.15.94:0/3668445625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec0004d50 0x7feec0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feed40d03a0 con 0x7feec0004d50 2024-04-25T16:51:59.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feec000bc10 con 0x7feec0003f70 2024-04-25T16:51:59.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feec000be10 con 0x7feed406fe90 2024-04-25T16:51:59.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed90b4640 1 --2- 172.21.15.94:0/3668445625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec0004d50 0x7feec0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed88b3640 1 --2- 172.21.15.94:0/3668445625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed406fe90 0x7feec00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed3fff640 1 --2- 172.21.15.94:0/3668445625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feec0003f70 0x7feec0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed90b4640 1 -- 172.21.15.94:0/3668445625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed406fe90 msgr2=0x7feec00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:59.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed90b4640 1 --2- 172.21.15.94:0/3668445625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed406fe90 0x7feec00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed90b4640 1 -- 172.21.15.94:0/3668445625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feec0003f70 msgr2=0x7feec0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:59.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed90b4640 1 --2- 172.21.15.94:0/3668445625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feec0003f70 0x7feec0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed90b4640 1 -- 172.21.15.94:0/3668445625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec0003530 con 0x7feec0004d50 2024-04-25T16:51:59.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed90b4640 1 --2- 172.21.15.94:0/3668445625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec0004d50 0x7feec0005160 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7feec400c210 tx=0x7feec401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92eb5da7e8f86ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:59.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4007d40 con 0x7feec0004d50 2024-04-25T16:51:59.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec4005b00 con 0x7feec0004d50 2024-04-25T16:51:59.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed37fe640 1 -- 172.21.15.94:0/3668445625 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4007070 con 0x7feec0004d50 2024-04-25T16:51:59.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feedab17640 1 -- 172.21.15.94:0/3668445625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec0004d50 msgr2=0x7feec0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feedab17640 1 --2- 172.21.15.94:0/3668445625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feec0004d50 0x7feec0005160 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7feec400c210 tx=0x7feec401f3b0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feed90b4640 1 -- 172.21.15.94:0/3668445625 reap_dead start 2024-04-25T16:51:59.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feedab17640 1 -- 172.21.15.94:0/3668445625 shutdown_connections 2024-04-25T16:51:59.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feedab17640 1 -- 172.21.15.94:0/3668445625 >> 172.21.15.94:0/3668445625 conn(0x7feed40ce490 msgr2=0x7feed40ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:59.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feedab17640 1 -- 172.21.15.94:0/3668445625 shutdown_connections 2024-04-25T16:51:59.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.575+0000 7feedab17640 1 -- 172.21.15.94:0/3668445625 wait complete. 2024-04-25T16:51:59.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feedab17640 1 Processor -- start 2024-04-25T16:51:59.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feedab17640 1 -- start start 2024-04-25T16:51:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feedab17640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec0004d50 0x7feed41f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feedab17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed41606d0 0x7feed41f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feedab17640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed41f9900 0x7feed40c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feedab17640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feed40cce60 con 0x7feed41606d0 2024-04-25T16:51:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feedab17640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feed40cd150 con 0x7feed41f9900 2024-04-25T16:51:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feedab17640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feed4164240 con 0x7feec0004d50 2024-04-25T16:51:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed41606d0 0x7feed41f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed41606d0 0x7feed41f93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54642/0 (socket says 172.21.15.94:54642) 2024-04-25T16:51:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed3fff640 1 -- 172.21.15.94:0/2885481934 learned_addr learned my addr 172.21.15.94:0/2885481934 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed90b4640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed41f9900 0x7feed40c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed88b3640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec0004d50 0x7feed41f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed3fff640 1 -- 172.21.15.94:0/2885481934 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec0004d50 msgr2=0x7feed41f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed3fff640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec0004d50 0x7feed41f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed3fff640 1 -- 172.21.15.94:0/2885481934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed41f9900 msgr2=0x7feed40c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed3fff640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed41f9900 0x7feed40c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed3fff640 1 -- 172.21.15.94:0/2885481934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed41f9d10 con 0x7feed41606d0 2024-04-25T16:51:59.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed90b4640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed41f9900 0x7feed40c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:59.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed3fff640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed41606d0 0x7feed41f93c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feebc00f350 tx=0x7feebc00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:59.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed17fa640 1 -- 172.21.15.94:0/2885481934 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc01c070 con 0x7feed41606d0 2024-04-25T16:51:59.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed17fa640 1 -- 172.21.15.94:0/2885481934 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc017620 con 0x7feed41606d0 2024-04-25T16:51:59.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feedab17640 1 -- 172.21.15.94:0/2885481934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed41e9380 con 0x7feed41606d0 2024-04-25T16:51:59.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feed17fa640 1 -- 172.21.15.94:0/2885481934 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc017a10 con 0x7feed41606d0 2024-04-25T16:51:59.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.579+0000 7feedab17640 1 -- 172.21.15.94:0/2885481934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed41e97f0 con 0x7feed41606d0 2024-04-25T16:51:59.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.583+0000 7feedab17640 1 -- 172.21.15.94:0/2885481934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed41e9550 con 0x7feed41606d0 2024-04-25T16:51:59.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.583+0000 7feed17fa640 1 -- 172.21.15.94:0/2885481934 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feebc01f070 con 0x7feed41606d0 2024-04-25T16:51:59.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.583+0000 7feed17fa640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feeb406f400 0x7feeb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.583+0000 7feed17fa640 1 -- 172.21.15.94:0/2885481934 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7feebc08fd70 con 0x7feed41606d0 2024-04-25T16:51:59.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.583+0000 7feed88b3640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feeb406f400 0x7feeb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.583+0000 7feed88b3640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feeb406f400 0x7feeb4071910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feed40d19e0 tx=0x7feecc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:59.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.587+0000 7feed17fa640 1 -- 172.21.15.94:0/2885481934 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feed41e9550 con 0x7feed41606d0 2024-04-25T16:51:59.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.759+0000 7feedab17640 1 -- 172.21.15.94:0/2885481934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feed406a730 con 0x7feed41606d0 2024-04-25T16:51:59.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.763+0000 7feed17fa640 1 -- 172.21.15.94:0/2885481934 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7feed41f9d10 con 0x7feed41606d0 2024-04-25T16:51:59.760 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:51:59.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.763+0000 7feedab17640 1 -- 172.21.15.94:0/2885481934 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feeb406f400 msgr2=0x7feeb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.763+0000 7feedab17640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feeb406f400 0x7feeb4071910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feed40d19e0 tx=0x7feecc009040 comp rx=0 tx=0).stop 2024-04-25T16:51:59.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.763+0000 7feedab17640 1 -- 172.21.15.94:0/2885481934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed41606d0 msgr2=0x7feed41f93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.763+0000 7feedab17640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed41606d0 0x7feed41f93c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feebc00f350 tx=0x7feebc00f810 comp rx=0 tx=0).stop 2024-04-25T16:51:59.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.763+0000 7feedab17640 1 -- 172.21.15.94:0/2885481934 shutdown_connections 2024-04-25T16:51:59.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.763+0000 7feedab17640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed41f9900 0x7feed40c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.763+0000 7feedab17640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed41606d0 0x7feed41f93c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.763+0000 7feedab17640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feeb406f400 0x7feeb4071910 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.763+0000 7feedab17640 1 --2- 172.21.15.94:0/2885481934 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec0004d50 0x7feed41f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.763+0000 7feedab17640 1 -- 172.21.15.94:0/2885481934 >> 172.21.15.94:0/2885481934 conn(0x7feed40ce490 msgr2=0x7feed40cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:59.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.763+0000 7feedab17640 1 -- 172.21.15.94:0/2885481934 shutdown_connections 2024-04-25T16:51:59.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.763+0000 7feedab17640 1 -- 172.21.15.94:0/2885481934 wait complete. 2024-04-25T16:51:59.782 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:51:59.782 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:51:59.789 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:51:59.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.935+0000 7fca7ecda640 1 Processor -- start 2024-04-25T16:51:59.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.935+0000 7fca7ecda640 1 -- start start 2024-04-25T16:51:59.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7ecda640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca7815cf40 0x7fca7806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7ecda640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fca7806dc50 0x7fca780af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7ecda640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca7806e060 con 0x7fca78070180 2024-04-25T16:51:59.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7ecda640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fca7815dec0 con 0x7fca7806dc50 2024-04-25T16:51:59.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7ecda640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca78160760 con 0x7fca7815cf40 2024-04-25T16:51:59.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7ca76640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fca78070180 0x7fca78160bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38794/0 (socket says 172.21.15.94:38794) 2024-04-25T16:51:59.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7ca76640 1 -- 172.21.15.94:0/2625403727 learned_addr learned my addr 172.21.15.94:0/2625403727 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:51:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1813106805 0 0) 0x7fca700014a0 con 0x7fca78070180 2024-04-25T16:51:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca77fff640 1 --2- 172.21.15.94:0/2625403727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca7815cf40 0x7fca7806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca64003530 con 0x7fca78070180 2024-04-25T16:51:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7d277640 1 --2- 172.21.15.94:0/2625403727 >> v2:172.21.15.138:3300/0 conn(0x7fca7806dc50 0x7fca780af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1792505101 0 0) 0x7fca64003530 con 0x7fca78070180 2024-04-25T16:51:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca7815cf40 msgr2=0x7fca7806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 --2- 172.21.15.94:0/2625403727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca7815cf40 0x7fca7806d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 >> v2:172.21.15.138:3300/0 conn(0x7fca7806dc50 msgr2=0x7fca780af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 --2- 172.21.15.94:0/2625403727 >> v2:172.21.15.138:3300/0 conn(0x7fca7806dc50 0x7fca780af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca78161300 con 0x7fca78070180 2024-04-25T16:51:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fca70001bd0 con 0x7fca78070180 2024-04-25T16:51:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 >> v1:172.21.15.94:6789/0 conn(0x7fca78070180 legacy=0x7fca78160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 --2- 172.21.15.94:0/2625403727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640039c0 0x7fca64003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 --2- 172.21.15.94:0/2625403727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca7815cf40 0x7fca64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 --2- 172.21.15.94:0/2625403727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca64004d50 0x7fca64005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fca7815dec0 con 0x7fca640039c0 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca78160760 con 0x7fca64004d50 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca6400ba80 con 0x7fca7815cf40 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7ca76640 1 --2- 172.21.15.94:0/2625403727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640039c0 0x7fca64003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7d277640 1 --2- 172.21.15.94:0/2625403727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca64004d50 0x7fca64005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca77fff640 1 --2- 172.21.15.94:0/2625403727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca7815cf40 0x7fca64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7ca76640 1 -- 172.21.15.94:0/2625403727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca7815cf40 msgr2=0x7fca64004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7ca76640 1 --2- 172.21.15.94:0/2625403727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca7815cf40 0x7fca64004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7ca76640 1 -- 172.21.15.94:0/2625403727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca64004d50 msgr2=0x7fca64005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7ca76640 1 --2- 172.21.15.94:0/2625403727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca64004d50 0x7fca64005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.939+0000 7fca7ca76640 1 -- 172.21.15.94:0/2625403727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca64003530 con 0x7fca640039c0 2024-04-25T16:51:59.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.943+0000 7fca7ca76640 1 --2- 172.21.15.94:0/2625403727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640039c0 0x7fca64003e30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fca780b3e20 tx=0x7fca7001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96528a57b0dcf69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:59.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.943+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7000a690 con 0x7fca640039c0 2024-04-25T16:51:59.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.943+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca700072f0 con 0x7fca640039c0 2024-04-25T16:51:59.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.943+0000 7fca777fe640 1 -- 172.21.15.94:0/2625403727 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca700076e0 con 0x7fca640039c0 2024-04-25T16:51:59.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.943+0000 7fca7ecda640 1 -- 172.21.15.94:0/2625403727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640039c0 msgr2=0x7fca64003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.943+0000 7fca7ecda640 1 --2- 172.21.15.94:0/2625403727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640039c0 0x7fca64003e30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fca780b3e20 tx=0x7fca7001d340 comp rx=0 tx=0).stop 2024-04-25T16:51:59.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.943+0000 7fca7d277640 1 -- 172.21.15.94:0/2625403727 reap_dead start 2024-04-25T16:51:59.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.943+0000 7fca7ecda640 1 -- 172.21.15.94:0/2625403727 shutdown_connections 2024-04-25T16:51:59.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.943+0000 7fca7ecda640 1 -- 172.21.15.94:0/2625403727 >> 172.21.15.94:0/2625403727 conn(0x7fca780b47f0 msgr2=0x7fca780b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:59.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.943+0000 7fca7ecda640 1 -- 172.21.15.94:0/2625403727 shutdown_connections 2024-04-25T16:51:59.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.943+0000 7fca7ecda640 1 -- 172.21.15.94:0/2625403727 wait complete. 2024-04-25T16:51:59.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.943+0000 7fca7ecda640 1 Processor -- start 2024-04-25T16:51:59.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.943+0000 7fca7ecda640 1 -- start start 2024-04-25T16:51:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ecda640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640039c0 0x7fca780b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ecda640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca7809a4d0 0x7fca78152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ecda640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca78152a40 0x7fca78161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ecda640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fca7806fea0 con 0x7fca640039c0 2024-04-25T16:51:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ecda640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca7815dbe0 con 0x7fca7809a4d0 2024-04-25T16:51:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ecda640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca780afc10 con 0x7fca78152a40 2024-04-25T16:51:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ca76640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640039c0 0x7fca780b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ca76640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640039c0 0x7fca780b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54660/0 (socket says 172.21.15.94:54660) 2024-04-25T16:51:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ca76640 1 -- 172.21.15.94:0/20303971 learned_addr learned my addr 172.21.15.94:0/20303971 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:51:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca77fff640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca7809a4d0 0x7fca78152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7d277640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca78152a40 0x7fca78161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ca76640 1 -- 172.21.15.94:0/20303971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca78152a40 msgr2=0x7fca78161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ca76640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca78152a40 0x7fca78161640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ca76640 1 -- 172.21.15.94:0/20303971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca7809a4d0 msgr2=0x7fca78152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ca76640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca7809a4d0 0x7fca78152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ca76640 1 -- 172.21.15.94:0/20303971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca78152e50 con 0x7fca640039c0 2024-04-25T16:51:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ca76640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640039c0 0x7fca780b2420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fca7000a150 tx=0x7fca64005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca757fa640 1 -- 172.21.15.94:0/20303971 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca700b2040 con 0x7fca640039c0 2024-04-25T16:51:59.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca757fa640 1 -- 172.21.15.94:0/20303971 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca700014a0 con 0x7fca640039c0 2024-04-25T16:51:59.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ecda640 1 -- 172.21.15.94:0/20303971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca780b0f40 con 0x7fca640039c0 2024-04-25T16:51:59.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca757fa640 1 -- 172.21.15.94:0/20303971 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70001990 con 0x7fca640039c0 2024-04-25T16:51:59.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ecda640 1 -- 172.21.15.94:0/20303971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca780b1500 con 0x7fca640039c0 2024-04-25T16:51:59.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca7ecda640 1 -- 172.21.15.94:0/20303971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca78076830 con 0x7fca640039c0 2024-04-25T16:51:59.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.947+0000 7fca757fa640 1 -- 172.21.15.94:0/20303971 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fca700bd020 con 0x7fca640039c0 2024-04-25T16:51:59.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.951+0000 7fca757fa640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fca5806f3a0 0x7fca580718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:59.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.951+0000 7fca757fa640 1 -- 172.21.15.94:0/20303971 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fca7012a6f0 con 0x7fca640039c0 2024-04-25T16:51:59.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.951+0000 7fca77fff640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fca5806f3a0 0x7fca580718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:59.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.951+0000 7fca77fff640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fca5806f3a0 0x7fca580718b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca7814dd90 tx=0x7fca60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:51:59.955+0000 7fca757fa640 1 -- 172.21.15.94:0/20303971 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fca78152e50 con 0x7fca640039c0 2024-04-25T16:52:00.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.115+0000 7fca7ecda640 1 -- 172.21.15.94:0/20303971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca7806fa50 con 0x7fca640039c0 2024-04-25T16:52:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.119+0000 7fca757fa640 1 -- 172.21.15.94:0/20303971 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7fca7806e060 con 0x7fca640039c0 2024-04-25T16:52:00.117 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:00.117 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:52:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.119+0000 7fca7ecda640 1 -- 172.21.15.94:0/20303971 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fca5806f3a0 msgr2=0x7fca580718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.119+0000 7fca7ecda640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fca5806f3a0 0x7fca580718b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca7814dd90 tx=0x7fca60008040 comp rx=0 tx=0).stop 2024-04-25T16:52:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.119+0000 7fca7ecda640 1 -- 172.21.15.94:0/20303971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640039c0 msgr2=0x7fca780b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.119+0000 7fca7ecda640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640039c0 0x7fca780b2420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fca7000a150 tx=0x7fca64005840 comp rx=0 tx=0).stop 2024-04-25T16:52:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.119+0000 7fca7ecda640 1 -- 172.21.15.94:0/20303971 shutdown_connections 2024-04-25T16:52:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.119+0000 7fca7ecda640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fca5806f3a0 0x7fca580718b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.119+0000 7fca7ecda640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca78152a40 0x7fca78161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.119+0000 7fca7ecda640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca7809a4d0 0x7fca78152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.119+0000 7fca7ecda640 1 --2- 172.21.15.94:0/20303971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca640039c0 0x7fca780b2420 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.119+0000 7fca7ecda640 1 -- 172.21.15.94:0/20303971 >> 172.21.15.94:0/20303971 conn(0x7fca780b47f0 msgr2=0x7fca78165980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.119+0000 7fca7ecda640 1 -- 172.21.15.94:0/20303971 shutdown_connections 2024-04-25T16:52:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.119+0000 7fca7ecda640 1 -- 172.21.15.94:0/20303971 wait complete. 2024-04-25T16:52:00.138 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:52:00.139 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:52:00.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d2cf7b640 1 Processor -- start 2024-04-25T16:52:00.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d2cf7b640 1 -- start start 2024-04-25T16:52:00.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d2cf7b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5d2806e6b0 0x7f5d280b7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d2cf7b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5d280cf5e0 0x7f5d280cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d2cf7b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d28070170 con 0x7f5d280b7060 2024-04-25T16:52:00.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d2cf7b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f5d280b79b0 con 0x7f5d2806e6b0 2024-04-25T16:52:00.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d2cf7b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5d280b1340 con 0x7f5d280cf5e0 2024-04-25T16:52:00.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d25d9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5d2806e6b0 0x7f5d280b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d25d9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5d2806e6b0 0x7f5d280b7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54666/0 (socket says 172.21.15.94:54666) 2024-04-25T16:52:00.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d25d9b640 1 -- 172.21.15.94:0/630687628 learned_addr learned my addr 172.21.15.94:0/630687628 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:00.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d26d9d640 1 --2- 172.21.15.94:0/630687628 >> v2:172.21.15.138:3300/0 conn(0x7f5d280cf5e0 0x7f5d280cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d25d9b640 1 -- 172.21.15.94:0/630687628 >> v2:172.21.15.138:3300/0 conn(0x7f5d280cf5e0 msgr2=0x7f5d280cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:00.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d25d9b640 1 --2- 172.21.15.94:0/630687628 >> v2:172.21.15.138:3300/0 conn(0x7f5d280cf5e0 0x7f5d280cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d25d9b640 1 -- 172.21.15.94:0/630687628 >> v1:172.21.15.138:6789/0 conn(0x7f5d280b7060 legacy=0x7f5d2806dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:00.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d25d9b640 1 -- 172.21.15.94:0/630687628 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2806f0d0 con 0x7f5d2806e6b0 2024-04-25T16:52:00.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.295+0000 7f5d26d9d640 1 --2- 172.21.15.94:0/630687628 >> v2:172.21.15.138:3300/0 conn(0x7f5d280cf5e0 0x7f5d280cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:00.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.299+0000 7f5d25d9b640 1 --2- 172.21.15.94:0/630687628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d2806e6b0 0x7f5d280b7470 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5d2806c790 tx=0x7f5d1001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5c59fb0abfce0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:00.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.299+0000 7f5d2559a640 1 -- 172.21.15.94:0/630687628 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d100b0070 con 0x7f5d2806e6b0 2024-04-25T16:52:00.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.299+0000 7f5d2559a640 1 -- 172.21.15.94:0/630687628 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d100ab600 con 0x7f5d2806e6b0 2024-04-25T16:52:00.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.299+0000 7f5d2559a640 1 -- 172.21.15.94:0/630687628 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d100ab9d0 con 0x7f5d2806e6b0 2024-04-25T16:52:00.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.299+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/630687628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d2806e6b0 msgr2=0x7f5d280b7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.299+0000 7f5d2cf7b640 1 --2- 172.21.15.94:0/630687628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d2806e6b0 0x7f5d280b7470 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5d2806c790 tx=0x7f5d1001c240 comp rx=0 tx=0).stop 2024-04-25T16:52:00.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.299+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/630687628 shutdown_connections 2024-04-25T16:52:00.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.299+0000 7f5d2cf7b640 1 --2- 172.21.15.94:0/630687628 >> v2:172.21.15.138:3300/0 conn(0x7f5d280cf5e0 0x7f5d280cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.299+0000 7f5d2cf7b640 1 --2- 172.21.15.94:0/630687628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d2806e6b0 0x7f5d280b7470 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.299+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/630687628 >> 172.21.15.94:0/630687628 conn(0x7f5d280ce490 msgr2=0x7f5d280ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:00.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.299+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/630687628 shutdown_connections 2024-04-25T16:52:00.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.299+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/630687628 wait complete. 2024-04-25T16:52:00.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d2cf7b640 1 Processor -- start 2024-04-25T16:52:00.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d2cf7b640 1 -- start start 2024-04-25T16:52:00.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d2cf7b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d280b7060 0x7f5d28157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d2cf7b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cf5e0 0x7f5d28151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d2cf7b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2814ea20 0x7f5d2815a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d2cf7b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d28169560 con 0x7f5d280cf5e0 2024-04-25T16:52:00.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d2cf7b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2806fc50 con 0x7f5d2814ea20 2024-04-25T16:52:00.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d2cf7b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d281696e0 con 0x7f5d280b7060 2024-04-25T16:52:00.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d25d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cf5e0 0x7f5d28151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d25d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cf5e0 0x7f5d28151dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54672/0 (socket says 172.21.15.94:54672) 2024-04-25T16:52:00.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d25d9b640 1 -- 172.21.15.94:0/2298073802 learned_addr learned my addr 172.21.15.94:0/2298073802 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:00.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d26d9d640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2814ea20 0x7f5d2815a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d2659c640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d280b7060 0x7f5d28157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d25d9b640 1 -- 172.21.15.94:0/2298073802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d280b7060 msgr2=0x7f5d28157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d25d9b640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d280b7060 0x7f5d28157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d25d9b640 1 -- 172.21.15.94:0/2298073802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2814ea20 msgr2=0x7f5d2815a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d25d9b640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2814ea20 0x7f5d2815a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d25d9b640 1 -- 172.21.15.94:0/2298073802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2815f880 con 0x7f5d280cf5e0 2024-04-25T16:52:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d26d9d640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2814ea20 0x7f5d2815a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d25d9b640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cf5e0 0x7f5d28151dd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5d10019e70 tx=0x7f5d10018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d1effd640 1 -- 172.21.15.94:0/2298073802 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d100b9070 con 0x7f5d280cf5e0 2024-04-25T16:52:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d1effd640 1 -- 172.21.15.94:0/2298073802 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d100b4600 con 0x7f5d280cf5e0 2024-04-25T16:52:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/2298073802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d28152310 con 0x7f5d280cf5e0 2024-04-25T16:52:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d1effd640 1 -- 172.21.15.94:0/2298073802 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d100b0040 con 0x7f5d280cf5e0 2024-04-25T16:52:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/2298073802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2815fe00 con 0x7f5d280cf5e0 2024-04-25T16:52:00.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.303+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/2298073802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d2806a730 con 0x7f5d280cf5e0 2024-04-25T16:52:00.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.307+0000 7f5d1effd640 1 -- 172.21.15.94:0/2298073802 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5d100bc070 con 0x7f5d280cf5e0 2024-04-25T16:52:00.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.307+0000 7f5d1effd640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5d0806f400 0x7f5d08071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.307+0000 7f5d1effd640 1 -- 172.21.15.94:0/2298073802 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5d10128780 con 0x7f5d280cf5e0 2024-04-25T16:52:00.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.307+0000 7f5d2659c640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5d0806f400 0x7f5d08071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.307+0000 7f5d2659c640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5d0806f400 0x7f5d08071910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5d28160470 tx=0x7f5d20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:00.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.311+0000 7f5d1effd640 1 -- 172.21.15.94:0/2298073802 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5d28070170 con 0x7f5d280cf5e0 2024-04-25T16:52:00.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.467+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/2298073802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5d280cfc00 con 0x7f5d280cf5e0 2024-04-25T16:52:00.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.471+0000 7f5d1effd640 1 -- 172.21.15.94:0/2298073802 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f5d100f51d0 con 0x7f5d280cf5e0 2024-04-25T16:52:00.468 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:52:00.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.471+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/2298073802 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5d0806f400 msgr2=0x7f5d08071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.471+0000 7f5d2cf7b640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5d0806f400 0x7f5d08071910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5d28160470 tx=0x7f5d20008040 comp rx=0 tx=0).stop 2024-04-25T16:52:00.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.471+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/2298073802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cf5e0 msgr2=0x7f5d28151dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.471+0000 7f5d2cf7b640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cf5e0 0x7f5d28151dd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5d10019e70 tx=0x7f5d10018fe0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.471+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/2298073802 shutdown_connections 2024-04-25T16:52:00.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.471+0000 7f5d2cf7b640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2814ea20 0x7f5d2815a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.471+0000 7f5d2cf7b640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cf5e0 0x7f5d28151dd0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.471+0000 7f5d2cf7b640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5d0806f400 0x7f5d08071910 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.471+0000 7f5d2cf7b640 1 --2- 172.21.15.94:0/2298073802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d280b7060 0x7f5d28157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.471+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/2298073802 >> 172.21.15.94:0/2298073802 conn(0x7f5d280ce490 msgr2=0x7f5d280b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:00.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.471+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/2298073802 shutdown_connections 2024-04-25T16:52:00.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.471+0000 7f5d2cf7b640 1 -- 172.21.15.94:0/2298073802 wait complete. 2024-04-25T16:52:00.489 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:52:00.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f780c83a640 1 Processor -- start 2024-04-25T16:52:00.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f780c83a640 1 -- start start 2024-04-25T16:52:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f780c83a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f78080d0190 0x7f7808073a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f780c83a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7808073f50 0x7f7808161890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f780c83a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f780815ef20 con 0x7f78080b68a0 2024-04-25T16:52:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f780c83a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7808074360 con 0x7f78080d0190 2024-04-25T16:52:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f780c83a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f780815e3a0 con 0x7f7808073f50 2024-04-25T16:52:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f7805d9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f78080d0190 0x7f7808073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f7805d9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f78080d0190 0x7f7808073a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54678/0 (socket says 172.21.15.94:54678) 2024-04-25T16:52:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f7805d9b640 1 -- 172.21.15.94:0/2694161687 learned_addr learned my addr 172.21.15.94:0/2694161687 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f7806d9d640 1 --2- 172.21.15.94:0/2694161687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7808073f50 0x7f7808161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f7805d9b640 1 -- 172.21.15.94:0/2694161687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7808073f50 msgr2=0x7f7808161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f7805d9b640 1 --2- 172.21.15.94:0/2694161687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7808073f50 0x7f7808161890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f7805d9b640 1 -- 172.21.15.94:0/2694161687 >> v1:172.21.15.138:6789/0 conn(0x7f78080b68a0 legacy=0x7f780815dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.631+0000 7f7805d9b640 1 -- 172.21.15.94:0/2694161687 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780815e680 con 0x7f78080d0190 2024-04-25T16:52:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.635+0000 7f7805d9b640 1 --2- 172.21.15.94:0/2694161687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78080d0190 0x7f7808073a10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009870 tx=0x7f77f801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae0e3113935afe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.635+0000 7f780559a640 1 -- 172.21.15.94:0/2694161687 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f80b1070 con 0x7f78080d0190 2024-04-25T16:52:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.635+0000 7f780559a640 1 -- 172.21.15.94:0/2694161687 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f80ac630 con 0x7f78080d0190 2024-04-25T16:52:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.635+0000 7f780559a640 1 -- 172.21.15.94:0/2694161687 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f80a8050 con 0x7f78080d0190 2024-04-25T16:52:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.635+0000 7f780c83a640 1 -- 172.21.15.94:0/2694161687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78080d0190 msgr2=0x7f7808073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.635+0000 7f780c83a640 1 --2- 172.21.15.94:0/2694161687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78080d0190 0x7f7808073a10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009870 tx=0x7f77f801c520 comp rx=0 tx=0).stop 2024-04-25T16:52:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.635+0000 7f780c83a640 1 -- 172.21.15.94:0/2694161687 shutdown_connections 2024-04-25T16:52:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.635+0000 7f780c83a640 1 --2- 172.21.15.94:0/2694161687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7808073f50 0x7f7808161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.635+0000 7f780c83a640 1 --2- 172.21.15.94:0/2694161687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78080d0190 0x7f7808073a10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.635+0000 7f780c83a640 1 -- 172.21.15.94:0/2694161687 >> 172.21.15.94:0/2694161687 conn(0x7f78080d2690 msgr2=0x7f78080d2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.635+0000 7f780c83a640 1 -- 172.21.15.94:0/2694161687 shutdown_connections 2024-04-25T16:52:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.635+0000 7f780c83a640 1 -- 172.21.15.94:0/2694161687 wait complete. 2024-04-25T16:52:00.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780c83a640 1 Processor -- start 2024-04-25T16:52:00.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780c83a640 1 -- start start 2024-04-25T16:52:00.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780c83a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7808073f50 0x7f78081516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780c83a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78080b68a0 0x7f7808152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780c83a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78080d0190 0x7f780806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780c83a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f780816aa60 con 0x7f7808073f50 2024-04-25T16:52:00.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780c83a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f780816a8e0 con 0x7f78080b68a0 2024-04-25T16:52:00.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780c83a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f780816abe0 con 0x7f78080d0190 2024-04-25T16:52:00.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7808073f50 0x7f78081516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7808073f50 0x7f78081516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54690/0 (socket says 172.21.15.94:54690) 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780659c640 1 -- 172.21.15.94:0/2233251178 learned_addr learned my addr 172.21.15.94:0/2233251178 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f7805d9b640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78080b68a0 0x7f7808152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f7806d9d640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78080d0190 0x7f780806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780659c640 1 -- 172.21.15.94:0/2233251178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78080d0190 msgr2=0x7f780806ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780659c640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78080d0190 0x7f780806ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780659c640 1 -- 172.21.15.94:0/2233251178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78080b68a0 msgr2=0x7f7808152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780659c640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78080b68a0 0x7f7808152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780659c640 1 -- 172.21.15.94:0/2233251178 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780814af70 con 0x7f7808073f50 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f7806d9d640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78080d0190 0x7f780806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780659c640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7808073f50 0x7f78081516a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b410 tx=0x7f77f000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f77feffd640 1 -- 172.21.15.94:0/2233251178 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0015030 con 0x7f7808073f50 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f77feffd640 1 -- 172.21.15.94:0/2233251178 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f0004020 con 0x7f7808073f50 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780c83a640 1 -- 172.21.15.94:0/2233251178 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780814b250 con 0x7f7808073f50 2024-04-25T16:52:00.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f77feffd640 1 -- 172.21.15.94:0/2233251178 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0004410 con 0x7f7808073f50 2024-04-25T16:52:00.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780c83a640 1 -- 172.21.15.94:0/2233251178 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780814aa20 con 0x7f7808073f50 2024-04-25T16:52:00.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f780c83a640 1 -- 172.21.15.94:0/2233251178 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780806a730 con 0x7f7808073f50 2024-04-25T16:52:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.639+0000 7f77feffd640 1 -- 172.21.15.94:0/2233251178 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f77f0018070 con 0x7f7808073f50 2024-04-25T16:52:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.643+0000 7f77feffd640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f77e806f400 0x7f77e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.643+0000 7f77feffd640 1 -- 172.21.15.94:0/2233251178 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f77f008f270 con 0x7f7808073f50 2024-04-25T16:52:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.643+0000 7f7805d9b640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f77e806f400 0x7f77e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.643+0000 7f7805d9b640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f77e806f400 0x7f77e8071910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f77f80ae000 tx=0x7f77f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:00.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.647+0000 7f77feffd640 1 -- 172.21.15.94:0/2233251178 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f77f005bd60 con 0x7f7808073f50 2024-04-25T16:52:00.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f780c83a640 1 -- 172.21.15.94:0/2233251178 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f78080d19d0 con 0x7f7808073f50 2024-04-25T16:52:00.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f77feffd640 1 -- 172.21.15.94:0/2233251178 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f77f005b660 con 0x7f7808073f50 2024-04-25T16:52:00.805 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:52:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f780c83a640 1 -- 172.21.15.94:0/2233251178 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f77e806f400 msgr2=0x7f77e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f780c83a640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f77e806f400 0x7f77e8071910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f77f80ae000 tx=0x7f77f8008040 comp rx=0 tx=0).stop 2024-04-25T16:52:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f780c83a640 1 -- 172.21.15.94:0/2233251178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7808073f50 msgr2=0x7f78081516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f780c83a640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7808073f50 0x7f78081516a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b410 tx=0x7f77f000b8d0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f780c83a640 1 -- 172.21.15.94:0/2233251178 shutdown_connections 2024-04-25T16:52:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f780c83a640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f77e806f400 0x7f77e8071910 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f780c83a640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78080d0190 0x7f780806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f780c83a640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78080b68a0 0x7f7808152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f780c83a640 1 --2- 172.21.15.94:0/2233251178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7808073f50 0x7f78081516a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f780c83a640 1 -- 172.21.15.94:0/2233251178 >> 172.21.15.94:0/2233251178 conn(0x7f78080d2690 msgr2=0x7f78080b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f780c83a640 1 -- 172.21.15.94:0/2233251178 shutdown_connections 2024-04-25T16:52:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.807+0000 7f780c83a640 1 -- 172.21.15.94:0/2233251178 wait complete. 2024-04-25T16:52:00.826 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:52:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9373250640 1 Processor -- start 2024-04-25T16:52:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9373250640 1 -- start start 2024-04-25T16:52:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9373250640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f936c0b0420 0x7f936c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9373250640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936c15ec30 0x7f936c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9373250640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936c070630 con 0x7f936c06f2a0 2024-04-25T16:52:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9373250640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f936c0aeaa0 con 0x7f936c0b0420 2024-04-25T16:52:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9373250640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f936c0703e0 con 0x7f936c15ec30 2024-04-25T16:52:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9370fec640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f936c0b0420 0x7f936c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9370fec640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f936c0b0420 0x7f936c06ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54700/0 (socket says 172.21.15.94:54700) 2024-04-25T16:52:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9370fec640 1 -- 172.21.15.94:0/910840416 learned_addr learned my addr 172.21.15.94:0/910840416 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f93717ed640 1 --2- 172.21.15.94:0/910840416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936c15ec30 0x7f936c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9370fec640 1 -- 172.21.15.94:0/910840416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936c15ec30 msgr2=0x7f936c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9370fec640 1 --2- 172.21.15.94:0/910840416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936c15ec30 0x7f936c06cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9370fec640 1 -- 172.21.15.94:0/910840416 >> v1:172.21.15.138:6789/0 conn(0x7f936c06f2a0 legacy=0x7f936c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.967+0000 7f9370fec640 1 -- 172.21.15.94:0/910840416 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936c06f6b0 con 0x7f936c0b0420 2024-04-25T16:52:00.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.971+0000 7f9370fec640 1 --2- 172.21.15.94:0/910840416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f936c0b0420 0x7f936c06ed60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9360009870 tx=0x7f936001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a42ab5394ff243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:00.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.971+0000 7f936b7fe640 1 -- 172.21.15.94:0/910840416 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600aa610 con 0x7f936c0b0420 2024-04-25T16:52:00.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.971+0000 7f936b7fe640 1 -- 172.21.15.94:0/910840416 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93600af070 con 0x7f936c0b0420 2024-04-25T16:52:00.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.971+0000 7f936b7fe640 1 -- 172.21.15.94:0/910840416 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600aa9d0 con 0x7f936c0b0420 2024-04-25T16:52:00.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.971+0000 7f9373250640 1 -- 172.21.15.94:0/910840416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f936c0b0420 msgr2=0x7f936c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.971+0000 7f9373250640 1 --2- 172.21.15.94:0/910840416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f936c0b0420 0x7f936c06ed60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9360009870 tx=0x7f936001c4f0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.971+0000 7f9373250640 1 -- 172.21.15.94:0/910840416 shutdown_connections 2024-04-25T16:52:00.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.971+0000 7f9373250640 1 --2- 172.21.15.94:0/910840416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936c15ec30 0x7f936c06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.971+0000 7f9373250640 1 --2- 172.21.15.94:0/910840416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f936c0b0420 0x7f936c06ed60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.971+0000 7f9373250640 1 -- 172.21.15.94:0/910840416 >> 172.21.15.94:0/910840416 conn(0x7f936c0b47f0 msgr2=0x7f936c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:00.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.971+0000 7f9373250640 1 -- 172.21.15.94:0/910840416 shutdown_connections 2024-04-25T16:52:00.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.971+0000 7f9373250640 1 -- 172.21.15.94:0/910840416 wait complete. 2024-04-25T16:52:00.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9373250640 1 Processor -- start 2024-04-25T16:52:00.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9373250640 1 -- start start 2024-04-25T16:52:00.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9373250640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f936c06f2a0 0x7f936c152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9373250640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f936c0b0420 0x7f936c1f9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9373250640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936c15ec30 0x7f936c1506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9373250640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f936c0b0830 con 0x7f936c06f2a0 2024-04-25T16:52:00.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9373250640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f936c0aef30 con 0x7f936c0b0420 2024-04-25T16:52:00.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9373250640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f936c0b09b0 con 0x7f936c15ec30 2024-04-25T16:52:00.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9370fec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f936c06f2a0 0x7f936c152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9370fec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f936c06f2a0 0x7f936c152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54712/0 (socket says 172.21.15.94:54712) 2024-04-25T16:52:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9370fec640 1 -- 172.21.15.94:0/476551295 learned_addr learned my addr 172.21.15.94:0/476551295 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9370fec640 1 -- 172.21.15.94:0/476551295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936c15ec30 msgr2=0x7f936c1506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f936bfff640 1 --2- 172.21.15.94:0/476551295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f936c0b0420 0x7f936c1f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9370fec640 1 --2- 172.21.15.94:0/476551295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936c15ec30 0x7f936c1506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9370fec640 1 -- 172.21.15.94:0/476551295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f936c0b0420 msgr2=0x7f936c1f9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9370fec640 1 --2- 172.21.15.94:0/476551295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f936c0b0420 0x7f936c1f9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9370fec640 1 -- 172.21.15.94:0/476551295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936c14f610 con 0x7f936c06f2a0 2024-04-25T16:52:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9370fec640 1 --2- 172.21.15.94:0/476551295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f936c06f2a0 0x7f936c152850 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f936001c320 tx=0x7f936001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f93697fa640 1 -- 172.21.15.94:0/476551295 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600bd630 con 0x7f936c06f2a0 2024-04-25T16:52:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f93697fa640 1 -- 172.21.15.94:0/476551295 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93600bd7d0 con 0x7f936c06f2a0 2024-04-25T16:52:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9373250640 1 -- 172.21.15.94:0/476551295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936c14f8f0 con 0x7f936c06f2a0 2024-04-25T16:52:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f93697fa640 1 -- 172.21.15.94:0/476551295 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600bdbc0 con 0x7f936c06f2a0 2024-04-25T16:52:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.975+0000 7f9373250640 1 -- 172.21.15.94:0/476551295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936c14feb0 con 0x7f936c06f2a0 2024-04-25T16:52:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.979+0000 7f9373250640 1 -- 172.21.15.94:0/476551295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f936c076830 con 0x7f936c06f2a0 2024-04-25T16:52:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.979+0000 7f93697fa640 1 -- 172.21.15.94:0/476551295 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f93600b33f0 con 0x7f936c06f2a0 2024-04-25T16:52:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.979+0000 7f93697fa640 1 --2- 172.21.15.94:0/476551295 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f935006f400 0x7f9350071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:00.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.979+0000 7f93697fa640 1 -- 172.21.15.94:0/476551295 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9360129ea0 con 0x7f936c06f2a0 2024-04-25T16:52:00.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.979+0000 7f936bfff640 1 --2- 172.21.15.94:0/476551295 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f935006f400 0x7f9350071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:00.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.979+0000 7f936bfff640 1 --2- 172.21.15.94:0/476551295 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f935006f400 0x7f9350071910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f936c15afc0 tx=0x7f9358008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:00.983+0000 7f93697fa640 1 -- 172.21.15.94:0/476551295 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f936c06f6b0 con 0x7f936c06f2a0 2024-04-25T16:52:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.131+0000 7f9373250640 1 -- 172.21.15.94:0/476551295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f936c0d1b00 con 0x7f936c06f2a0 2024-04-25T16:52:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.131+0000 7f93697fa640 1 -- 172.21.15.94:0/476551295 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f93600a7070 con 0x7f936c06f2a0 2024-04-25T16:52:01.131 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:52:01.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.135+0000 7f9373250640 1 -- 172.21.15.94:0/476551295 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f935006f400 msgr2=0x7f9350071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.135+0000 7f9373250640 1 --2- 172.21.15.94:0/476551295 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f935006f400 0x7f9350071910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f936c15afc0 tx=0x7f9358008040 comp rx=0 tx=0).stop 2024-04-25T16:52:01.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.135+0000 7f9373250640 1 -- 172.21.15.94:0/476551295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f936c06f2a0 msgr2=0x7f936c152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.135+0000 7f9373250640 1 --2- 172.21.15.94:0/476551295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f936c06f2a0 0x7f936c152850 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f936001c320 tx=0x7f936001ad30 comp rx=0 tx=0).stop 2024-04-25T16:52:01.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.135+0000 7f9373250640 1 -- 172.21.15.94:0/476551295 shutdown_connections 2024-04-25T16:52:01.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.135+0000 7f9373250640 1 --2- 172.21.15.94:0/476551295 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f935006f400 0x7f9350071910 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.135+0000 7f9373250640 1 --2- 172.21.15.94:0/476551295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936c15ec30 0x7f936c1506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.135+0000 7f9373250640 1 --2- 172.21.15.94:0/476551295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f936c0b0420 0x7f936c1f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.135+0000 7f9373250640 1 --2- 172.21.15.94:0/476551295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f936c06f2a0 0x7f936c152850 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.135+0000 7f9373250640 1 -- 172.21.15.94:0/476551295 >> 172.21.15.94:0/476551295 conn(0x7f936c0b47f0 msgr2=0x7f936c15f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.135+0000 7f9373250640 1 -- 172.21.15.94:0/476551295 shutdown_connections 2024-04-25T16:52:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.135+0000 7f9373250640 1 -- 172.21.15.94:0/476551295 wait complete. 2024-04-25T16:52:01.152 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:52:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1687fe9640 1 Processor -- start 2024-04-25T16:52:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1687fe9640 1 -- start start 2024-04-25T16:52:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1687fe9640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1680146ad0 0x7f16800cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1687fe9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16800d0920 con 0x7f16800b7ae0 2024-04-25T16:52:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1687fe9640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16800d0c00 con 0x7f1680161d30 2024-04-25T16:52:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1687fe9640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1680147350 con 0x7f1680146ad0 2024-04-25T16:52:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1686586640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f16800b7ae0 0x7f1680071a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38798/0 (socket says 172.21.15.94:38798) 2024-04-25T16:52:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1686586640 1 -- 172.21.15.94:0/118640227 learned_addr learned my addr 172.21.15.94:0/118640227 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1685d85640 1 --2- 172.21.15.94:0/118640227 >> v2:172.21.15.138:3300/0 conn(0x7f1680146ad0 0x7f16800cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3316563391 0 0) 0x7f16800d0920 con 0x7f16800b7ae0 2024-04-25T16:52:01.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f166c003530 con 0x7f16800b7ae0 2024-04-25T16:52:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 938289993 0 0) 0x7f166c003530 con 0x7f16800b7ae0 2024-04-25T16:52:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 >> v2:172.21.15.138:3300/0 conn(0x7f1680146ad0 msgr2=0x7f16800cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1684d83640 1 --2- 172.21.15.94:0/118640227 >> v2:172.21.15.138:3300/0 conn(0x7f1680146ad0 0x7f16800cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 >> v1:172.21.15.138:6789/0 conn(0x7f1680161d30 legacy=0x7f16800b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1680161530 con 0x7f16800b7ae0 2024-04-25T16:52:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.291+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f167c001680 con 0x7f16800b7ae0 2024-04-25T16:52:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 >> v1:172.21.15.94:6789/0 conn(0x7f16800b7ae0 legacy=0x7f1680071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1684d83640 1 --2- 172.21.15.94:0/118640227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f166c0039c0 0x7f166c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1684d83640 1 --2- 172.21.15.94:0/118640227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f166c004810 0x7f166c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1684d83640 1 --2- 172.21.15.94:0/118640227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f166c005360 0x7f166c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f166c00c260 con 0x7f166c0039c0 2024-04-25T16:52:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f166c00c410 con 0x7f166c004810 2024-04-25T16:52:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f166c00c610 con 0x7f166c005360 2024-04-25T16:52:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1685d85640 1 --2- 172.21.15.94:0/118640227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f166c0039c0 0x7f166c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1685584640 1 --2- 172.21.15.94:0/118640227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f166c004810 0x7f166c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1685d85640 1 -- 172.21.15.94:0/118640227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f166c005360 msgr2=0x7f166c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1685d85640 1 --2- 172.21.15.94:0/118640227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f166c005360 0x7f166c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1685d85640 1 -- 172.21.15.94:0/118640227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f166c004810 msgr2=0x7f166c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1685d85640 1 --2- 172.21.15.94:0/118640227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f166c004810 0x7f166c004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1685d85640 1 -- 172.21.15.94:0/118640227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c003530 con 0x7f166c0039c0 2024-04-25T16:52:01.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1686586640 1 -- 172.21.15.94:0/118640227 reap_dead start 2024-04-25T16:52:01.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1685d85640 1 --2- 172.21.15.94:0/118640227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f166c0039c0 0x7f166c003e30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1678009fc0 tx=0x7f167801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50962eed256e30c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:01.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16780b2070 con 0x7f166c0039c0 2024-04-25T16:52:01.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16780a5a70 con 0x7f166c0039c0 2024-04-25T16:52:01.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1684d83640 1 -- 172.21.15.94:0/118640227 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678009c10 con 0x7f166c0039c0 2024-04-25T16:52:01.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1687fe9640 1 -- 172.21.15.94:0/118640227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f166c0039c0 msgr2=0x7f166c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1687fe9640 1 --2- 172.21.15.94:0/118640227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f166c0039c0 0x7f166c003e30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1678009fc0 tx=0x7f167801cc80 comp rx=0 tx=0).stop 2024-04-25T16:52:01.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1687fe9640 1 -- 172.21.15.94:0/118640227 shutdown_connections 2024-04-25T16:52:01.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1687fe9640 1 --2- 172.21.15.94:0/118640227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f166c0039c0 0x7f166c003e30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1687fe9640 1 -- 172.21.15.94:0/118640227 >> 172.21.15.94:0/118640227 conn(0x7f16800d33d0 msgr2=0x7f16800d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:01.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1687fe9640 1 -- 172.21.15.94:0/118640227 shutdown_connections 2024-04-25T16:52:01.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.295+0000 7f1687fe9640 1 -- 172.21.15.94:0/118640227 wait complete. 2024-04-25T16:52:01.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1687fe9640 1 Processor -- start 2024-04-25T16:52:01.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1687fe9640 1 -- start start 2024-04-25T16:52:01.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1687fe9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f166c0039c0 0x7f168006e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1687fe9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16801e9be0 0x7f16801fa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1687fe9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16801faed0 0x7f16801fb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1687fe9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16801622c0 con 0x7f166c0039c0 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1687fe9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1680162140 con 0x7f16801faed0 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1687fe9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1680071a60 con 0x7f16801e9be0 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1685d85640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f166c0039c0 0x7f168006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1685d85640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f166c0039c0 0x7f168006e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54732/0 (socket says 172.21.15.94:54732) 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1685d85640 1 -- 172.21.15.94:0/316064457 learned_addr learned my addr 172.21.15.94:0/316064457 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1685584640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16801e9be0 0x7f16801fa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1686586640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16801faed0 0x7f16801fb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1685d85640 1 -- 172.21.15.94:0/316064457 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16801e9be0 msgr2=0x7f16801fa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1685d85640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16801e9be0 0x7f16801fa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1685d85640 1 -- 172.21.15.94:0/316064457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16801faed0 msgr2=0x7f16801fb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1685d85640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16801faed0 0x7f16801fb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1685d85640 1 -- 172.21.15.94:0/316064457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16801fb820 con 0x7f166c0039c0 2024-04-25T16:52:01.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1685584640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16801e9be0 0x7f16801fa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:01.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1685d85640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f166c0039c0 0x7f168006e980 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f16780b0040 tx=0x7f1678009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:01.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f16767fc640 1 -- 172.21.15.94:0/316064457 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16780b2040 con 0x7f166c0039c0 2024-04-25T16:52:01.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f16767fc640 1 -- 172.21.15.94:0/316064457 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16780a5870 con 0x7f166c0039c0 2024-04-25T16:52:01.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1687fe9640 1 -- 172.21.15.94:0/316064457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16801fa1b0 con 0x7f166c0039c0 2024-04-25T16:52:01.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f16767fc640 1 -- 172.21.15.94:0/316064457 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16780adc60 con 0x7f166c0039c0 2024-04-25T16:52:01.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.299+0000 7f1687fe9640 1 -- 172.21.15.94:0/316064457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f168006f590 con 0x7f166c0039c0 2024-04-25T16:52:01.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.303+0000 7f1687fe9640 1 -- 172.21.15.94:0/316064457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f168006a730 con 0x7f166c0039c0 2024-04-25T16:52:01.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.303+0000 7f16767fc640 1 -- 172.21.15.94:0/316064457 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f16780bf030 con 0x7f166c0039c0 2024-04-25T16:52:01.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.303+0000 7f16767fc640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f166406f430 0x7f1664071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.303+0000 7f1685584640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f166406f430 0x7f1664071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.303+0000 7f16767fc640 1 -- 172.21.15.94:0/316064457 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f167812a260 con 0x7f166c0039c0 2024-04-25T16:52:01.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.303+0000 7f1685584640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f166406f430 0x7f1664071940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f16801435e0 tx=0x7f1670009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:01.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.307+0000 7f16767fc640 1 -- 172.21.15.94:0/316064457 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f16780c6060 con 0x7f166c0039c0 2024-04-25T16:52:01.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.455+0000 7f1687fe9640 1 -- 172.21.15.94:0/316064457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f16800b4190 con 0x7f166c0039c0 2024-04-25T16:52:01.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.455+0000 7f16767fc640 1 -- 172.21.15.94:0/316064457 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f1678016990 con 0x7f166c0039c0 2024-04-25T16:52:01.453 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:52:01.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.455+0000 7f1687fe9640 1 -- 172.21.15.94:0/316064457 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f166406f430 msgr2=0x7f1664071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.455+0000 7f1687fe9640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f166406f430 0x7f1664071940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f16801435e0 tx=0x7f1670009040 comp rx=0 tx=0).stop 2024-04-25T16:52:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.455+0000 7f1687fe9640 1 -- 172.21.15.94:0/316064457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f166c0039c0 msgr2=0x7f168006e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.455+0000 7f1687fe9640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f166c0039c0 0x7f168006e980 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f16780b0040 tx=0x7f1678009750 comp rx=0 tx=0).stop 2024-04-25T16:52:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.455+0000 7f1687fe9640 1 -- 172.21.15.94:0/316064457 shutdown_connections 2024-04-25T16:52:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.459+0000 7f1687fe9640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16801faed0 0x7f16801fb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.459+0000 7f1687fe9640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f166406f430 0x7f1664071940 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.459+0000 7f1687fe9640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16801e9be0 0x7f16801fa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.459+0000 7f1687fe9640 1 --2- 172.21.15.94:0/316064457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f166c0039c0 0x7f168006e980 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.459+0000 7f1687fe9640 1 -- 172.21.15.94:0/316064457 >> 172.21.15.94:0/316064457 conn(0x7f16800d33d0 msgr2=0x7f16800b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.459+0000 7f1687fe9640 1 -- 172.21.15.94:0/316064457 shutdown_connections 2024-04-25T16:52:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.459+0000 7f1687fe9640 1 -- 172.21.15.94:0/316064457 wait complete. 2024-04-25T16:52:01.474 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:52:01.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.611+0000 7fca7f74a640 1 Processor -- start 2024-04-25T16:52:01.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.611+0000 7fca7f74a640 1 -- start start 2024-04-25T16:52:01.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.611+0000 7fca7f74a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca780b7970 0x7fca7815ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.611+0000 7fca7f74a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fca780cc900 0x7fca780ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.611+0000 7fca7f74a640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca780b5bc0 con 0x7fca7815dac0 2024-04-25T16:52:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.611+0000 7fca7f74a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fca780b5ea0 con 0x7fca780cc900 2024-04-25T16:52:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.611+0000 7fca7f74a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca780b6180 con 0x7fca780b7970 2024-04-25T16:52:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.611+0000 7fca7d4e6640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fca7815dac0 0x7fca780b7260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38814/0 (socket says 172.21.15.94:38814) 2024-04-25T16:52:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.611+0000 7fca7d4e6640 1 -- 172.21.15.94:0/1433161585 learned_addr learned my addr 172.21.15.94:0/1433161585 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.611+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3243262738 0 0) 0x7fca780b5bc0 con 0x7fca7815dac0 2024-04-25T16:52:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.611+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca64003530 con 0x7fca7815dac0 2024-04-25T16:52:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.611+0000 7fca7dce7640 1 --2- 172.21.15.94:0/1433161585 >> v2:172.21.15.138:3300/0 conn(0x7fca780cc900 0x7fca780ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.611+0000 7fca7cce5640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca780b7970 0x7fca7815ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 212719003 0 0) 0x7fca64003530 con 0x7fca7815dac0 2024-04-25T16:52:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca780b7970 msgr2=0x7fca7815ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca780b7970 0x7fca7815ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 >> v2:172.21.15.138:3300/0 conn(0x7fca780cc900 msgr2=0x7fca780ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 --2- 172.21.15.94:0/1433161585 >> v2:172.21.15.138:3300/0 conn(0x7fca780cc900 0x7fca780ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca780b6460 con 0x7fca7815dac0 2024-04-25T16:52:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fca74001680 con 0x7fca7815dac0 2024-04-25T16:52:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7dce7640 1 --2- 172.21.15.94:0/1433161585 >> v2:172.21.15.138:3300/0 conn(0x7fca780cc900 0x7fca780ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 >> v1:172.21.15.94:6789/0 conn(0x7fca7815dac0 legacy=0x7fca780b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca640039c0 0x7fca64003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca64004810 0x7fca64004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca64005160 0x7fca6400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fca780b5ea0 con 0x7fca64005160 2024-04-25T16:52:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca780b6180 con 0x7fca640039c0 2024-04-25T16:52:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca6400bf10 con 0x7fca64004810 2024-04-25T16:52:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7dce7640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca64005160 0x7fca6400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7cce5640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca64004810 0x7fca64004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7d4e6640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca640039c0 0x7fca64003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7dce7640 1 -- 172.21.15.94:0/1433161585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca64004810 msgr2=0x7fca64004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7dce7640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca64004810 0x7fca64004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7dce7640 1 -- 172.21.15.94:0/1433161585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca640039c0 msgr2=0x7fca64003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7dce7640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca640039c0 0x7fca64003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7dce7640 1 -- 172.21.15.94:0/1433161585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca64003530 con 0x7fca64005160 2024-04-25T16:52:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7d4e6640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca640039c0 0x7fca64003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7cce5640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca64004810 0x7fca64004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7dce7640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca64005160 0x7fca6400b9d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fca700073b0 tx=0x7fca7001cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b68110f6d05b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca700b2070 con 0x7fca64005160 2024-04-25T16:52:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca700a5de0 con 0x7fca64005160 2024-04-25T16:52:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca6ffff640 1 -- 172.21.15.94:0/1433161585 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70009c80 con 0x7fca64005160 2024-04-25T16:52:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7f74a640 1 -- 172.21.15.94:0/1433161585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca64005160 msgr2=0x7fca6400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7f74a640 1 --2- 172.21.15.94:0/1433161585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca64005160 0x7fca6400b9d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fca700073b0 tx=0x7fca7001cee0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7dce7640 1 -- 172.21.15.94:0/1433161585 reap_dead start 2024-04-25T16:52:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7f74a640 1 -- 172.21.15.94:0/1433161585 shutdown_connections 2024-04-25T16:52:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7f74a640 1 -- 172.21.15.94:0/1433161585 >> 172.21.15.94:0/1433161585 conn(0x7fca780ced30 msgr2=0x7fca780cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7f74a640 1 -- 172.21.15.94:0/1433161585 shutdown_connections 2024-04-25T16:52:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.615+0000 7fca7f74a640 1 -- 172.21.15.94:0/1433161585 wait complete. 2024-04-25T16:52:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7f74a640 1 Processor -- start 2024-04-25T16:52:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7f74a640 1 -- start start 2024-04-25T16:52:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7f74a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca64005160 0x7fca78154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7f74a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca781519e0 0x7fca78152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7f74a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca7806e320 0x7fca7806e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7f74a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fca780b7970 con 0x7fca7806e320 2024-04-25T16:52:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7f74a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca780b5850 con 0x7fca64005160 2024-04-25T16:52:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7f74a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca780b7af0 con 0x7fca781519e0 2024-04-25T16:52:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7dce7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca7806e320 0x7fca7806e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7dce7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca7806e320 0x7fca7806e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54746/0 (socket says 172.21.15.94:54746) 2024-04-25T16:52:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7dce7640 1 -- 172.21.15.94:0/4085012651 learned_addr learned my addr 172.21.15.94:0/4085012651 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7cce5640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca781519e0 0x7fca78152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7d4e6640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca64005160 0x7fca78154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7dce7640 1 -- 172.21.15.94:0/4085012651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca781519e0 msgr2=0x7fca78152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7dce7640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca781519e0 0x7fca78152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7dce7640 1 -- 172.21.15.94:0/4085012651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca64005160 msgr2=0x7fca78154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7dce7640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca64005160 0x7fca78154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7dce7640 1 -- 172.21.15.94:0/4085012651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca780d1e60 con 0x7fca7806e320 2024-04-25T16:52:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7dce7640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca7806e320 0x7fca7806e730 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fca70002240 tx=0x7fca70019c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca6dffb640 1 -- 172.21.15.94:0/4085012651 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca700b2040 con 0x7fca7806e320 2024-04-25T16:52:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca6dffb640 1 -- 172.21.15.94:0/4085012651 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca700a5be0 con 0x7fca7806e320 2024-04-25T16:52:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7f74a640 1 -- 172.21.15.94:0/4085012651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca780d18a0 con 0x7fca7806e320 2024-04-25T16:52:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca6dffb640 1 -- 172.21.15.94:0/4085012651 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70009c80 con 0x7fca7806e320 2024-04-25T16:52:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.619+0000 7fca7f74a640 1 -- 172.21.15.94:0/4085012651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7806db20 con 0x7fca7806e320 2024-04-25T16:52:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.623+0000 7fca7f74a640 1 -- 172.21.15.94:0/4085012651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca7806a730 con 0x7fca7806e320 2024-04-25T16:52:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.623+0000 7fca6dffb640 1 -- 172.21.15.94:0/4085012651 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fca700bf060 con 0x7fca7806e320 2024-04-25T16:52:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.623+0000 7fca6dffb640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fca5c06f400 0x7fca5c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.623+0000 7fca7d4e6640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fca5c06f400 0x7fca5c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.623+0000 7fca6dffb640 1 -- 172.21.15.94:0/4085012651 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fca70129a30 con 0x7fca7806e320 2024-04-25T16:52:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.623+0000 7fca7d4e6640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fca5c06f400 0x7fca5c071910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fca74001ec0 tx=0x7fca74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:01.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.627+0000 7fca6dffb640 1 -- 172.21.15.94:0/4085012651 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fca70016bf0 con 0x7fca7806e320 2024-04-25T16:52:01.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.799+0000 7fca7f74a640 1 -- 172.21.15.94:0/4085012651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fca7815e700 con 0x7fca7806e320 2024-04-25T16:52:01.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.799+0000 7fca6dffb640 1 -- 172.21.15.94:0/4085012651 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fca700f6690 con 0x7fca7806e320 2024-04-25T16:52:01.799 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:52:01.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.803+0000 7fca7f74a640 1 -- 172.21.15.94:0/4085012651 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fca5c06f400 msgr2=0x7fca5c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.803+0000 7fca7f74a640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fca5c06f400 0x7fca5c071910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fca74001ec0 tx=0x7fca74009040 comp rx=0 tx=0).stop 2024-04-25T16:52:01.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.803+0000 7fca7f74a640 1 -- 172.21.15.94:0/4085012651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca7806e320 msgr2=0x7fca7806e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:01.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.803+0000 7fca7f74a640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca7806e320 0x7fca7806e730 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fca70002240 tx=0x7fca70019c80 comp rx=0 tx=0).stop 2024-04-25T16:52:01.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.803+0000 7fca7f74a640 1 -- 172.21.15.94:0/4085012651 shutdown_connections 2024-04-25T16:52:01.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.803+0000 7fca7f74a640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca7806e320 0x7fca7806e730 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.803+0000 7fca7f74a640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fca5c06f400 0x7fca5c071910 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.803+0000 7fca7f74a640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca781519e0 0x7fca78152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.803+0000 7fca7f74a640 1 --2- 172.21.15.94:0/4085012651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca64005160 0x7fca78154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:01.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.803+0000 7fca7f74a640 1 -- 172.21.15.94:0/4085012651 >> 172.21.15.94:0/4085012651 conn(0x7fca780ced30 msgr2=0x7fca780b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:01.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.803+0000 7fca7f74a640 1 -- 172.21.15.94:0/4085012651 shutdown_connections 2024-04-25T16:52:01.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:01.803+0000 7fca7f74a640 1 -- 172.21.15.94:0/4085012651 wait complete. 2024-04-25T16:52:01.821 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-04-25T16:52:01.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:01.822 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T16:52:01.896 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:52:01.897 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:52:01.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:01.897 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T16:52:01.932 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:52:01.932 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:52:01.938 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:52:01.938 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:52:01.985 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:52:02.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557fdf9640 1 Processor -- start 2024-04-25T16:52:02.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557fdf9640 1 -- start start 2024-04-25T16:52:02.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557fdf9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f557815e520 0x7f557806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557fdf9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5578070340 0x7f557815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557fdf9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f557806ce90 con 0x7f557806d950 2024-04-25T16:52:02.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557fdf9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f557806d4c0 con 0x7f5578070340 2024-04-25T16:52:02.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557fdf9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f557806e330 con 0x7f557815e520 2024-04-25T16:52:02.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557e396640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5578070340 0x7f557815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557e396640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5578070340 0x7f557815efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54760/0 (socket says 172.21.15.94:54760) 2024-04-25T16:52:02.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557e396640 1 -- 172.21.15.94:0/1323568974 learned_addr learned my addr 172.21.15.94:0/1323568974 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:02.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557d394640 1 --2- 172.21.15.94:0/1323568974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f557815e520 0x7f557806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557e396640 1 -- 172.21.15.94:0/1323568974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f557815e520 msgr2=0x7f557806fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:02.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557e396640 1 --2- 172.21.15.94:0/1323568974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f557815e520 0x7f557806fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557e396640 1 -- 172.21.15.94:0/1323568974 >> v1:172.21.15.94:6789/0 conn(0x7f557806d950 legacy=0x7f557815de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.179+0000 7f557e396640 1 -- 172.21.15.94:0/1323568974 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557815e9d0 con 0x7f5578070340 2024-04-25T16:52:02.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557e396640 1 --2- 172.21.15.94:0/1323568974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5578070340 0x7f557815efd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5570009e50 tx=0x7f557001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207d010ddbacdb06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:02.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557cb93640 1 -- 172.21.15.94:0/1323568974 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55700a5800 con 0x7f5578070340 2024-04-25T16:52:02.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557cb93640 1 -- 172.21.15.94:0/1323568974 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55700a59a0 con 0x7f5578070340 2024-04-25T16:52:02.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557cb93640 1 -- 172.21.15.94:0/1323568974 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55700a5d70 con 0x7f5578070340 2024-04-25T16:52:02.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557fdf9640 1 -- 172.21.15.94:0/1323568974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5578070340 msgr2=0x7f557815efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557fdf9640 1 --2- 172.21.15.94:0/1323568974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5578070340 0x7f557815efd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5570009e50 tx=0x7f557001c940 comp rx=0 tx=0).stop 2024-04-25T16:52:02.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557fdf9640 1 -- 172.21.15.94:0/1323568974 shutdown_connections 2024-04-25T16:52:02.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557fdf9640 1 --2- 172.21.15.94:0/1323568974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5578070340 0x7f557815efd0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557fdf9640 1 --2- 172.21.15.94:0/1323568974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f557815e520 0x7f557806fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557fdf9640 1 -- 172.21.15.94:0/1323568974 >> 172.21.15.94:0/1323568974 conn(0x7f5578070f40 msgr2=0x7f5578071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:02.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557fdf9640 1 -- 172.21.15.94:0/1323568974 shutdown_connections 2024-04-25T16:52:02.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557fdf9640 1 -- 172.21.15.94:0/1323568974 wait complete. 2024-04-25T16:52:02.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557fdf9640 1 Processor -- start 2024-04-25T16:52:02.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.183+0000 7f557fdf9640 1 -- start start 2024-04-25T16:52:02.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557fdf9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f557806d950 0x7f5578159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557fdf9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5578070340 0x7f557813ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557fdf9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f557815e520 0x7f5578140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557fdf9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f557816a7a0 con 0x7f557806d950 2024-04-25T16:52:02.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557fdf9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5578070750 con 0x7f5578070340 2024-04-25T16:52:02.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557fdf9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f557816a920 con 0x7f557815e520 2024-04-25T16:52:02.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557db95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f557806d950 0x7f5578159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557db95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f557806d950 0x7f5578159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54772/0 (socket says 172.21.15.94:54772) 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557db95640 1 -- 172.21.15.94:0/480500261 learned_addr learned my addr 172.21.15.94:0/480500261 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557e396640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f557815e520 0x7f5578140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557d394640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5578070340 0x7f557813ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557db95640 1 -- 172.21.15.94:0/480500261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f557815e520 msgr2=0x7f5578140460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557db95640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f557815e520 0x7f5578140460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557db95640 1 -- 172.21.15.94:0/480500261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5578070340 msgr2=0x7f557813ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557db95640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5578070340 0x7f557813ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557db95640 1 -- 172.21.15.94:0/480500261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5578158e70 con 0x7f557806d950 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557e396640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f557815e520 0x7f5578140460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557db95640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f557806d950 0x7f5578159c10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f556800c370 tx=0x7f556800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f55767fc640 1 -- 172.21.15.94:0/480500261 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55680052d0 con 0x7f557806d950 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f55767fc640 1 -- 172.21.15.94:0/480500261 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5568005470 con 0x7f557806d950 2024-04-25T16:52:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f55767fc640 1 -- 172.21.15.94:0/480500261 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568007260 con 0x7f557806d950 2024-04-25T16:52:02.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557fdf9640 1 -- 172.21.15.94:0/480500261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5578168420 con 0x7f557806d950 2024-04-25T16:52:02.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557fdf9640 1 -- 172.21.15.94:0/480500261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5578159410 con 0x7f557806d950 2024-04-25T16:52:02.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f557fdf9640 1 -- 172.21.15.94:0/480500261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557806a730 con 0x7f557806d950 2024-04-25T16:52:02.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.187+0000 7f55767fc640 1 -- 172.21.15.94:0/480500261 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5568007400 con 0x7f557806d950 2024-04-25T16:52:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.191+0000 7f55767fc640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f556406f400 0x7f5564071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.191+0000 7f55767fc640 1 -- 172.21.15.94:0/480500261 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f556808eb50 con 0x7f557806d950 2024-04-25T16:52:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.191+0000 7f557d394640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f556406f400 0x7f5564071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.191+0000 7f557d394640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f556406f400 0x7f5564071910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5578160210 tx=0x7f556c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:02.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.195+0000 7f55767fc640 1 -- 172.21.15.94:0/480500261 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f556805b640 con 0x7f557806d950 2024-04-25T16:52:02.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.343+0000 7f557fdf9640 1 -- 172.21.15.94:0/480500261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5578160800 con 0x7f557806d950 2024-04-25T16:52:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.343+0000 7f55767fc640 1 -- 172.21.15.94:0/480500261 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7f5568085e20 con 0x7f557806d950 2024-04-25T16:52:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:02.343 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:52:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.347+0000 7f557fdf9640 1 -- 172.21.15.94:0/480500261 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f556406f400 msgr2=0x7f5564071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.347+0000 7f557fdf9640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f556406f400 0x7f5564071910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5578160210 tx=0x7f556c008040 comp rx=0 tx=0).stop 2024-04-25T16:52:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.347+0000 7f557fdf9640 1 -- 172.21.15.94:0/480500261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f557806d950 msgr2=0x7f5578159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.347+0000 7f557fdf9640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f557806d950 0x7f5578159c10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f556800c370 tx=0x7f556800c830 comp rx=0 tx=0).stop 2024-04-25T16:52:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.347+0000 7f557fdf9640 1 -- 172.21.15.94:0/480500261 shutdown_connections 2024-04-25T16:52:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.347+0000 7f557fdf9640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f556406f400 0x7f5564071910 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.347+0000 7f557fdf9640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f557815e520 0x7f5578140460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.347+0000 7f557fdf9640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5578070340 0x7f557813ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.347+0000 7f557fdf9640 1 --2- 172.21.15.94:0/480500261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f557806d950 0x7f5578159c10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.347+0000 7f557fdf9640 1 -- 172.21.15.94:0/480500261 >> 172.21.15.94:0/480500261 conn(0x7f5578070f40 msgr2=0x7f55780b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.347+0000 7f557fdf9640 1 -- 172.21.15.94:0/480500261 shutdown_connections 2024-04-25T16:52:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.347+0000 7f557fdf9640 1 -- 172.21.15.94:0/480500261 wait complete. 2024-04-25T16:52:02.363 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:52:02.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:52:02.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff917f05640 1 Processor -- start 2024-04-25T16:52:02.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff917f05640 1 -- start start 2024-04-25T16:52:02.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff917f05640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff91006d710 0x7ff91006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff917f05640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff91006e7a0 0x7ff91006ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff917f05640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff910074380 con 0x7ff91015c7e0 2024-04-25T16:52:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff917f05640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ff91006e060 con 0x7ff91006e7a0 2024-04-25T16:52:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff917f05640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff9100727e0 con 0x7ff91006d710 2024-04-25T16:52:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff9164a2640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff91006e7a0 0x7ff91006ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff9164a2640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff91006e7a0 0x7ff91006ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54782/0 (socket says 172.21.15.94:54782) 2024-04-25T16:52:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff9164a2640 1 -- 172.21.15.94:0/3312187693 learned_addr learned my addr 172.21.15.94:0/3312187693 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff9154a0640 1 --2- 172.21.15.94:0/3312187693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff91006d710 0x7ff91006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff9164a2640 1 -- 172.21.15.94:0/3312187693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff91006d710 msgr2=0x7ff91006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff9164a2640 1 --2- 172.21.15.94:0/3312187693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff91006d710 0x7ff91006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff9164a2640 1 -- 172.21.15.94:0/3312187693 >> v1:172.21.15.138:6789/0 conn(0x7ff91015c7e0 legacy=0x7ff9100afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.503+0000 7ff9164a2640 1 -- 172.21.15.94:0/3312187693 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff910072ac0 con 0x7ff91006e7a0 2024-04-25T16:52:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.507+0000 7ff9164a2640 1 --2- 172.21.15.94:0/3312187693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91006e7a0 0x7ff91006ebb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff908009e50 tx=0x7ff90801c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2a6966ce32ecc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.507+0000 7ff914c9f640 1 -- 172.21.15.94:0/3312187693 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9080a55d0 con 0x7ff91006e7a0 2024-04-25T16:52:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.507+0000 7ff914c9f640 1 -- 172.21.15.94:0/3312187693 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9080a5770 con 0x7ff91006e7a0 2024-04-25T16:52:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.507+0000 7ff914c9f640 1 -- 172.21.15.94:0/3312187693 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9080a5b40 con 0x7ff91006e7a0 2024-04-25T16:52:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.507+0000 7ff917f05640 1 -- 172.21.15.94:0/3312187693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91006e7a0 msgr2=0x7ff91006ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.507+0000 7ff917f05640 1 --2- 172.21.15.94:0/3312187693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91006e7a0 0x7ff91006ebb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff908009e50 tx=0x7ff90801c750 comp rx=0 tx=0).stop 2024-04-25T16:52:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.507+0000 7ff917f05640 1 -- 172.21.15.94:0/3312187693 shutdown_connections 2024-04-25T16:52:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.507+0000 7ff917f05640 1 --2- 172.21.15.94:0/3312187693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91006e7a0 0x7ff91006ebb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.507+0000 7ff917f05640 1 --2- 172.21.15.94:0/3312187693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff91006d710 0x7ff91006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.507+0000 7ff917f05640 1 -- 172.21.15.94:0/3312187693 >> 172.21.15.94:0/3312187693 conn(0x7ff9100b47f0 msgr2=0x7ff9100b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.507+0000 7ff917f05640 1 -- 172.21.15.94:0/3312187693 shutdown_connections 2024-04-25T16:52:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.507+0000 7ff917f05640 1 -- 172.21.15.94:0/3312187693 wait complete. 2024-04-25T16:52:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff917f05640 1 Processor -- start 2024-04-25T16:52:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff917f05640 1 -- start start 2024-04-25T16:52:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff917f05640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91006d710 0x7ff9101f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff917f05640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff91006e7a0 0x7ff91015ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff917f05640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff91015c7e0 0x7ff91015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff917f05640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff91015cbf0 con 0x7ff91006d710 2024-04-25T16:52:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff917f05640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff910073ca0 con 0x7ff91006e7a0 2024-04-25T16:52:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff917f05640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff9100b0500 con 0x7ff91015c7e0 2024-04-25T16:52:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff915ca1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91006d710 0x7ff9101f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff915ca1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91006d710 0x7ff9101f7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54790/0 (socket says 172.21.15.94:54790) 2024-04-25T16:52:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff915ca1640 1 -- 172.21.15.94:0/3592467059 learned_addr learned my addr 172.21.15.94:0/3592467059 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff9154a0640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff91006e7a0 0x7ff91015ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff9164a2640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff91015c7e0 0x7ff91015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff915ca1640 1 -- 172.21.15.94:0/3592467059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff91015c7e0 msgr2=0x7ff91015f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff915ca1640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff91015c7e0 0x7ff91015f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff915ca1640 1 -- 172.21.15.94:0/3592467059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff91006e7a0 msgr2=0x7ff91015ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff915ca1640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff91006e7a0 0x7ff91015ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff915ca1640 1 -- 172.21.15.94:0/3592467059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91009be90 con 0x7ff91006d710 2024-04-25T16:52:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff9164a2640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff91015c7e0 0x7ff91015f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff915ca1640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91006d710 0x7ff9101f7350 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00b440 tx=0x7ff90c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff9067fc640 1 -- 172.21.15.94:0/3592467059 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c015020 con 0x7ff91006d710 2024-04-25T16:52:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff9067fc640 1 -- 172.21.15.94:0/3592467059 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff90c011890 con 0x7ff91006d710 2024-04-25T16:52:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff917f05640 1 -- 172.21.15.94:0/3592467059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91009c170 con 0x7ff91006d710 2024-04-25T16:52:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff9067fc640 1 -- 172.21.15.94:0/3592467059 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c011c80 con 0x7ff91006d710 2024-04-25T16:52:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff917f05640 1 -- 172.21.15.94:0/3592467059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9101f7af0 con 0x7ff91006d710 2024-04-25T16:52:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.511+0000 7ff917f05640 1 -- 172.21.15.94:0/3592467059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff910076830 con 0x7ff91006d710 2024-04-25T16:52:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.515+0000 7ff9067fc640 1 -- 172.21.15.94:0/3592467059 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff90c018020 con 0x7ff91006d710 2024-04-25T16:52:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.515+0000 7ff9067fc640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff8f406f400 0x7ff8f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.515+0000 7ff9154a0640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff8f406f400 0x7ff8f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.515+0000 7ff9067fc640 1 -- 172.21.15.94:0/3592467059 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff90c08fc90 con 0x7ff91006d710 2024-04-25T16:52:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.515+0000 7ff9154a0640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff8f406f400 0x7ff8f4071910 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff91009c900 tx=0x7ff900008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.519+0000 7ff9067fc640 1 -- 172.21.15.94:0/3592467059 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff90c01f020 con 0x7ff91006d710 2024-04-25T16:52:02.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.663+0000 7ff917f05640 1 -- 172.21.15.94:0/3592467059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff9101601d0 con 0x7ff91006d710 2024-04-25T16:52:02.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.667+0000 7ff9067fc640 1 -- 172.21.15.94:0/3592467059 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7ff90c00d040 con 0x7ff91006d710 2024-04-25T16:52:02.664 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:52:02.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.667+0000 7ff917f05640 1 -- 172.21.15.94:0/3592467059 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff8f406f400 msgr2=0x7ff8f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.667+0000 7ff917f05640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff8f406f400 0x7ff8f4071910 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff91009c900 tx=0x7ff900008040 comp rx=0 tx=0).stop 2024-04-25T16:52:02.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.667+0000 7ff917f05640 1 -- 172.21.15.94:0/3592467059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91006d710 msgr2=0x7ff9101f7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.667+0000 7ff917f05640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91006d710 0x7ff9101f7350 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00b440 tx=0x7ff90c00b900 comp rx=0 tx=0).stop 2024-04-25T16:52:02.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.667+0000 7ff917f05640 1 -- 172.21.15.94:0/3592467059 shutdown_connections 2024-04-25T16:52:02.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.667+0000 7ff917f05640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff8f406f400 0x7ff8f4071910 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.667+0000 7ff917f05640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff91015c7e0 0x7ff91015f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.667+0000 7ff917f05640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff91006e7a0 0x7ff91015ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.667+0000 7ff917f05640 1 --2- 172.21.15.94:0/3592467059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff91006d710 0x7ff9101f7350 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.667+0000 7ff917f05640 1 -- 172.21.15.94:0/3592467059 >> 172.21.15.94:0/3592467059 conn(0x7ff9100b47f0 msgr2=0x7ff91006f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:02.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.667+0000 7ff917f05640 1 -- 172.21.15.94:0/3592467059 shutdown_connections 2024-04-25T16:52:02.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.667+0000 7ff917f05640 1 -- 172.21.15.94:0/3592467059 wait complete. 2024-04-25T16:52:02.685 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:52:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f12928b4640 1 Processor -- start 2024-04-25T16:52:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f12928b4640 1 -- start start 2024-04-25T16:52:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f12928b4640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f128c15c7e0 0x7f128c0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f12928b4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f128c0b0330 0x7f128c06d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f12928b4640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f128c074380 con 0x7f128c06dc50 2024-04-25T16:52:02.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f12928b4640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f128c06e060 con 0x7f128c15c7e0 2024-04-25T16:52:02.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f12928b4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f128c0727e0 con 0x7f128c0b0330 2024-04-25T16:52:02.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f128bfff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f128c15c7e0 0x7f128c0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f1283fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f128c0b0330 0x7f128c06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f1290e51640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f128c06dc50 0x7f128c06e7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:52910/0 (socket says 172.21.15.94:52910) 2024-04-25T16:52:02.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f128bfff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f128c15c7e0 0x7f128c0afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:42440/0 (socket says 172.21.15.94:42440) 2024-04-25T16:52:02.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f1283fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f128c0b0330 0x7f128c06d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:51718/0 (socket says 172.21.15.94:51718) 2024-04-25T16:52:02.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f1290e51640 1 -- 172.21.15.94:0/530811183 learned_addr learned my addr 172.21.15.94:0/530811183 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:02.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f1283fff640 1 -- 172.21.15.94:0/530811183 >> v2:172.21.15.138:3300/0 conn(0x7f128c15c7e0 msgr2=0x7f128c0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:02.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f1283fff640 1 --2- 172.21.15.94:0/530811183 >> v2:172.21.15.138:3300/0 conn(0x7f128c15c7e0 0x7f128c0afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f1283fff640 1 -- 172.21.15.94:0/530811183 >> v1:172.21.15.138:6789/0 conn(0x7f128c06dc50 legacy=0x7f128c06e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.827+0000 7f1283fff640 1 -- 172.21.15.94:0/530811183 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c072ac0 con 0x7f128c0b0330 2024-04-25T16:52:02.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.831+0000 7f1283fff640 1 --2- 172.21.15.94:0/530811183 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f128c0b0330 0x7f128c06d710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f12740097c0 tx=0x7f127401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7956b30670d869c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:02.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.831+0000 7f128b7fe640 1 -- 172.21.15.94:0/530811183 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740b1070 con 0x7f128c0b0330 2024-04-25T16:52:02.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.831+0000 7f128b7fe640 1 -- 172.21.15.94:0/530811183 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12740a8050 con 0x7f128c0b0330 2024-04-25T16:52:02.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.831+0000 7f128b7fe640 1 -- 172.21.15.94:0/530811183 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740ac860 con 0x7f128c0b0330 2024-04-25T16:52:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.831+0000 7f12928b4640 1 -- 172.21.15.94:0/530811183 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f128c0b0330 msgr2=0x7f128c06d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.831+0000 7f12928b4640 1 --2- 172.21.15.94:0/530811183 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f128c0b0330 0x7f128c06d710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f12740097c0 tx=0x7f127401c3e0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.831+0000 7f12928b4640 1 -- 172.21.15.94:0/530811183 shutdown_connections 2024-04-25T16:52:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.831+0000 7f12928b4640 1 --2- 172.21.15.94:0/530811183 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f128c0b0330 0x7f128c06d710 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.831+0000 7f12928b4640 1 --2- 172.21.15.94:0/530811183 >> v2:172.21.15.138:3300/0 conn(0x7f128c15c7e0 0x7f128c0afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.831+0000 7f12928b4640 1 -- 172.21.15.94:0/530811183 >> 172.21.15.94:0/530811183 conn(0x7f128c0b47f0 msgr2=0x7f128c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.831+0000 7f12928b4640 1 -- 172.21.15.94:0/530811183 shutdown_connections 2024-04-25T16:52:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.831+0000 7f12928b4640 1 -- 172.21.15.94:0/530811183 wait complete. 2024-04-25T16:52:02.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12928b4640 1 Processor -- start 2024-04-25T16:52:02.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12928b4640 1 -- start start 2024-04-25T16:52:02.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12928b4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f128c06dc50 0x7f128c14c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12928b4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f128c0b0330 0x7f128c14b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12928b4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f128c15c7e0 0x7f128c0b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12928b4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f128c06f2c0 con 0x7f128c06dc50 2024-04-25T16:52:02.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12928b4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f128c073ca0 con 0x7f128c0b0330 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12928b4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f128c15cbf0 con 0x7f128c15c7e0 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f128bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f128c06dc50 0x7f128c14c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f128bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f128c06dc50 0x7f128c14c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54800/0 (socket says 172.21.15.94:54800) 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f128bfff640 1 -- 172.21.15.94:0/46446136 learned_addr learned my addr 172.21.15.94:0/46446136 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f1283fff640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f128c0b0330 0x7f128c14b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f1290e51640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f128c15c7e0 0x7f128c0b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f128bfff640 1 -- 172.21.15.94:0/46446136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f128c15c7e0 msgr2=0x7f128c0b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f128bfff640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f128c15c7e0 0x7f128c0b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f128bfff640 1 -- 172.21.15.94:0/46446136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f128c0b0330 msgr2=0x7f128c14b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f128bfff640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f128c0b0330 0x7f128c14b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f128bfff640 1 -- 172.21.15.94:0/46446136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c15da60 con 0x7f128c06dc50 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f1290e51640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f128c15c7e0 0x7f128c0b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f128bfff640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f128c06dc50 0x7f128c14c290 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f127c00e8a0 tx=0x7f127c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12897fa640 1 -- 172.21.15.94:0/46446136 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c017070 con 0x7f128c06dc50 2024-04-25T16:52:02.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12897fa640 1 -- 172.21.15.94:0/46446136 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127c00c940 con 0x7f128c06dc50 2024-04-25T16:52:02.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12928b4640 1 -- 172.21.15.94:0/46446136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128c15dd40 con 0x7f128c06dc50 2024-04-25T16:52:02.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12897fa640 1 -- 172.21.15.94:0/46446136 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c00ccf0 con 0x7f128c06dc50 2024-04-25T16:52:02.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12928b4640 1 -- 172.21.15.94:0/46446136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f128c15e300 con 0x7f128c06dc50 2024-04-25T16:52:02.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.835+0000 7f12928b4640 1 -- 172.21.15.94:0/46446136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f128c076830 con 0x7f128c06dc50 2024-04-25T16:52:02.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.839+0000 7f12897fa640 1 -- 172.21.15.94:0/46446136 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f127c01a070 con 0x7f128c06dc50 2024-04-25T16:52:02.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.839+0000 7f12897fa640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f126c06f400 0x7f126c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:02.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.839+0000 7f12897fa640 1 -- 172.21.15.94:0/46446136 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f127c08f900 con 0x7f128c06dc50 2024-04-25T16:52:02.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.839+0000 7f1283fff640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f126c06f400 0x7f126c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:02.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.839+0000 7f1283fff640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f126c06f400 0x7f126c071910 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f12740ae000 tx=0x7f1274008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:02.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.843+0000 7f12897fa640 1 -- 172.21.15.94:0/46446136 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f127c05c490 con 0x7f128c06dc50 2024-04-25T16:52:02.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.991+0000 7f12928b4640 1 -- 172.21.15.94:0/46446136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f128c1601d0 con 0x7f128c06dc50 2024-04-25T16:52:02.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.991+0000 7f12897fa640 1 -- 172.21.15.94:0/46446136 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f127c05bd90 con 0x7f128c06dc50 2024-04-25T16:52:02.990 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:52:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.995+0000 7f12928b4640 1 -- 172.21.15.94:0/46446136 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f126c06f400 msgr2=0x7f126c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.995+0000 7f12928b4640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f126c06f400 0x7f126c071910 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f12740ae000 tx=0x7f1274008040 comp rx=0 tx=0).stop 2024-04-25T16:52:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.995+0000 7f12928b4640 1 -- 172.21.15.94:0/46446136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f128c06dc50 msgr2=0x7f128c14c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.995+0000 7f12928b4640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f128c06dc50 0x7f128c14c290 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f127c00e8a0 tx=0x7f127c00eca0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.995+0000 7f12928b4640 1 -- 172.21.15.94:0/46446136 shutdown_connections 2024-04-25T16:52:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.995+0000 7f12928b4640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f126c06f400 0x7f126c071910 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.995+0000 7f12928b4640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f128c15c7e0 0x7f128c0b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.995+0000 7f12928b4640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f128c0b0330 0x7f128c14b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.995+0000 7f12928b4640 1 --2- 172.21.15.94:0/46446136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f128c06dc50 0x7f128c14c290 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:02.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.995+0000 7f12928b4640 1 -- 172.21.15.94:0/46446136 >> 172.21.15.94:0/46446136 conn(0x7f128c0b47f0 msgr2=0x7f128c06fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:02.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.995+0000 7f12928b4640 1 -- 172.21.15.94:0/46446136 shutdown_connections 2024-04-25T16:52:02.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:02.995+0000 7f12928b4640 1 -- 172.21.15.94:0/46446136 wait complete. 2024-04-25T16:52:03.011 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:52:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.151+0000 7f2b1dc52640 1 Processor -- start 2024-04-25T16:52:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.151+0000 7f2b1dc52640 1 -- start start 2024-04-25T16:52:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b1dc52640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2b18158170 0x7f2b1815de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b1dc52640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2b1806e750 0x7f2b1815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b1dc52640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b1806f180 con 0x7f2b18071740 2024-04-25T16:52:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b1dc52640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f2b180cf6e0 con 0x7f2b1806e750 2024-04-25T16:52:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b1dc52640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2b1806f460 con 0x7f2b18158170 2024-04-25T16:52:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b17fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2b1806e750 0x7f2b1815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b17fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2b1806e750 0x7f2b1815e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54814/0 (socket says 172.21.15.94:54814) 2024-04-25T16:52:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b17fff640 1 -- 172.21.15.94:0/1366611633 learned_addr learned my addr 172.21.15.94:0/1366611633 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b16ffd640 1 --2- 172.21.15.94:0/1366611633 >> v2:172.21.15.138:3300/0 conn(0x7f2b18158170 0x7f2b1815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b17fff640 1 -- 172.21.15.94:0/1366611633 >> v2:172.21.15.138:3300/0 conn(0x7f2b18158170 msgr2=0x7f2b1815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:03.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b17fff640 1 --2- 172.21.15.94:0/1366611633 >> v2:172.21.15.138:3300/0 conn(0x7f2b18158170 0x7f2b1815de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b17fff640 1 -- 172.21.15.94:0/1366611633 >> v1:172.21.15.94:6789/0 conn(0x7f2b18071740 legacy=0x7f2b1806e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b17fff640 1 -- 172.21.15.94:0/1366611633 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1806f740 con 0x7f2b1806e750 2024-04-25T16:52:03.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.155+0000 7f2b16ffd640 1 --2- 172.21.15.94:0/1366611633 >> v2:172.21.15.138:3300/0 conn(0x7f2b18158170 0x7f2b1815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:03.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b17fff640 1 --2- 172.21.15.94:0/1366611633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b1806e750 0x7f2b1815e3b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b180686f0 tx=0x7f2b1001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb83e3d2b1e9616 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:03.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b167fc640 1 -- 172.21.15.94:0/1366611633 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b100a57d0 con 0x7f2b1806e750 2024-04-25T16:52:03.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b167fc640 1 -- 172.21.15.94:0/1366611633 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b100a5970 con 0x7f2b1806e750 2024-04-25T16:52:03.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b167fc640 1 -- 172.21.15.94:0/1366611633 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b100a5d40 con 0x7f2b1806e750 2024-04-25T16:52:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b1dc52640 1 -- 172.21.15.94:0/1366611633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b1806e750 msgr2=0x7f2b1815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b1dc52640 1 --2- 172.21.15.94:0/1366611633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b1806e750 0x7f2b1815e3b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b180686f0 tx=0x7f2b1001c9f0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b1dc52640 1 -- 172.21.15.94:0/1366611633 shutdown_connections 2024-04-25T16:52:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b1dc52640 1 --2- 172.21.15.94:0/1366611633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b1806e750 0x7f2b1815e3b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b1dc52640 1 --2- 172.21.15.94:0/1366611633 >> v2:172.21.15.138:3300/0 conn(0x7f2b18158170 0x7f2b1815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b1dc52640 1 -- 172.21.15.94:0/1366611633 >> 172.21.15.94:0/1366611633 conn(0x7f2b18073570 msgr2=0x7f2b18073980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b1dc52640 1 -- 172.21.15.94:0/1366611633 shutdown_connections 2024-04-25T16:52:03.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b1dc52640 1 -- 172.21.15.94:0/1366611633 wait complete. 2024-04-25T16:52:03.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b1dc52640 1 Processor -- start 2024-04-25T16:52:03.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.159+0000 7f2b1dc52640 1 -- start start 2024-04-25T16:52:03.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b1dc52640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b18071740 0x7f2b1814d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b1dc52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b18158170 0x7f2b1815c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b1dc52640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b181501d0 0x7f2b181505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b1dc52640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1816b1e0 con 0x7f2b18158170 2024-04-25T16:52:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b1dc52640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1816b060 con 0x7f2b18071740 2024-04-25T16:52:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b1dc52640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2b1816b360 con 0x7f2b181501d0 2024-04-25T16:52:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b16ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b18158170 0x7f2b1815c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b16ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b18158170 0x7f2b1815c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54818/0 (socket says 172.21.15.94:54818) 2024-04-25T16:52:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b16ffd640 1 -- 172.21.15.94:0/3629185077 learned_addr learned my addr 172.21.15.94:0/3629185077 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b177fe640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b18071740 0x7f2b1814d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b17fff640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b181501d0 0x7f2b181505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b16ffd640 1 -- 172.21.15.94:0/3629185077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b181501d0 msgr2=0x7f2b181505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b16ffd640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b181501d0 0x7f2b181505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b16ffd640 1 -- 172.21.15.94:0/3629185077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b18071740 msgr2=0x7f2b1814d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b16ffd640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b18071740 0x7f2b1814d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b16ffd640 1 -- 172.21.15.94:0/3629185077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b180b7040 con 0x7f2b18158170 2024-04-25T16:52:03.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b16ffd640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b18158170 0x7f2b1815c890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00c820 tx=0x7f2b0c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:03.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2afffff640 1 -- 172.21.15.94:0/3629185077 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c015070 con 0x7f2b18158170 2024-04-25T16:52:03.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2afffff640 1 -- 172.21.15.94:0/3629185077 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b0c007a30 con 0x7f2b18158170 2024-04-25T16:52:03.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b1dc52640 1 -- 172.21.15.94:0/3629185077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b18150b20 con 0x7f2b18158170 2024-04-25T16:52:03.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2afffff640 1 -- 172.21.15.94:0/3629185077 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c007e40 con 0x7f2b18158170 2024-04-25T16:52:03.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b1dc52640 1 -- 172.21.15.94:0/3629185077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b18159400 con 0x7f2b18158170 2024-04-25T16:52:03.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2b1dc52640 1 -- 172.21.15.94:0/3629185077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b18158e90 con 0x7f2b18158170 2024-04-25T16:52:03.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2afffff640 1 -- 172.21.15.94:0/3629185077 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2b0c018070 con 0x7f2b18158170 2024-04-25T16:52:03.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.163+0000 7f2afffff640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2b0406f430 0x7f2b04071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.167+0000 7f2b177fe640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2b0406f430 0x7f2b04071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.167+0000 7f2afffff640 1 -- 172.21.15.94:0/3629185077 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2b0c08f530 con 0x7f2b18158170 2024-04-25T16:52:03.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.167+0000 7f2b177fe640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2b0406f430 0x7f2b04071940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b180b7d60 tx=0x7f2b08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:03.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.171+0000 7f2afffff640 1 -- 172.21.15.94:0/3629185077 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2b18150b20 con 0x7f2b18158170 2024-04-25T16:52:03.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.315+0000 7f2b1dc52640 1 -- 172.21.15.94:0/3629185077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b18076830 con 0x7f2b18158170 2024-04-25T16:52:03.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.319+0000 7f2afffff640 1 -- 172.21.15.94:0/3629185077 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f2b0c05c0b0 con 0x7f2b18158170 2024-04-25T16:52:03.316 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:52:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.319+0000 7f2b1dc52640 1 -- 172.21.15.94:0/3629185077 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2b0406f430 msgr2=0x7f2b04071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.319+0000 7f2b1dc52640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2b0406f430 0x7f2b04071940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b180b7d60 tx=0x7f2b08009040 comp rx=0 tx=0).stop 2024-04-25T16:52:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.319+0000 7f2b1dc52640 1 -- 172.21.15.94:0/3629185077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b18158170 msgr2=0x7f2b1815c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.319+0000 7f2b1dc52640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b18158170 0x7f2b1815c890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00c820 tx=0x7f2b0c00cce0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.319+0000 7f2b1dc52640 1 -- 172.21.15.94:0/3629185077 shutdown_connections 2024-04-25T16:52:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.319+0000 7f2b1dc52640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2b0406f430 0x7f2b04071940 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.319+0000 7f2b1dc52640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b181501d0 0x7f2b181505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.319+0000 7f2b1dc52640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b18158170 0x7f2b1815c890 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.319+0000 7f2b1dc52640 1 --2- 172.21.15.94:0/3629185077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b18071740 0x7f2b1814d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.319+0000 7f2b1dc52640 1 -- 172.21.15.94:0/3629185077 >> 172.21.15.94:0/3629185077 conn(0x7f2b18073570 msgr2=0x7f2b181515d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.319+0000 7f2b1dc52640 1 -- 172.21.15.94:0/3629185077 shutdown_connections 2024-04-25T16:52:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.319+0000 7f2b1dc52640 1 -- 172.21.15.94:0/3629185077 wait complete. 2024-04-25T16:52:03.336 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:52:03.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c84eb4640 1 Processor -- start 2024-04-25T16:52:03.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c84eb4640 1 -- start start 2024-04-25T16:52:03.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c84eb4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5c800b6470 0x7f5c800b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c84eb4640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5c801636a0 0x7f5c80163ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c84eb4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c80163ff0 0x7f5c80160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c84eb4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f5c800b6dc0 con 0x7f5c800b6470 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c84eb4640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5c8015da10 con 0x7f5c801636a0 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c84eb4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5c800b0c10 con 0x7f5c80163ff0 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c7e59c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5c800b6470 0x7f5c800b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c7e59c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5c800b6470 0x7f5c800b6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54828/0 (socket says 172.21.15.94:54828) 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c7e59c640 1 -- 172.21.15.94:0/1837818655 learned_addr learned my addr 172.21.15.94:0/1837818655 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c7ed9d640 1 --2- 172.21.15.94:0/1837818655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c80163ff0 0x7f5c80160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c7dd9b640 1 --2- 172.21.15.94:0/1837818655 >> v2:172.21.15.138:3300/0 conn(0x7f5c801636a0 0x7f5c80163ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c7e59c640 1 -- 172.21.15.94:0/1837818655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c80163ff0 msgr2=0x7f5c80160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c7e59c640 1 --2- 172.21.15.94:0/1837818655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c80163ff0 0x7f5c80160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c7e59c640 1 -- 172.21.15.94:0/1837818655 >> v2:172.21.15.138:3300/0 conn(0x7f5c801636a0 msgr2=0x7f5c80163ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:03.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c7e59c640 1 --2- 172.21.15.94:0/1837818655 >> v2:172.21.15.138:3300/0 conn(0x7f5c801636a0 0x7f5c80163ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.479+0000 7f5c7e59c640 1 -- 172.21.15.94:0/1837818655 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c800b0ef0 con 0x7f5c800b6470 2024-04-25T16:52:03.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c7e59c640 1 --2- 172.21.15.94:0/1837818655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c800b6470 0x7f5c800b6880 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5c80074510 tx=0x7f5c7001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a0934d97cbf528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:03.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c7d59a640 1 -- 172.21.15.94:0/1837818655 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c700b0070 con 0x7f5c800b6470 2024-04-25T16:52:03.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c7d59a640 1 -- 172.21.15.94:0/1837818655 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c700ab600 con 0x7f5c800b6470 2024-04-25T16:52:03.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c7d59a640 1 -- 172.21.15.94:0/1837818655 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c700ab9d0 con 0x7f5c800b6470 2024-04-25T16:52:03.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c84eb4640 1 -- 172.21.15.94:0/1837818655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c800b6470 msgr2=0x7f5c800b6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c84eb4640 1 --2- 172.21.15.94:0/1837818655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c800b6470 0x7f5c800b6880 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5c80074510 tx=0x7f5c7001c520 comp rx=0 tx=0).stop 2024-04-25T16:52:03.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c84eb4640 1 -- 172.21.15.94:0/1837818655 shutdown_connections 2024-04-25T16:52:03.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c84eb4640 1 --2- 172.21.15.94:0/1837818655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c80163ff0 0x7f5c80160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c84eb4640 1 --2- 172.21.15.94:0/1837818655 >> v2:172.21.15.138:3300/0 conn(0x7f5c801636a0 0x7f5c80163ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c84eb4640 1 --2- 172.21.15.94:0/1837818655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c800b6470 0x7f5c800b6880 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c84eb4640 1 -- 172.21.15.94:0/1837818655 >> 172.21.15.94:0/1837818655 conn(0x7f5c80071f00 msgr2=0x7f5c80073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:03.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c84eb4640 1 -- 172.21.15.94:0/1837818655 shutdown_connections 2024-04-25T16:52:03.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c84eb4640 1 -- 172.21.15.94:0/1837818655 wait complete. 2024-04-25T16:52:03.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.483+0000 7f5c84eb4640 1 Processor -- start 2024-04-25T16:52:03.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c84eb4640 1 -- start start 2024-04-25T16:52:03.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c84eb4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c800b6470 0x7f5c8015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c84eb4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c801636a0 0x7f5c800cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c84eb4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c80163ff0 0x7f5c800d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c84eb4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c800b1e30 con 0x7f5c800b6470 2024-04-25T16:52:03.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c84eb4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c800b1cb0 con 0x7f5c80163ff0 2024-04-25T16:52:03.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c84eb4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5c800aebf0 con 0x7f5c801636a0 2024-04-25T16:52:03.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c7e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c800b6470 0x7f5c8015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c7e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c800b6470 0x7f5c8015c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54836/0 (socket says 172.21.15.94:54836) 2024-04-25T16:52:03.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c7e59c640 1 -- 172.21.15.94:0/3512551119 learned_addr learned my addr 172.21.15.94:0/3512551119 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:03.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c7ed9d640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c80163ff0 0x7f5c800d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c7dd9b640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c801636a0 0x7f5c800cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c7e59c640 1 -- 172.21.15.94:0/3512551119 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c801636a0 msgr2=0x7f5c800cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c7e59c640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c801636a0 0x7f5c800cf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c7e59c640 1 -- 172.21.15.94:0/3512551119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c80163ff0 msgr2=0x7f5c800d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c7e59c640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c80163ff0 0x7f5c800d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c7e59c640 1 -- 172.21.15.94:0/3512551119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8006d0e0 con 0x7f5c800b6470 2024-04-25T16:52:03.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c7ed9d640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c80163ff0 0x7f5c800d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:03.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c7e59c640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c800b6470 0x7f5c8015c1a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5c7001cdc0 tx=0x7f5c700192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:03.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c6effd640 1 -- 172.21.15.94:0/3512551119 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c700ba070 con 0x7f5c800b6470 2024-04-25T16:52:03.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c6effd640 1 -- 172.21.15.94:0/3512551119 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c700b4640 con 0x7f5c800b6470 2024-04-25T16:52:03.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c6effd640 1 -- 172.21.15.94:0/3512551119 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c700abe60 con 0x7f5c800b6470 2024-04-25T16:52:03.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c84eb4640 1 -- 172.21.15.94:0/3512551119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c8006d3c0 con 0x7f5c800b6470 2024-04-25T16:52:03.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c84eb4640 1 -- 172.21.15.94:0/3512551119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c8006d980 con 0x7f5c800b6470 2024-04-25T16:52:03.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c84eb4640 1 -- 172.21.15.94:0/3512551119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c80076830 con 0x7f5c800b6470 2024-04-25T16:52:03.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.487+0000 7f5c6effd640 1 -- 172.21.15.94:0/3512551119 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5c700bd070 con 0x7f5c800b6470 2024-04-25T16:52:03.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.491+0000 7f5c6effd640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5c6006f430 0x7f5c60071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.491+0000 7f5c6effd640 1 -- 172.21.15.94:0/3512551119 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5c70128690 con 0x7f5c800b6470 2024-04-25T16:52:03.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.491+0000 7f5c7dd9b640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5c6006f430 0x7f5c60071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.491+0000 7f5c7dd9b640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5c6006f430 0x7f5c60071940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5c800709d0 tx=0x7f5c74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:03.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.495+0000 7f5c6effd640 1 -- 172.21.15.94:0/3512551119 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5c70016110 con 0x7f5c800b6470 2024-04-25T16:52:03.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.651+0000 7f5c84eb4640 1 -- 172.21.15.94:0/3512551119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5c80146830 con 0x7f5c800b6470 2024-04-25T16:52:03.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.651+0000 7f5c6effd640 1 -- 172.21.15.94:0/3512551119 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f5c700fd030 con 0x7f5c800b6470 2024-04-25T16:52:03.651 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:52:03.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.655+0000 7f5c84eb4640 1 -- 172.21.15.94:0/3512551119 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5c6006f430 msgr2=0x7f5c60071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.655+0000 7f5c84eb4640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5c6006f430 0x7f5c60071940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5c800709d0 tx=0x7f5c74008040 comp rx=0 tx=0).stop 2024-04-25T16:52:03.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.655+0000 7f5c84eb4640 1 -- 172.21.15.94:0/3512551119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c800b6470 msgr2=0x7f5c8015c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.655+0000 7f5c84eb4640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c800b6470 0x7f5c8015c1a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5c7001cdc0 tx=0x7f5c700192c0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.655+0000 7f5c84eb4640 1 -- 172.21.15.94:0/3512551119 shutdown_connections 2024-04-25T16:52:03.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.655+0000 7f5c84eb4640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c80163ff0 0x7f5c800d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.655+0000 7f5c84eb4640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5c6006f430 0x7f5c60071940 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.655+0000 7f5c84eb4640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c801636a0 0x7f5c800cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.655+0000 7f5c84eb4640 1 --2- 172.21.15.94:0/3512551119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c800b6470 0x7f5c8015c1a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.655+0000 7f5c84eb4640 1 -- 172.21.15.94:0/3512551119 >> 172.21.15.94:0/3512551119 conn(0x7f5c80071f00 msgr2=0x7f5c8015dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.655+0000 7f5c84eb4640 1 -- 172.21.15.94:0/3512551119 shutdown_connections 2024-04-25T16:52:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.655+0000 7f5c84eb4640 1 -- 172.21.15.94:0/3512551119 wait complete. 2024-04-25T16:52:03.672 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:52:03.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.811+0000 7f214da46640 1 Processor -- start 2024-04-25T16:52:03.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.811+0000 7f214da46640 1 -- start start 2024-04-25T16:52:03.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f214da46640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2148072b60 0x7f214815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f214da46640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f214815cd60 0x7f214815fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f214da46640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f214806f360 con 0x7f2148160020 2024-04-25T16:52:03.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f214da46640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f21480703e0 con 0x7f2148072b60 2024-04-25T16:52:03.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f214da46640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f214806f620 con 0x7f214815cd60 2024-04-25T16:52:03.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f213e7fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f214815cd60 0x7f214815fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f2146ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2148072b60 0x7f214815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f21477fe640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2148160020 0x7f21480aea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:52916/0 (socket says 172.21.15.94:52916) 2024-04-25T16:52:03.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f21477fe640 1 -- 172.21.15.94:0/924583102 learned_addr learned my addr 172.21.15.94:0/924583102 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:03.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f2146ffd640 1 -- 172.21.15.94:0/924583102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f214815cd60 msgr2=0x7f214815fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:03.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f2146ffd640 1 --2- 172.21.15.94:0/924583102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f214815cd60 0x7f214815fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f2146ffd640 1 -- 172.21.15.94:0/924583102 >> v1:172.21.15.138:6789/0 conn(0x7f2148160020 legacy=0x7f21480aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f2146ffd640 1 -- 172.21.15.94:0/924583102 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214806f900 con 0x7f2148072b60 2024-04-25T16:52:03.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f21467fc640 1 -- 172.21.15.94:0/924583102 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2348695560 0 0) 0x7f214806f360 con 0x7f2148160020 2024-04-25T16:52:03.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.815+0000 7f213e7fc640 1 --2- 172.21.15.94:0/924583102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f214815cd60 0x7f214815fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:03.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.819+0000 7f2146ffd640 1 --2- 172.21.15.94:0/924583102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2148072b60 0x7f214815c820 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f21380097c0 tx=0x7f213801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=504f2c10ba70873e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:03.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.819+0000 7f21467fc640 1 -- 172.21.15.94:0/924583102 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21380b2070 con 0x7f2148072b60 2024-04-25T16:52:03.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.819+0000 7f21467fc640 1 -- 172.21.15.94:0/924583102 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21380a9070 con 0x7f2148072b60 2024-04-25T16:52:03.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.819+0000 7f21467fc640 1 -- 172.21.15.94:0/924583102 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21380ad880 con 0x7f2148072b60 2024-04-25T16:52:03.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.819+0000 7f214da46640 1 -- 172.21.15.94:0/924583102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2148072b60 msgr2=0x7f214815c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.819+0000 7f214da46640 1 --2- 172.21.15.94:0/924583102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2148072b60 0x7f214815c820 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f21380097c0 tx=0x7f213801c3e0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.819+0000 7f214da46640 1 -- 172.21.15.94:0/924583102 shutdown_connections 2024-04-25T16:52:03.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.819+0000 7f214da46640 1 --2- 172.21.15.94:0/924583102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f214815cd60 0x7f214815fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.819+0000 7f214da46640 1 --2- 172.21.15.94:0/924583102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2148072b60 0x7f214815c820 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.819+0000 7f214da46640 1 -- 172.21.15.94:0/924583102 >> 172.21.15.94:0/924583102 conn(0x7f21480b47f0 msgr2=0x7f21480b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:03.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.819+0000 7f214da46640 1 -- 172.21.15.94:0/924583102 shutdown_connections 2024-04-25T16:52:03.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.819+0000 7f214da46640 1 -- 172.21.15.94:0/924583102 wait complete. 2024-04-25T16:52:03.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f214da46640 1 Processor -- start 2024-04-25T16:52:03.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f214da46640 1 -- start start 2024-04-25T16:52:03.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f214da46640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2148072b60 0x7f21480b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f214da46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f214815cd60 0x7f21481fba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f214da46640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2148160020 0x7f21481f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f214da46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f21480730f0 con 0x7f214815cd60 2024-04-25T16:52:03.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f214da46640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2148072f70 con 0x7f2148160020 2024-04-25T16:52:03.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f214da46640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2148159cc0 con 0x7f2148072b60 2024-04-25T16:52:03.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f213e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f214815cd60 0x7f21481fba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f213e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f214815cd60 0x7f21481fba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54852/0 (socket says 172.21.15.94:54852) 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f213e7fc640 1 -- 172.21.15.94:0/2467805182 learned_addr learned my addr 172.21.15.94:0/2467805182 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f2146ffd640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2148072b60 0x7f21480b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f21477fe640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2148160020 0x7f21481f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f213e7fc640 1 -- 172.21.15.94:0/2467805182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2148072b60 msgr2=0x7f21480b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f213e7fc640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2148072b60 0x7f21480b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f213e7fc640 1 -- 172.21.15.94:0/2467805182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2148160020 msgr2=0x7f21481f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f213e7fc640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2148160020 0x7f21481f7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f213e7fc640 1 -- 172.21.15.94:0/2467805182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21481f81a0 con 0x7f214815cd60 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f213e7fc640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f214815cd60 0x7f21481fba30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f213000e940 tx=0x7f213000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f2144ff9640 1 -- 172.21.15.94:0/2467805182 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130016070 con 0x7f214815cd60 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f2144ff9640 1 -- 172.21.15.94:0/2467805182 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f213000cd00 con 0x7f214815cd60 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f214da46640 1 -- 172.21.15.94:0/2467805182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21481f8480 con 0x7f214815cd60 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f2144ff9640 1 -- 172.21.15.94:0/2467805182 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213000b040 con 0x7f214815cd60 2024-04-25T16:52:03.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f214da46640 1 -- 172.21.15.94:0/2467805182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21480b0de0 con 0x7f214815cd60 2024-04-25T16:52:03.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.823+0000 7f214da46640 1 -- 172.21.15.94:0/2467805182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2148076830 con 0x7f214815cd60 2024-04-25T16:52:03.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.827+0000 7f2144ff9640 1 -- 172.21.15.94:0/2467805182 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2130019070 con 0x7f214815cd60 2024-04-25T16:52:03.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.827+0000 7f2144ff9640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f212806f400 0x7f2128071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:03.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.827+0000 7f2144ff9640 1 -- 172.21.15.94:0/2467805182 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f213008f5c0 con 0x7f214815cd60 2024-04-25T16:52:03.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.827+0000 7f2146ffd640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f212806f400 0x7f2128071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:03.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.827+0000 7f2146ffd640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f212806f400 0x7f2128071910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f21380af000 tx=0x7f2138008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:03.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.831+0000 7f2144ff9640 1 -- 172.21.15.94:0/2467805182 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f213005c0b0 con 0x7f214815cd60 2024-04-25T16:52:03.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.999+0000 7f214da46640 1 -- 172.21.15.94:0/2467805182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2148167c60 con 0x7f214815cd60 2024-04-25T16:52:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:03.999+0000 7f2144ff9640 1 -- 172.21.15.94:0/2467805182 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f213005b9b0 con 0x7f214815cd60 2024-04-25T16:52:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:52:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.003+0000 7f214da46640 1 -- 172.21.15.94:0/2467805182 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f212806f400 msgr2=0x7f2128071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.003+0000 7f214da46640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f212806f400 0x7f2128071910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f21380af000 tx=0x7f2138008040 comp rx=0 tx=0).stop 2024-04-25T16:52:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.003+0000 7f214da46640 1 -- 172.21.15.94:0/2467805182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f214815cd60 msgr2=0x7f21481fba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.003+0000 7f214da46640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f214815cd60 0x7f21481fba30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f213000e940 tx=0x7f213000ee00 comp rx=0 tx=0).stop 2024-04-25T16:52:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.003+0000 7f214da46640 1 -- 172.21.15.94:0/2467805182 shutdown_connections 2024-04-25T16:52:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.003+0000 7f214da46640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2148160020 0x7f21481f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.003+0000 7f214da46640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f214815cd60 0x7f21481fba30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.003+0000 7f214da46640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f212806f400 0x7f2128071910 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.003+0000 7f214da46640 1 --2- 172.21.15.94:0/2467805182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2148072b60 0x7f21480b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.003+0000 7f214da46640 1 -- 172.21.15.94:0/2467805182 >> 172.21.15.94:0/2467805182 conn(0x7f21480b47f0 msgr2=0x7f21480af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.003+0000 7f214da46640 1 -- 172.21.15.94:0/2467805182 shutdown_connections 2024-04-25T16:52:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.003+0000 7f214da46640 1 -- 172.21.15.94:0/2467805182 wait complete. 2024-04-25T16:52:04.020 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:52:04.020 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:52:04.025 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:52:04.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.215+0000 7f2b1495a640 1 Processor -- start 2024-04-25T16:52:04.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.215+0000 7f2b1495a640 1 -- start start 2024-04-25T16:52:04.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.215+0000 7f2b1495a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2b1006de20 0x7f2b100af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.215+0000 7f2b1495a640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b10161300 con 0x7f2b1015cf40 2024-04-25T16:52:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.215+0000 7f2b1495a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b1015d350 con 0x7f2b10070180 2024-04-25T16:52:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.215+0000 7f2b1495a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2b1015dec0 con 0x7f2b1006de20 2024-04-25T16:52:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.215+0000 7f2b0dd9b640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f2b1015cf40 0x7f2b1006d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38842/0 (socket says 172.21.15.94:38842) 2024-04-25T16:52:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.215+0000 7f2b0dd9b640 1 -- 172.21.15.94:0/25735132 learned_addr learned my addr 172.21.15.94:0/25735132 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.215+0000 7f2b0ed9d640 1 --2- 172.21.15.94:0/25735132 >> v2:172.21.15.138:3300/0 conn(0x7f2b1006de20 0x7f2b100af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.215+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3587190715 0 0) 0x7f2b10161300 con 0x7f2b1015cf40 2024-04-25T16:52:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2af8003530 con 0x7f2b1015cf40 2024-04-25T16:52:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4020299353 0 0) 0x7f2af8003530 con 0x7f2b1015cf40 2024-04-25T16:52:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 >> v2:172.21.15.138:3300/0 conn(0x7f2b1006de20 msgr2=0x7f2b100af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 --2- 172.21.15.94:0/25735132 >> v2:172.21.15.138:3300/0 conn(0x7f2b1006de20 0x7f2b100af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 >> v1:172.21.15.138:6789/0 conn(0x7f2b10070180 legacy=0x7f2b10160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1015d620 con 0x7f2b1015cf40 2024-04-25T16:52:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f2b04001680 con 0x7f2b1015cf40 2024-04-25T16:52:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 >> v1:172.21.15.94:6789/0 conn(0x7f2b1015cf40 legacy=0x7f2b1006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 --2- 172.21.15.94:0/25735132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af80039c0 0x7f2af8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 --2- 172.21.15.94:0/25735132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af8004810 0x7f2af8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 --2- 172.21.15.94:0/25735132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af80051d0 0x7f2af800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1015dec0 con 0x7f2af8004810 2024-04-25T16:52:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2af800c170 con 0x7f2af80051d0 2024-04-25T16:52:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2af800c370 con 0x7f2af80039c0 2024-04-25T16:52:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0dd9b640 1 --2- 172.21.15.94:0/25735132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af8004810 0x7f2af8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0ed9d640 1 --2- 172.21.15.94:0/25735132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af80051d0 0x7f2af800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0dd9b640 1 -- 172.21.15.94:0/25735132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af80039c0 msgr2=0x7f2af8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0e59c640 1 --2- 172.21.15.94:0/25735132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af80039c0 0x7f2af8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0dd9b640 1 --2- 172.21.15.94:0/25735132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af80039c0 0x7f2af8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0dd9b640 1 -- 172.21.15.94:0/25735132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af80051d0 msgr2=0x7f2af800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:04.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0dd9b640 1 --2- 172.21.15.94:0/25735132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af80051d0 0x7f2af800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0dd9b640 1 -- 172.21.15.94:0/25735132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af8003530 con 0x7f2af8004810 2024-04-25T16:52:04.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0ed9d640 1 --2- 172.21.15.94:0/25735132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af80051d0 0x7f2af800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:04.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0ed9d640 1 -- 172.21.15.94:0/25735132 reap_dead start 2024-04-25T16:52:04.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0e59c640 1 --2- 172.21.15.94:0/25735132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2af80039c0 0x7f2af8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0dd9b640 1 --2- 172.21.15.94:0/25735132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af8004810 0x7f2af8004c20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400c210 tx=0x7f2b0401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a436b2e177258e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04007d40 con 0x7f2af8004810 2024-04-25T16:52:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b04005b00 con 0x7f2af8004810 2024-04-25T16:52:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b0d59a640 1 -- 172.21.15.94:0/25735132 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04007070 con 0x7f2af8004810 2024-04-25T16:52:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b1495a640 1 -- 172.21.15.94:0/25735132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af8004810 msgr2=0x7f2af8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b1495a640 1 --2- 172.21.15.94:0/25735132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af8004810 0x7f2af8004c20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400c210 tx=0x7f2b0401f3b0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b1495a640 1 -- 172.21.15.94:0/25735132 shutdown_connections 2024-04-25T16:52:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b1495a640 1 --2- 172.21.15.94:0/25735132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af8004810 0x7f2af8004c20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.219+0000 7f2b1495a640 1 -- 172.21.15.94:0/25735132 >> 172.21.15.94:0/25735132 conn(0x7f2b100b47f0 msgr2=0x7f2b100b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b1495a640 1 -- 172.21.15.94:0/25735132 shutdown_connections 2024-04-25T16:52:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b1495a640 1 -- 172.21.15.94:0/25735132 wait complete. 2024-04-25T16:52:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b1495a640 1 Processor -- start 2024-04-25T16:52:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b1495a640 1 -- start start 2024-04-25T16:52:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b1495a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8004810 0x7f2b101f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b1495a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b100ca470 0x7f2b100cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b1495a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b1015f600 0x7f2b101f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b1495a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1015dbe0 con 0x7f2b100ca470 2024-04-25T16:52:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b1495a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b10160760 con 0x7f2af8004810 2024-04-25T16:52:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b1495a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2b1006fea0 con 0x7f2b1015f600 2024-04-25T16:52:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b0dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b100ca470 0x7f2b100cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b0dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b100ca470 0x7f2b100cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54862/0 (socket says 172.21.15.94:54862) 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b0dd9b640 1 -- 172.21.15.94:0/2422613245 learned_addr learned my addr 172.21.15.94:0/2422613245 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b0ed9d640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b1015f600 0x7f2b101f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b0e59c640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8004810 0x7f2b101f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b0dd9b640 1 -- 172.21.15.94:0/2422613245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b1015f600 msgr2=0x7f2b101f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b0dd9b640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b1015f600 0x7f2b101f6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b0dd9b640 1 -- 172.21.15.94:0/2422613245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8004810 msgr2=0x7f2b101f74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b0dd9b640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8004810 0x7f2b101f74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b0dd9b640 1 -- 172.21.15.94:0/2422613245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b10073bb0 con 0x7f2b100ca470 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b0ed9d640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b1015f600 0x7f2b101f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.223+0000 7f2b0e59c640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8004810 0x7f2b101f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.227+0000 7f2b0dd9b640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b100ca470 0x7f2b100cc450 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2b04001980 tx=0x7f2af8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.227+0000 7f2afeffd640 1 -- 172.21.15.94:0/2422613245 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b040049f0 con 0x7f2b100ca470 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.227+0000 7f2afeffd640 1 -- 172.21.15.94:0/2422613245 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b040b6070 con 0x7f2b100ca470 2024-04-25T16:52:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.227+0000 7f2afeffd640 1 -- 172.21.15.94:0/2422613245 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04004d90 con 0x7f2b100ca470 2024-04-25T16:52:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.227+0000 7f2b1495a640 1 -- 172.21.15.94:0/2422613245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b100cabc0 con 0x7f2b100ca470 2024-04-25T16:52:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.227+0000 7f2b1495a640 1 -- 172.21.15.94:0/2422613245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b100cb180 con 0x7f2b100ca470 2024-04-25T16:52:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.227+0000 7f2b1495a640 1 -- 172.21.15.94:0/2422613245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b101f6310 con 0x7f2b100ca470 2024-04-25T16:52:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.227+0000 7f2afeffd640 1 -- 172.21.15.94:0/2422613245 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2b04007070 con 0x7f2b100ca470 2024-04-25T16:52:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.227+0000 7f2afeffd640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2af006f430 0x7f2af0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.227+0000 7f2afeffd640 1 -- 172.21.15.94:0/2422613245 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2b0412c600 con 0x7f2b100ca470 2024-04-25T16:52:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.227+0000 7f2b0e59c640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2af006f430 0x7f2af0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.227+0000 7f2b0e59c640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2af006f430 0x7f2af0071940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2b00000c10 tx=0x7f2b00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:04.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.235+0000 7f2afeffd640 1 -- 172.21.15.94:0/2422613245 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2b040c2030 con 0x7f2b100ca470 2024-04-25T16:52:04.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.395+0000 7f2b1495a640 1 -- 172.21.15.94:0/2422613245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b10076830 con 0x7f2b100ca470 2024-04-25T16:52:04.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.399+0000 7f2afeffd640 1 -- 172.21.15.94:0/2422613245 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7f2b040190c0 con 0x7f2b100ca470 2024-04-25T16:52:04.397 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:04.397 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:52:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.399+0000 7f2b1495a640 1 -- 172.21.15.94:0/2422613245 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2af006f430 msgr2=0x7f2af0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.399+0000 7f2b1495a640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2af006f430 0x7f2af0071940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2b00000c10 tx=0x7f2b00009040 comp rx=0 tx=0).stop 2024-04-25T16:52:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.399+0000 7f2b1495a640 1 -- 172.21.15.94:0/2422613245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b100ca470 msgr2=0x7f2b100cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.399+0000 7f2b1495a640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b100ca470 0x7f2b100cc450 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2b04001980 tx=0x7f2af8005840 comp rx=0 tx=0).stop 2024-04-25T16:52:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.399+0000 7f2b1495a640 1 -- 172.21.15.94:0/2422613245 shutdown_connections 2024-04-25T16:52:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.399+0000 7f2b1495a640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2af006f430 0x7f2af0071940 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.399+0000 7f2b1495a640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b1015f600 0x7f2b101f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.399+0000 7f2b1495a640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b100ca470 0x7f2b100cc450 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.399+0000 7f2b1495a640 1 --2- 172.21.15.94:0/2422613245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8004810 0x7f2b101f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.399+0000 7f2b1495a640 1 -- 172.21.15.94:0/2422613245 >> 172.21.15.94:0/2422613245 conn(0x7f2b100b47f0 msgr2=0x7f2b10160bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.399+0000 7f2b1495a640 1 -- 172.21.15.94:0/2422613245 shutdown_connections 2024-04-25T16:52:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.399+0000 7f2b1495a640 1 -- 172.21.15.94:0/2422613245 wait complete. 2024-04-25T16:52:04.419 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:52:04.420 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:52:04.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f33630c2640 1 Processor -- start 2024-04-25T16:52:04.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f33630c2640 1 -- start start 2024-04-25T16:52:04.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f33630c2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c15c7e0 0x7f335c0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f33630c2640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f335c074380 con 0x7f335c0b0330 2024-04-25T16:52:04.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f33630c2640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f335c0727e0 con 0x7f335c06de20 2024-04-25T16:52:04.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f33630c2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f335c072ac0 con 0x7f335c15c7e0 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335bfff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f335c0b0330 0x7f335c06d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38846/0 (socket says 172.21.15.94:38846) 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335bfff640 1 -- 172.21.15.94:0/3568840759 learned_addr learned my addr 172.21.15.94:0/3568840759 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f3360e5e640 1 --2- 172.21.15.94:0/3568840759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c15c7e0 0x7f335c0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1945301254 0 0) 0x7f334c001120 con 0x7f335c0b0330 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3344003530 con 0x7f335c0b0330 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2637833012 0 0) 0x7f3344003530 con 0x7f335c0b0330 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c15c7e0 msgr2=0x7f335c0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 --2- 172.21.15.94:0/3568840759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c15c7e0 0x7f335c0afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 >> v1:172.21.15.138:6789/0 conn(0x7f335c06de20 legacy=0x7f335c06e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335c072da0 con 0x7f335c0b0330 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f334c001850 con 0x7f335c0b0330 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 >> v1:172.21.15.94:6789/0 conn(0x7f335c0b0330 legacy=0x7f335c06d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 --2- 172.21.15.94:0/3568840759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c15c7e0 0x7f33440039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 --2- 172.21.15.94:0/3568840759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3344003f70 0x7f3344004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 --2- 172.21.15.94:0/3568840759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344004d50 0x7f3344005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f335c072ac0 con 0x7f3344004d50 2024-04-25T16:52:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f334400bb60 con 0x7f3344003f70 2024-04-25T16:52:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f334400bd60 con 0x7f335c15c7e0 2024-04-25T16:52:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f336165f640 1 --2- 172.21.15.94:0/3568840759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344004d50 0x7f3344005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f335bfff640 1 --2- 172.21.15.94:0/3568840759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3344003f70 0x7f3344004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f3360e5e640 1 --2- 172.21.15.94:0/3568840759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c15c7e0 0x7f33440039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f336165f640 1 -- 172.21.15.94:0/3568840759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c15c7e0 msgr2=0x7f33440039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f336165f640 1 --2- 172.21.15.94:0/3568840759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c15c7e0 0x7f33440039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f336165f640 1 -- 172.21.15.94:0/3568840759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3344003f70 msgr2=0x7f3344004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f336165f640 1 --2- 172.21.15.94:0/3568840759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3344003f70 0x7f3344004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.571+0000 7f336165f640 1 -- 172.21.15.94:0/3568840759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3344003530 con 0x7f3344004d50 2024-04-25T16:52:04.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.575+0000 7f336165f640 1 --2- 172.21.15.94:0/3568840759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344004d50 0x7f3344005160 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3354001e40 tx=0x7f335401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b64db2a4787c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:04.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.575+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354001a90 con 0x7f3344004d50 2024-04-25T16:52:04.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.575+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3354003b80 con 0x7f3344004d50 2024-04-25T16:52:04.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.575+0000 7f335b7fe640 1 -- 172.21.15.94:0/3568840759 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354005080 con 0x7f3344004d50 2024-04-25T16:52:04.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.575+0000 7f33630c2640 1 -- 172.21.15.94:0/3568840759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344004d50 msgr2=0x7f3344005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.575+0000 7f33630c2640 1 --2- 172.21.15.94:0/3568840759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344004d50 0x7f3344005160 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3354001e40 tx=0x7f335401c760 comp rx=0 tx=0).stop 2024-04-25T16:52:04.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.575+0000 7f336165f640 1 -- 172.21.15.94:0/3568840759 reap_dead start 2024-04-25T16:52:04.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.575+0000 7f33630c2640 1 -- 172.21.15.94:0/3568840759 shutdown_connections 2024-04-25T16:52:04.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.575+0000 7f33630c2640 1 -- 172.21.15.94:0/3568840759 >> 172.21.15.94:0/3568840759 conn(0x7f335c0b47f0 msgr2=0x7f335c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:04.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.575+0000 7f33630c2640 1 -- 172.21.15.94:0/3568840759 shutdown_connections 2024-04-25T16:52:04.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.575+0000 7f33630c2640 1 -- 172.21.15.94:0/3568840759 wait complete. 2024-04-25T16:52:04.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33630c2640 1 Processor -- start 2024-04-25T16:52:04.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33630c2640 1 -- start start 2024-04-25T16:52:04.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33630c2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c150c70 0x7f335c1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33630c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f335c157780 0x7f335c157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33630c2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f335c06c600 0x7f335c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33630c2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f335c073ca0 con 0x7f335c157780 2024-04-25T16:52:04.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33630c2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f335c073360 con 0x7f335c06c600 2024-04-25T16:52:04.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33630c2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f335c15cbf0 con 0x7f335c150c70 2024-04-25T16:52:04.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f335bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f335c157780 0x7f335c157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f335bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f335c157780 0x7f335c157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54886/0 (socket says 172.21.15.94:54886) 2024-04-25T16:52:04.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f335bfff640 1 -- 172.21.15.94:0/2592031094 learned_addr learned my addr 172.21.15.94:0/2592031094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:04.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f336165f640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f335c06c600 0x7f335c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f3360e5e640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c150c70 0x7f335c1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f335bfff640 1 -- 172.21.15.94:0/2592031094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c150c70 msgr2=0x7f335c1592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f335bfff640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c150c70 0x7f335c1592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f335bfff640 1 -- 172.21.15.94:0/2592031094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f335c06c600 msgr2=0x7f335c06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f335bfff640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f335c06c600 0x7f335c06ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f335bfff640 1 -- 172.21.15.94:0/2592031094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335c1580d0 con 0x7f335c157780 2024-04-25T16:52:04.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f3360e5e640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c150c70 0x7f335c1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:04.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f335bfff640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f335c157780 0x7f335c157b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f334c005f80 tx=0x7f334c0074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:04.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33597fa640 1 -- 172.21.15.94:0/2592031094 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c007730 con 0x7f335c157780 2024-04-25T16:52:04.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33597fa640 1 -- 172.21.15.94:0/2592031094 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f334c018070 con 0x7f335c157780 2024-04-25T16:52:04.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33630c2640 1 -- 172.21.15.94:0/2592031094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335c1f60a0 con 0x7f335c157780 2024-04-25T16:52:04.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33597fa640 1 -- 172.21.15.94:0/2592031094 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c00eb90 con 0x7f335c157780 2024-04-25T16:52:04.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33630c2640 1 -- 172.21.15.94:0/2592031094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f335c1f6640 con 0x7f335c157780 2024-04-25T16:52:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.579+0000 7f33630c2640 1 -- 172.21.15.94:0/2592031094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f335c076830 con 0x7f335c157780 2024-04-25T16:52:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.583+0000 7f33597fa640 1 -- 172.21.15.94:0/2592031094 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f334c01d460 con 0x7f335c157780 2024-04-25T16:52:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.583+0000 7f33597fa640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f334806f400 0x7f3348071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.583+0000 7f33597fa640 1 -- 172.21.15.94:0/2592031094 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f334c08f6c0 con 0x7f335c157780 2024-04-25T16:52:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.583+0000 7f3360e5e640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f334806f400 0x7f3348071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.583+0000 7f3360e5e640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f334806f400 0x7f3348071910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f335c1e7c10 tx=0x7f3350008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.587+0000 7f33597fa640 1 -- 172.21.15.94:0/2592031094 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f335c1580d0 con 0x7f335c157780 2024-04-25T16:52:04.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.735+0000 7f33630c2640 1 -- 172.21.15.94:0/2592031094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f335c151de0 con 0x7f335c157780 2024-04-25T16:52:04.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.735+0000 7f33597fa640 1 -- 172.21.15.94:0/2592031094 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f335c074380 con 0x7f335c157780 2024-04-25T16:52:04.735 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:52:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.739+0000 7f33630c2640 1 -- 172.21.15.94:0/2592031094 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f334806f400 msgr2=0x7f3348071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.739+0000 7f33630c2640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f334806f400 0x7f3348071910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f335c1e7c10 tx=0x7f3350008040 comp rx=0 tx=0).stop 2024-04-25T16:52:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.739+0000 7f33630c2640 1 -- 172.21.15.94:0/2592031094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f335c157780 msgr2=0x7f335c157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.739+0000 7f33630c2640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f335c157780 0x7f335c157b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f334c005f80 tx=0x7f334c0074e0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.739+0000 7f33630c2640 1 -- 172.21.15.94:0/2592031094 shutdown_connections 2024-04-25T16:52:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.739+0000 7f33630c2640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f335c06c600 0x7f335c06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.739+0000 7f33630c2640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f335c157780 0x7f335c157b90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.739+0000 7f33630c2640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f334806f400 0x7f3348071910 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.739+0000 7f33630c2640 1 --2- 172.21.15.94:0/2592031094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c150c70 0x7f335c1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.739+0000 7f33630c2640 1 -- 172.21.15.94:0/2592031094 >> 172.21.15.94:0/2592031094 conn(0x7f335c0b47f0 msgr2=0x7f335c0b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.739+0000 7f33630c2640 1 -- 172.21.15.94:0/2592031094 shutdown_connections 2024-04-25T16:52:04.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.739+0000 7f33630c2640 1 -- 172.21.15.94:0/2592031094 wait complete. 2024-04-25T16:52:04.756 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:52:04.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f64a5fa7640 1 Processor -- start 2024-04-25T16:52:04.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f64a5fa7640 1 -- start start 2024-04-25T16:52:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f64a5fa7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a015c7e0 0x7f64a00afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f64a5fa7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f64a00b0330 0x7f64a006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f64a5fa7640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64a0074380 con 0x7f64a006dc50 2024-04-25T16:52:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f64a5fa7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f64a006e060 con 0x7f64a00b0330 2024-04-25T16:52:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f64a5fa7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f64a00727e0 con 0x7f64a015c7e0 2024-04-25T16:52:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649ffff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f64a006dc50 0x7f64a006e7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38852/0 (socket says 172.21.15.94:38852) 2024-04-25T16:52:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649ffff640 1 -- 172.21.15.94:0/1984342325 learned_addr learned my addr 172.21.15.94:0/1984342325 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649f7fe640 1 --2- 172.21.15.94:0/1984342325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a015c7e0 0x7f64a00afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649effd640 1 --2- 172.21.15.94:0/1984342325 >> v2:172.21.15.138:3300/0 conn(0x7f64a00b0330 0x7f64a006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 203493444 0 0) 0x7f64980014a0 con 0x7f64a006dc50 2024-04-25T16:52:04.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6488003530 con 0x7f64a006dc50 2024-04-25T16:52:04.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 409347613 0 0) 0x7f6488003530 con 0x7f64a006dc50 2024-04-25T16:52:04.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a015c7e0 msgr2=0x7f64a00afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 --2- 172.21.15.94:0/1984342325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a015c7e0 0x7f64a00afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 >> v2:172.21.15.138:3300/0 conn(0x7f64a00b0330 msgr2=0x7f64a006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 --2- 172.21.15.94:0/1984342325 >> v2:172.21.15.138:3300/0 conn(0x7f64a00b0330 0x7f64a006d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a0072ac0 con 0x7f64a006dc50 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f6498001bd0 con 0x7f64a006dc50 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 >> v1:172.21.15.94:6789/0 conn(0x7f64a006dc50 legacy=0x7f64a006e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 --2- 172.21.15.94:0/1984342325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64880039c0 0x7f6488003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 --2- 172.21.15.94:0/1984342325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6488004810 0x7f6488004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 --2- 172.21.15.94:0/1984342325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a015c7e0 0x7f6488005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64a006e060 con 0x7f6488004810 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64a00727e0 con 0x7f64880039c0 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f648800b9d0 con 0x7f64a015c7e0 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649effd640 1 --2- 172.21.15.94:0/1984342325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6488004810 0x7f6488004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649ffff640 1 --2- 172.21.15.94:0/1984342325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a015c7e0 0x7f6488005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.911+0000 7f649f7fe640 1 --2- 172.21.15.94:0/1984342325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64880039c0 0x7f6488003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f649effd640 1 -- 172.21.15.94:0/1984342325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a015c7e0 msgr2=0x7f6488005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:04.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f649effd640 1 --2- 172.21.15.94:0/1984342325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a015c7e0 0x7f6488005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f649effd640 1 -- 172.21.15.94:0/1984342325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64880039c0 msgr2=0x7f6488003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:04.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f649effd640 1 --2- 172.21.15.94:0/1984342325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64880039c0 0x7f6488003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f649effd640 1 -- 172.21.15.94:0/1984342325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6488003530 con 0x7f6488004810 2024-04-25T16:52:04.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f649ffff640 1 --2- 172.21.15.94:0/1984342325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a015c7e0 0x7f6488005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:04.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f649effd640 1 --2- 172.21.15.94:0/1984342325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6488004810 0x7f6488004c20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f64a00b5020 tx=0x7f649401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4658f189dbfae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:04.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940b2070 con 0x7f6488004810 2024-04-25T16:52:04.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64940a5640 con 0x7f6488004810 2024-04-25T16:52:04.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f649e7fc640 1 -- 172.21.15.94:0/1984342325 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940096b0 con 0x7f6488004810 2024-04-25T16:52:04.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1984342325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6488004810 msgr2=0x7f6488004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f64a5fa7640 1 --2- 172.21.15.94:0/1984342325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6488004810 0x7f6488004c20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f64a00b5020 tx=0x7f649401ca50 comp rx=0 tx=0).stop 2024-04-25T16:52:04.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f649ffff640 1 -- 172.21.15.94:0/1984342325 reap_dead start 2024-04-25T16:52:04.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1984342325 shutdown_connections 2024-04-25T16:52:04.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1984342325 >> 172.21.15.94:0/1984342325 conn(0x7f64a00b47f0 msgr2=0x7f64a00b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:04.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1984342325 shutdown_connections 2024-04-25T16:52:04.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.915+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1984342325 wait complete. 2024-04-25T16:52:04.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f64a5fa7640 1 Processor -- start 2024-04-25T16:52:04.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f64a5fa7640 1 -- start start 2024-04-25T16:52:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f64a5fa7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a014c2d0 0x7f64a014c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f64a5fa7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a01e62a0 0x7f64a01e66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f64a5fa7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a01e6bf0 0x7f64a015d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f64a5fa7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0073360 con 0x7f64a01e62a0 2024-04-25T16:52:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f64a5fa7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0073080 con 0x7f64a01e6bf0 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f64a5fa7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f64a0073ca0 con 0x7f64a014c2d0 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f649effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a01e62a0 0x7f64a01e66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f649effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a01e62a0 0x7f64a01e66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54904/0 (socket says 172.21.15.94:54904) 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f649effd640 1 -- 172.21.15.94:0/1693234572 learned_addr learned my addr 172.21.15.94:0/1693234572 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f649effd640 1 -- 172.21.15.94:0/1693234572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a014c2d0 msgr2=0x7f64a014c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f649f7fe640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a014c2d0 0x7f64a014c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f649ffff640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a01e6bf0 0x7f64a015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f649effd640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a014c2d0 0x7f64a014c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f649effd640 1 -- 172.21.15.94:0/1693234572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a01e6bf0 msgr2=0x7f64a015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f649effd640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a01e6bf0 0x7f64a015d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f649effd640 1 -- 172.21.15.94:0/1693234572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a0152dc0 con 0x7f64a01e62a0 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f649effd640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a01e62a0 0x7f64a01e66b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f64940b0040 tx=0x7f64940197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f6487fff640 1 -- 172.21.15.94:0/1693234572 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940bc020 con 0x7f64a01e62a0 2024-04-25T16:52:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f6487fff640 1 -- 172.21.15.94:0/1693234572 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6494004020 con 0x7f64a01e62a0 2024-04-25T16:52:04.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1693234572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a014d090 con 0x7f64a01e62a0 2024-04-25T16:52:04.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f6487fff640 1 -- 172.21.15.94:0/1693234572 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940b2040 con 0x7f64a01e62a0 2024-04-25T16:52:04.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.919+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1693234572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a015cd20 con 0x7f64a01e62a0 2024-04-25T16:52:04.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.923+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1693234572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a0076830 con 0x7f64a01e62a0 2024-04-25T16:52:04.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.923+0000 7f6487fff640 1 -- 172.21.15.94:0/1693234572 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6494003620 con 0x7f64a01e62a0 2024-04-25T16:52:04.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.923+0000 7f6487fff640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f648c06f400 0x7f648c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:04.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.923+0000 7f6487fff640 1 -- 172.21.15.94:0/1693234572 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6494129610 con 0x7f64a01e62a0 2024-04-25T16:52:04.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.923+0000 7f649f7fe640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f648c06f400 0x7f648c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:04.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.923+0000 7f649f7fe640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f648c06f400 0x7f648c071910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6490009160 tx=0x7f6490008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:04.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:04.927+0000 7f6487fff640 1 -- 172.21.15.94:0/1693234572 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6494016760 con 0x7f64a01e62a0 2024-04-25T16:52:05.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.079+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1693234572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f64a00b1100 con 0x7f64a01e62a0 2024-04-25T16:52:05.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.079+0000 7f6487fff640 1 -- 172.21.15.94:0/1693234572 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f64940f6300 con 0x7f64a01e62a0 2024-04-25T16:52:05.078 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:52:05.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.083+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1693234572 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f648c06f400 msgr2=0x7f648c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.083+0000 7f64a5fa7640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f648c06f400 0x7f648c071910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6490009160 tx=0x7f6490008040 comp rx=0 tx=0).stop 2024-04-25T16:52:05.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.083+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1693234572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a01e62a0 msgr2=0x7f64a01e66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.083+0000 7f64a5fa7640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a01e62a0 0x7f64a01e66b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f64940b0040 tx=0x7f64940197f0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.083+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1693234572 shutdown_connections 2024-04-25T16:52:05.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.083+0000 7f64a5fa7640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a01e6bf0 0x7f64a015d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.083+0000 7f64a5fa7640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a01e62a0 0x7f64a01e66b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.083+0000 7f64a5fa7640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f648c06f400 0x7f648c071910 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.083+0000 7f64a5fa7640 1 --2- 172.21.15.94:0/1693234572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64a014c2d0 0x7f64a014c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.083+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1693234572 >> 172.21.15.94:0/1693234572 conn(0x7f64a00b47f0 msgr2=0x7f64a00afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:05.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.083+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1693234572 shutdown_connections 2024-04-25T16:52:05.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.083+0000 7f64a5fa7640 1 -- 172.21.15.94:0/1693234572 wait complete. 2024-04-25T16:52:05.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:52:05.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f193739c640 1 Processor -- start 2024-04-25T16:52:05.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f193739c640 1 -- start start 2024-04-25T16:52:05.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f193739c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19300b0420 0x7f193006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f193739c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1930070630 con 0x7f193006f2a0 2024-04-25T16:52:05.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f193739c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193006f6b0 con 0x7f193015ec30 2024-04-25T16:52:05.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f193739c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f19300aeaa0 con 0x7f19300b0420 2024-04-25T16:52:05.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1934937640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f193006f2a0 0x7f193006c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38860/0 (socket says 172.21.15.94:38860) 2024-04-25T16:52:05.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1934937640 1 -- 172.21.15.94:0/3175395461 learned_addr learned my addr 172.21.15.94:0/3175395461 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1935138640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19300b0420 0x7f193006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3857746469 0 0) 0x7f1930070630 con 0x7f193006f2a0 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f191c003530 con 0x7f193006f2a0 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2987322912 0 0) 0x7f191c003530 con 0x7f193006f2a0 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19300b0420 msgr2=0x7f193006ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19300b0420 0x7f193006ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 >> v1:172.21.15.138:6789/0 conn(0x7f193015ec30 legacy=0x7f193006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193006d420 con 0x7f193006f2a0 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f1920001680 con 0x7f193006f2a0 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 >> v1:172.21.15.94:6789/0 conn(0x7f193006f2a0 legacy=0x7f193006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f191c0039f0 0x7f191c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f191c004810 0x7f191c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19300b0420 0x7f191c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19300aeaa0 con 0x7f191c004810 2024-04-25T16:52:05.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f191c00bb00 con 0x7f191c0039f0 2024-04-25T16:52:05.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f191c00bd00 con 0x7f19300b0420 2024-04-25T16:52:05.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.255+0000 7f1934937640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f191c004810 0x7f191c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1935138640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f191c0039f0 0x7f191c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1935939640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19300b0420 0x7f191c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1934937640 1 -- 172.21.15.94:0/3175395461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19300b0420 msgr2=0x7f191c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:05.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1934937640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19300b0420 0x7f191c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1934937640 1 -- 172.21.15.94:0/3175395461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f191c0039f0 msgr2=0x7f191c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:05.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1934937640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f191c0039f0 0x7f191c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1934937640 1 -- 172.21.15.94:0/3175395461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c003530 con 0x7f191c004810 2024-04-25T16:52:05.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1935939640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19300b0420 0x7f191c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:05.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1935138640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f191c0039f0 0x7f191c003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1934937640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f191c004810 0x7f191c004c20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f19300afeb0 tx=0x7f192001f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2585ebe062c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920007d00 con 0x7f191c004810 2024-04-25T16:52:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1920005b00 con 0x7f191c004810 2024-04-25T16:52:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1927fff640 1 -- 172.21.15.94:0/3175395461 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920007070 con 0x7f191c004810 2024-04-25T16:52:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f193739c640 1 -- 172.21.15.94:0/3175395461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f191c004810 msgr2=0x7f191c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f193739c640 1 --2- 172.21.15.94:0/3175395461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f191c004810 0x7f191c004c20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f19300afeb0 tx=0x7f192001f330 comp rx=0 tx=0).stop 2024-04-25T16:52:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f1935939640 1 -- 172.21.15.94:0/3175395461 reap_dead start 2024-04-25T16:52:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f193739c640 1 -- 172.21.15.94:0/3175395461 shutdown_connections 2024-04-25T16:52:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f193739c640 1 -- 172.21.15.94:0/3175395461 >> 172.21.15.94:0/3175395461 conn(0x7f19300b47f0 msgr2=0x7f19300b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f193739c640 1 -- 172.21.15.94:0/3175395461 shutdown_connections 2024-04-25T16:52:05.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.259+0000 7f193739c640 1 -- 172.21.15.94:0/3175395461 wait complete. 2024-04-25T16:52:05.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f193739c640 1 Processor -- start 2024-04-25T16:52:05.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f193739c640 1 -- start start 2024-04-25T16:52:05.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f193739c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f191c004810 0x7f19301f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f193739c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1930154f20 0x7f1930155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f193739c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1930157610 0x7f1930157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f193739c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19300ae450 con 0x7f1930154f20 2024-04-25T16:52:05.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f193739c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19300703e0 con 0x7f1930157610 2024-04-25T16:52:05.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f193739c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f19300ae730 con 0x7f191c004810 2024-04-25T16:52:05.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1934937640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1930154f20 0x7f1930155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1934937640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1930154f20 0x7f1930155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54920/0 (socket says 172.21.15.94:54920) 2024-04-25T16:52:05.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1934937640 1 -- 172.21.15.94:0/4019915742 learned_addr learned my addr 172.21.15.94:0/4019915742 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:05.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1935939640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1930157610 0x7f1930157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1934937640 1 -- 172.21.15.94:0/4019915742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f191c004810 msgr2=0x7f19301f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1935138640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f191c004810 0x7f19301f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1934937640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f191c004810 0x7f19301f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1934937640 1 -- 172.21.15.94:0/4019915742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1930157610 msgr2=0x7f1930157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1934937640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1930157610 0x7f1930157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1934937640 1 -- 172.21.15.94:0/4019915742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1930155c50 con 0x7f1930154f20 2024-04-25T16:52:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1934937640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1930154f20 0x7f1930155330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1920001980 tx=0x7f191c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1925ffb640 1 -- 172.21.15.94:0/4019915742 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920005cb0 con 0x7f1930154f20 2024-04-25T16:52:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1925ffb640 1 -- 172.21.15.94:0/4019915742 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1920005e50 con 0x7f1930154f20 2024-04-25T16:52:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f193739c640 1 -- 172.21.15.94:0/4019915742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1930155f30 con 0x7f1930154f20 2024-04-25T16:52:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f1925ffb640 1 -- 172.21.15.94:0/4019915742 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19200049f0 con 0x7f1930154f20 2024-04-25T16:52:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.263+0000 7f193739c640 1 -- 172.21.15.94:0/4019915742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19301564f0 con 0x7f1930154f20 2024-04-25T16:52:05.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.267+0000 7f193739c640 1 -- 172.21.15.94:0/4019915742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1930076830 con 0x7f1930154f20 2024-04-25T16:52:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.267+0000 7f1925ffb640 1 -- 172.21.15.94:0/4019915742 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f19200ba030 con 0x7f1930154f20 2024-04-25T16:52:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.267+0000 7f1925ffb640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f191406f400 0x7f1914071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.267+0000 7f1925ffb640 1 -- 172.21.15.94:0/4019915742 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f192012c260 con 0x7f1930154f20 2024-04-25T16:52:05.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.267+0000 7f1935138640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f191406f400 0x7f1914071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.267+0000 7f1935138640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f191406f400 0x7f1914071910 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f19300cd2f0 tx=0x7f192c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:05.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.271+0000 7f1925ffb640 1 -- 172.21.15.94:0/4019915742 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1920019040 con 0x7f1930154f20 2024-04-25T16:52:05.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f193739c640 1 -- 172.21.15.94:0/4019915742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f19301559e0 con 0x7f1930154f20 2024-04-25T16:52:05.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f1925ffb640 1 -- 172.21.15.94:0/4019915742 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f19200f8de0 con 0x7f1930154f20 2024-04-25T16:52:05.433 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:52:05.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f193739c640 1 -- 172.21.15.94:0/4019915742 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f191406f400 msgr2=0x7f1914071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f193739c640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f191406f400 0x7f1914071910 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f19300cd2f0 tx=0x7f192c008040 comp rx=0 tx=0).stop 2024-04-25T16:52:05.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f193739c640 1 -- 172.21.15.94:0/4019915742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1930154f20 msgr2=0x7f1930155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f193739c640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1930154f20 0x7f1930155330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1920001980 tx=0x7f191c005840 comp rx=0 tx=0).stop 2024-04-25T16:52:05.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f193739c640 1 -- 172.21.15.94:0/4019915742 shutdown_connections 2024-04-25T16:52:05.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f193739c640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1930157610 0x7f1930157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f193739c640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1930154f20 0x7f1930155330 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f193739c640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f191406f400 0x7f1914071910 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f193739c640 1 --2- 172.21.15.94:0/4019915742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f191c004810 0x7f19301f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f193739c640 1 -- 172.21.15.94:0/4019915742 >> 172.21.15.94:0/4019915742 conn(0x7f19300b47f0 msgr2=0x7f1930160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:05.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f193739c640 1 -- 172.21.15.94:0/4019915742 shutdown_connections 2024-04-25T16:52:05.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.435+0000 7f193739c640 1 -- 172.21.15.94:0/4019915742 wait complete. 2024-04-25T16:52:05.455 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:52:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23e0fd5640 1 Processor -- start 2024-04-25T16:52:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23e0fd5640 1 -- start start 2024-04-25T16:52:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23e0fd5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc06e7a0 0x7f23dc06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23e0fd5640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23dc074380 con 0x7f23dc15c7e0 2024-04-25T16:52:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23e0fd5640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23dc0727e0 con 0x7f23dc06d710 2024-04-25T16:52:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23e0fd5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc072ac0 con 0x7f23dc06e7a0 2024-04-25T16:52:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23da59c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f23dc15c7e0 0x7f23dc0afdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38866/0 (socket says 172.21.15.94:38866) 2024-04-25T16:52:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23da59c640 1 -- 172.21.15.94:0/885557935 learned_addr learned my addr 172.21.15.94:0/885557935 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23dad9d640 1 --2- 172.21.15.94:0/885557935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc06e7a0 0x7f23dc06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3950443306 0 0) 0x7f23cc001120 con 0x7f23dc15c7e0 2024-04-25T16:52:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23c8003530 con 0x7f23dc15c7e0 2024-04-25T16:52:05.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4113507675 0 0) 0x7f23c8003530 con 0x7f23dc15c7e0 2024-04-25T16:52:05.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc06e7a0 msgr2=0x7f23dc06ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:05.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 --2- 172.21.15.94:0/885557935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc06e7a0 0x7f23dc06ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 >> v1:172.21.15.138:6789/0 conn(0x7f23dc06d710 legacy=0x7f23dc06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc072da0 con 0x7f23dc15c7e0 2024-04-25T16:52:05.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f23cc001850 con 0x7f23dc15c7e0 2024-04-25T16:52:05.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 >> v1:172.21.15.94:6789/0 conn(0x7f23dc15c7e0 legacy=0x7f23dc0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 --2- 172.21.15.94:0/885557935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c80039f0 0x7f23c8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 --2- 172.21.15.94:0/885557935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c8004810 0x7f23c8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 --2- 172.21.15.94:0/885557935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc06e7a0 0x7f23c80052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc072ac0 con 0x7f23c8004810 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23c800bb90 con 0x7f23c80039f0 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f23c800bd90 con 0x7f23dc06e7a0 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d3fff640 1 --2- 172.21.15.94:0/885557935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c8004810 0x7f23c8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23da59c640 1 --2- 172.21.15.94:0/885557935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c80039f0 0x7f23c8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23dad9d640 1 --2- 172.21.15.94:0/885557935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc06e7a0 0x7f23c80052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d3fff640 1 -- 172.21.15.94:0/885557935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc06e7a0 msgr2=0x7f23c80052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d3fff640 1 --2- 172.21.15.94:0/885557935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc06e7a0 0x7f23c80052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d3fff640 1 -- 172.21.15.94:0/885557935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c80039f0 msgr2=0x7f23c8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d3fff640 1 --2- 172.21.15.94:0/885557935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c80039f0 0x7f23c8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.615+0000 7f23d3fff640 1 -- 172.21.15.94:0/885557935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c8003530 con 0x7f23c8004810 2024-04-25T16:52:05.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.619+0000 7f23d3fff640 1 --2- 172.21.15.94:0/885557935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c8004810 0x7f23c8004c20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f23c4001750 tx=0x7f23c401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef84cf3e3276b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:05.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.619+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c40013a0 con 0x7f23c8004810 2024-04-25T16:52:05.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.619+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c40052f0 con 0x7f23c8004810 2024-04-25T16:52:05.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.619+0000 7f23d9d9b640 1 -- 172.21.15.94:0/885557935 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4003840 con 0x7f23c8004810 2024-04-25T16:52:05.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.619+0000 7f23e0fd5640 1 -- 172.21.15.94:0/885557935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c8004810 msgr2=0x7f23c8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.619+0000 7f23e0fd5640 1 --2- 172.21.15.94:0/885557935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c8004810 0x7f23c8004c20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f23c4001750 tx=0x7f23c401c3b0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.619+0000 7f23dad9d640 1 -- 172.21.15.94:0/885557935 reap_dead start 2024-04-25T16:52:05.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.619+0000 7f23e0fd5640 1 -- 172.21.15.94:0/885557935 shutdown_connections 2024-04-25T16:52:05.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.619+0000 7f23e0fd5640 1 -- 172.21.15.94:0/885557935 >> 172.21.15.94:0/885557935 conn(0x7f23dc0b47f0 msgr2=0x7f23dc0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:05.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.619+0000 7f23e0fd5640 1 -- 172.21.15.94:0/885557935 shutdown_connections 2024-04-25T16:52:05.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.619+0000 7f23e0fd5640 1 -- 172.21.15.94:0/885557935 wait complete. 2024-04-25T16:52:05.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23e0fd5640 1 Processor -- start 2024-04-25T16:52:05.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23e0fd5640 1 -- start start 2024-04-25T16:52:05.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23e0fd5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c8004810 0x7f23dc147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23e0fd5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23dc15d750 0x7f23dc15db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23e0fd5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc1542f0 0x7f23dc154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23e0fd5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc0b0500 con 0x7f23dc15d750 2024-04-25T16:52:05.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23e0fd5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc15cbf0 con 0x7f23c8004810 2024-04-25T16:52:05.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23e0fd5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc0b0680 con 0x7f23dc1542f0 2024-04-25T16:52:05.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23dc15d750 0x7f23dc15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23dc15d750 0x7f23dc15db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54944/0 (socket says 172.21.15.94:54944) 2024-04-25T16:52:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23d3fff640 1 -- 172.21.15.94:0/3143837600 learned_addr learned my addr 172.21.15.94:0/3143837600 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23da59c640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c8004810 0x7f23dc147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23dad9d640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc1542f0 0x7f23dc154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23d3fff640 1 -- 172.21.15.94:0/3143837600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc1542f0 msgr2=0x7f23dc154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23d3fff640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc1542f0 0x7f23dc154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23d3fff640 1 -- 172.21.15.94:0/3143837600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c8004810 msgr2=0x7f23dc147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23d3fff640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c8004810 0x7f23dc147ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23d3fff640 1 -- 172.21.15.94:0/3143837600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc154c40 con 0x7f23dc15d750 2024-04-25T16:52:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23dad9d640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc1542f0 0x7f23dc154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23d3fff640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23dc15d750 0x7f23dc15db60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f23c4005040 tx=0x7f23c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23d2ffd640 1 -- 172.21.15.94:0/3143837600 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c40013a0 con 0x7f23dc15d750 2024-04-25T16:52:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23d2ffd640 1 -- 172.21.15.94:0/3143837600 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c40bb070 con 0x7f23dc15d750 2024-04-25T16:52:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23e0fd5640 1 -- 172.21.15.94:0/3143837600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc15e450 con 0x7f23dc15d750 2024-04-25T16:52:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23d2ffd640 1 -- 172.21.15.94:0/3143837600 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4007a40 con 0x7f23dc15d750 2024-04-25T16:52:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23e0fd5640 1 -- 172.21.15.94:0/3143837600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc0b0f20 con 0x7f23dc15d750 2024-04-25T16:52:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.623+0000 7f23e0fd5640 1 -- 172.21.15.94:0/3143837600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23dc076830 con 0x7f23dc15d750 2024-04-25T16:52:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.627+0000 7f23d2ffd640 1 -- 172.21.15.94:0/3143837600 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f23c40b4040 con 0x7f23dc15d750 2024-04-25T16:52:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.627+0000 7f23d2ffd640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f23bc06f430 0x7f23bc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.627+0000 7f23da59c640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f23bc06f430 0x7f23bc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.627+0000 7f23d2ffd640 1 -- 172.21.15.94:0/3143837600 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f23c412a0c0 con 0x7f23dc15d750 2024-04-25T16:52:05.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.627+0000 7f23da59c640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f23bc06f430 0x7f23bc071940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f23cc0013e0 tx=0x7f23cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:05.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.631+0000 7f23d2ffd640 1 -- 172.21.15.94:0/3143837600 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f23dc154c40 con 0x7f23dc15d750 2024-04-25T16:52:05.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.779+0000 7f23e0fd5640 1 -- 172.21.15.94:0/3143837600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f23dc15e1c0 con 0x7f23dc15d750 2024-04-25T16:52:05.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.779+0000 7f23d2ffd640 1 -- 172.21.15.94:0/3143837600 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f23c40160f0 con 0x7f23dc15d750 2024-04-25T16:52:05.779 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:52:05.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.783+0000 7f23e0fd5640 1 -- 172.21.15.94:0/3143837600 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f23bc06f430 msgr2=0x7f23bc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.783+0000 7f23e0fd5640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f23bc06f430 0x7f23bc071940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f23cc0013e0 tx=0x7f23cc009040 comp rx=0 tx=0).stop 2024-04-25T16:52:05.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.783+0000 7f23e0fd5640 1 -- 172.21.15.94:0/3143837600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23dc15d750 msgr2=0x7f23dc15db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.783+0000 7f23e0fd5640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23dc15d750 0x7f23dc15db60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f23c4005040 tx=0x7f23c8005840 comp rx=0 tx=0).stop 2024-04-25T16:52:05.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.783+0000 7f23e0fd5640 1 -- 172.21.15.94:0/3143837600 shutdown_connections 2024-04-25T16:52:05.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.783+0000 7f23e0fd5640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f23bc06f430 0x7f23bc071940 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.783+0000 7f23e0fd5640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23dc1542f0 0x7f23dc154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.783+0000 7f23e0fd5640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23dc15d750 0x7f23dc15db60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.783+0000 7f23e0fd5640 1 --2- 172.21.15.94:0/3143837600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c8004810 0x7f23dc147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.783+0000 7f23e0fd5640 1 -- 172.21.15.94:0/3143837600 >> 172.21.15.94:0/3143837600 conn(0x7f23dc0b47f0 msgr2=0x7f23dc151780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:05.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.783+0000 7f23e0fd5640 1 -- 172.21.15.94:0/3143837600 shutdown_connections 2024-04-25T16:52:05.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.783+0000 7f23e0fd5640 1 -- 172.21.15.94:0/3143837600 wait complete. 2024-04-25T16:52:05.800 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:52:05.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f1501e640 1 Processor -- start 2024-04-25T16:52:05.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f1501e640 1 -- start start 2024-04-25T16:52:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f1501e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5f100b0420 0x7f5f1006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f1501e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f1006f2a0 0x7f5f1006c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f1501e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f10070630 con 0x7f5f1006cb40 2024-04-25T16:52:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f1501e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f5f100aeaa0 con 0x7f5f100b0420 2024-04-25T16:52:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f1501e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5f100703e0 con 0x7f5f1006f2a0 2024-04-25T16:52:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f0ed9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5f100b0420 0x7f5f1006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f0ed9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5f100b0420 0x7f5f1006ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54946/0 (socket says 172.21.15.94:54946) 2024-04-25T16:52:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f0ed9d640 1 -- 172.21.15.94:0/1511124400 learned_addr learned my addr 172.21.15.94:0/1511124400 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f0e59c640 1 --2- 172.21.15.94:0/1511124400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f1006f2a0 0x7f5f1006c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f0ed9d640 1 -- 172.21.15.94:0/1511124400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f1006f2a0 msgr2=0x7f5f1006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f0ed9d640 1 --2- 172.21.15.94:0/1511124400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f1006f2a0 0x7f5f1006c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f0ed9d640 1 -- 172.21.15.94:0/1511124400 >> v1:172.21.15.138:6789/0 conn(0x7f5f1006cb40 legacy=0x7f5f1015dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f0ed9d640 1 -- 172.21.15.94:0/1511124400 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1006f6b0 con 0x7f5f100b0420 2024-04-25T16:52:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.939+0000 7f5f0e59c640 1 --2- 172.21.15.94:0/1511124400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f1006f2a0 0x7f5f1006c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:05.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.943+0000 7f5f0ed9d640 1 --2- 172.21.15.94:0/1511124400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f100b0420 0x7f5f1006ed60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009870 tx=0x7f5f0001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce267e45c9c1b62f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:05.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.943+0000 7f5f0dd9b640 1 -- 172.21.15.94:0/1511124400 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f000b1070 con 0x7f5f100b0420 2024-04-25T16:52:05.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.943+0000 7f5f0dd9b640 1 -- 172.21.15.94:0/1511124400 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f000ac630 con 0x7f5f100b0420 2024-04-25T16:52:05.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.943+0000 7f5f0dd9b640 1 -- 172.21.15.94:0/1511124400 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f000a8050 con 0x7f5f100b0420 2024-04-25T16:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.943+0000 7f5f1501e640 1 -- 172.21.15.94:0/1511124400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f100b0420 msgr2=0x7f5f1006ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.943+0000 7f5f1501e640 1 --2- 172.21.15.94:0/1511124400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f100b0420 0x7f5f1006ed60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009870 tx=0x7f5f0001c520 comp rx=0 tx=0).stop 2024-04-25T16:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.943+0000 7f5f1501e640 1 -- 172.21.15.94:0/1511124400 shutdown_connections 2024-04-25T16:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.943+0000 7f5f1501e640 1 --2- 172.21.15.94:0/1511124400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f1006f2a0 0x7f5f1006c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.943+0000 7f5f1501e640 1 --2- 172.21.15.94:0/1511124400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f100b0420 0x7f5f1006ed60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.943+0000 7f5f1501e640 1 -- 172.21.15.94:0/1511124400 >> 172.21.15.94:0/1511124400 conn(0x7f5f100b47f0 msgr2=0x7f5f100b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.943+0000 7f5f1501e640 1 -- 172.21.15.94:0/1511124400 shutdown_connections 2024-04-25T16:52:05.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.943+0000 7f5f1501e640 1 -- 172.21.15.94:0/1511124400 wait complete. 2024-04-25T16:52:05.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f1501e640 1 Processor -- start 2024-04-25T16:52:05.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f1501e640 1 -- start start 2024-04-25T16:52:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f1501e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f1006cb40 0x7f5f10165a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f1501e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1006f2a0 0x7f5f101f8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f1501e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f100b0420 0x7f5f101f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f1501e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1006d360 con 0x7f5f100b0420 2024-04-25T16:52:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f1501e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f100aef30 con 0x7f5f1006f2a0 2024-04-25T16:52:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f1501e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5f100b0830 con 0x7f5f1006cb40 2024-04-25T16:52:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f0f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f100b0420 0x7f5f101f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f0f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f100b0420 0x7f5f101f91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54956/0 (socket says 172.21.15.94:54956) 2024-04-25T16:52:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f0f59e640 1 -- 172.21.15.94:0/3350722512 learned_addr learned my addr 172.21.15.94:0/3350722512 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f0e59c640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1006f2a0 0x7f5f101f8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f0ed9d640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f1006cb40 0x7f5f10165a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f0f59e640 1 -- 172.21.15.94:0/3350722512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f1006cb40 msgr2=0x7f5f10165a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f0f59e640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f1006cb40 0x7f5f10165a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f0f59e640 1 -- 172.21.15.94:0/3350722512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1006f2a0 msgr2=0x7f5f101f8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:05.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f0f59e640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1006f2a0 0x7f5f101f8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:05.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f0f59e640 1 -- 172.21.15.94:0/3350722512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f101675a0 con 0x7f5f100b0420 2024-04-25T16:52:05.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f0e59c640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1006f2a0 0x7f5f101f8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:05.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f0f59e640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f100b0420 0x7f5f101f91e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800ba10 tx=0x7f5f0800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:05.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5ef77fe640 1 -- 172.21.15.94:0/3350722512 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08019070 con 0x7f5f100b0420 2024-04-25T16:52:05.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5ef77fe640 1 -- 172.21.15.94:0/3350722512 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f0800cb40 con 0x7f5f100b0420 2024-04-25T16:52:05.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f1501e640 1 -- 172.21.15.94:0/3350722512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f10166370 con 0x7f5f100b0420 2024-04-25T16:52:05.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5ef77fe640 1 -- 172.21.15.94:0/3350722512 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08014440 con 0x7f5f100b0420 2024-04-25T16:52:05.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.947+0000 7f5f1501e640 1 -- 172.21.15.94:0/3350722512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f10166930 con 0x7f5f100b0420 2024-04-25T16:52:05.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.951+0000 7f5f1501e640 1 -- 172.21.15.94:0/3350722512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f10076830 con 0x7f5f100b0420 2024-04-25T16:52:05.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.951+0000 7f5ef77fe640 1 -- 172.21.15.94:0/3350722512 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5f0801c070 con 0x7f5f100b0420 2024-04-25T16:52:05.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.951+0000 7f5ef77fe640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5efc06f430 0x7f5efc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:05.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.951+0000 7f5ef77fe640 1 -- 172.21.15.94:0/3350722512 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5f0808f210 con 0x7f5f100b0420 2024-04-25T16:52:05.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.951+0000 7f5f0ed9d640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5efc06f430 0x7f5efc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:05.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.951+0000 7f5f0ed9d640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5efc06f430 0x7f5efc071940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5f000ae000 tx=0x7f5f00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:05.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:05.955+0000 7f5ef77fe640 1 -- 172.21.15.94:0/3350722512 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5f0805bd90 con 0x7f5f100b0420 2024-04-25T16:52:06.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.127+0000 7f5f1501e640 1 -- 172.21.15.94:0/3350722512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5f100b1f80 con 0x7f5f100b0420 2024-04-25T16:52:06.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.131+0000 7f5ef77fe640 1 -- 172.21.15.94:0/3350722512 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f5f0805b690 con 0x7f5f100b0420 2024-04-25T16:52:06.128 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:52:06.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.131+0000 7f5f1501e640 1 -- 172.21.15.94:0/3350722512 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5efc06f430 msgr2=0x7f5efc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.131+0000 7f5f1501e640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5efc06f430 0x7f5efc071940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5f000ae000 tx=0x7f5f00008040 comp rx=0 tx=0).stop 2024-04-25T16:52:06.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.131+0000 7f5f1501e640 1 -- 172.21.15.94:0/3350722512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f100b0420 msgr2=0x7f5f101f91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.131+0000 7f5f1501e640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f100b0420 0x7f5f101f91e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800ba10 tx=0x7f5f0800bed0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.131+0000 7f5f1501e640 1 -- 172.21.15.94:0/3350722512 shutdown_connections 2024-04-25T16:52:06.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.131+0000 7f5f1501e640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f100b0420 0x7f5f101f91e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.131+0000 7f5f1501e640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1006f2a0 0x7f5f101f8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.131+0000 7f5f1501e640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5efc06f430 0x7f5efc071940 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.131+0000 7f5f1501e640 1 --2- 172.21.15.94:0/3350722512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f1006cb40 0x7f5f10165a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.131+0000 7f5f1501e640 1 -- 172.21.15.94:0/3350722512 >> 172.21.15.94:0/3350722512 conn(0x7f5f100b47f0 msgr2=0x7f5f100b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.131+0000 7f5f1501e640 1 -- 172.21.15.94:0/3350722512 shutdown_connections 2024-04-25T16:52:06.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.131+0000 7f5f1501e640 1 -- 172.21.15.94:0/3350722512 wait complete. 2024-04-25T16:52:06.149 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-04-25T16:52:06.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:06.149 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T16:52:06.159 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:52:06.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T16:52:06.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:06.159 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T16:52:06.204 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:52:06.204 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:52:06.211 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:52:06.211 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:52:06.259 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:52:06.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f95927e4640 1 Processor -- start 2024-04-25T16:52:06.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f95927e4640 1 -- start start 2024-04-25T16:52:06.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f95927e4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f958c070080 0x7f958c15e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f95927e4640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f958c15ebd0 0x7f958c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f95927e4640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f958c06fc60 con 0x7f958c15e7c0 2024-04-25T16:52:06.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f95927e4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f958c158550 con 0x7f958c070080 2024-04-25T16:52:06.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f95927e4640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f958c06f7d0 con 0x7f958c15ebd0 2024-04-25T16:52:06.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f958bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f958c070080 0x7f958c15e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f958bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f958c070080 0x7f958c15e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54958/0 (socket says 172.21.15.94:54958) 2024-04-25T16:52:06.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f958bfff640 1 -- 172.21.15.94:0/179476167 learned_addr learned my addr 172.21.15.94:0/179476167 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:06.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f9590d81640 1 --2- 172.21.15.94:0/179476167 >> v2:172.21.15.138:3300/0 conn(0x7f958c15ebd0 0x7f958c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f958bfff640 1 -- 172.21.15.94:0/179476167 >> v2:172.21.15.138:3300/0 conn(0x7f958c15ebd0 msgr2=0x7f958c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:06.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f958bfff640 1 --2- 172.21.15.94:0/179476167 >> v2:172.21.15.138:3300/0 conn(0x7f958c15ebd0 0x7f958c06cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f958bfff640 1 -- 172.21.15.94:0/179476167 >> v1:172.21.15.138:6789/0 conn(0x7f958c15e7c0 legacy=0x7f958c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.411+0000 7f958bfff640 1 -- 172.21.15.94:0/179476167 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c0aef70 con 0x7f958c070080 2024-04-25T16:52:06.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.415+0000 7f958bfff640 1 --2- 172.21.15.94:0/179476167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c070080 0x7f958c15e280 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f958c071c10 tx=0x7f957c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36044aefcfd22f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:06.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.415+0000 7f958affd640 1 -- 172.21.15.94:0/179476167 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c0ab600 con 0x7f958c070080 2024-04-25T16:52:06.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.415+0000 7f958affd640 1 -- 172.21.15.94:0/179476167 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957c0b0070 con 0x7f958c070080 2024-04-25T16:52:06.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.415+0000 7f958affd640 1 -- 172.21.15.94:0/179476167 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c0ab9c0 con 0x7f958c070080 2024-04-25T16:52:06.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.415+0000 7f95927e4640 1 -- 172.21.15.94:0/179476167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c070080 msgr2=0x7f958c15e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.415+0000 7f95927e4640 1 --2- 172.21.15.94:0/179476167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c070080 0x7f958c15e280 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f958c071c10 tx=0x7f957c01c4f0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.415+0000 7f95927e4640 1 -- 172.21.15.94:0/179476167 shutdown_connections 2024-04-25T16:52:06.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.415+0000 7f95927e4640 1 --2- 172.21.15.94:0/179476167 >> v2:172.21.15.138:3300/0 conn(0x7f958c15ebd0 0x7f958c06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.415+0000 7f95927e4640 1 --2- 172.21.15.94:0/179476167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c070080 0x7f958c15e280 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.415+0000 7f95927e4640 1 -- 172.21.15.94:0/179476167 >> 172.21.15.94:0/179476167 conn(0x7f958c0b47f0 msgr2=0x7f958c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:06.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.415+0000 7f95927e4640 1 -- 172.21.15.94:0/179476167 shutdown_connections 2024-04-25T16:52:06.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.415+0000 7f95927e4640 1 -- 172.21.15.94:0/179476167 wait complete. 2024-04-25T16:52:06.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f95927e4640 1 Processor -- start 2024-04-25T16:52:06.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f95927e4640 1 -- start start 2024-04-25T16:52:06.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f95927e4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f958c070080 0x7f958c09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f95927e4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c15e7c0 0x7f958c09b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f95927e4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f958c15ebd0 0x7f958c15cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f95927e4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f958c070490 con 0x7f958c15e7c0 2024-04-25T16:52:06.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f95927e4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f958c0ae6c0 con 0x7f958c070080 2024-04-25T16:52:06.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f95927e4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f958c16ae90 con 0x7f958c15ebd0 2024-04-25T16:52:06.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f958b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c15e7c0 0x7f958c09b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f958b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c15e7c0 0x7f958c09b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54972/0 (socket says 172.21.15.94:54972) 2024-04-25T16:52:06.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f958b7fe640 1 -- 172.21.15.94:0/4035143251 learned_addr learned my addr 172.21.15.94:0/4035143251 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:06.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f958bfff640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f958c070080 0x7f958c09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f9590d81640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f958c15ebd0 0x7f958c15cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f958b7fe640 1 -- 172.21.15.94:0/4035143251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f958c15ebd0 msgr2=0x7f958c15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f958b7fe640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f958c15ebd0 0x7f958c15cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f958b7fe640 1 -- 172.21.15.94:0/4035143251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f958c070080 msgr2=0x7f958c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f958b7fe640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f958c070080 0x7f958c09aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f958b7fe640 1 -- 172.21.15.94:0/4035143251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c156520 con 0x7f958c15e7c0 2024-04-25T16:52:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f958bfff640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f958c070080 0x7f958c09aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f958b7fe640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c15e7c0 0x7f958c09b9e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f958000c310 tx=0x7f958000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f9588ff9640 1 -- 172.21.15.94:0/4035143251 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800052d0 con 0x7f958c15e7c0 2024-04-25T16:52:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f9588ff9640 1 -- 172.21.15.94:0/4035143251 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9580005470 con 0x7f958c15e7c0 2024-04-25T16:52:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f95927e4640 1 -- 172.21.15.94:0/4035143251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958c156800 con 0x7f958c15e7c0 2024-04-25T16:52:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f9588ff9640 1 -- 172.21.15.94:0/4035143251 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580007260 con 0x7f958c15e7c0 2024-04-25T16:52:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f95927e4640 1 -- 172.21.15.94:0/4035143251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f958c156dc0 con 0x7f958c15e7c0 2024-04-25T16:52:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.419+0000 7f95927e4640 1 -- 172.21.15.94:0/4035143251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f958c076830 con 0x7f958c15e7c0 2024-04-25T16:52:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.423+0000 7f9588ff9640 1 -- 172.21.15.94:0/4035143251 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9580007400 con 0x7f958c15e7c0 2024-04-25T16:52:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.423+0000 7f9588ff9640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f957806f430 0x7f9578071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.423+0000 7f9588ff9640 1 -- 172.21.15.94:0/4035143251 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f958008e9d0 con 0x7f958c15e7c0 2024-04-25T16:52:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.423+0000 7f958bfff640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f957806f430 0x7f9578071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.423+0000 7f958bfff640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f957806f430 0x7f9578071940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f957c0af000 tx=0x7f957c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:06.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.427+0000 7f9588ff9640 1 -- 172.21.15.94:0/4035143251 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f958005b610 con 0x7f958c15e7c0 2024-04-25T16:52:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.575+0000 7f95927e4640 1 -- 172.21.15.94:0/4035143251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f958c151aa0 con 0x7f958c15e7c0 2024-04-25T16:52:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.579+0000 7f9588ff9640 1 -- 172.21.15.94:0/4035143251 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7f958005af10 con 0x7f958c15e7c0 2024-04-25T16:52:06.576 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:06.576 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:52:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.579+0000 7f95927e4640 1 -- 172.21.15.94:0/4035143251 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f957806f430 msgr2=0x7f9578071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.579+0000 7f95927e4640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f957806f430 0x7f9578071940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f957c0af000 tx=0x7f957c008040 comp rx=0 tx=0).stop 2024-04-25T16:52:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.579+0000 7f95927e4640 1 -- 172.21.15.94:0/4035143251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c15e7c0 msgr2=0x7f958c09b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.579+0000 7f95927e4640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c15e7c0 0x7f958c09b9e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f958000c310 tx=0x7f958000c7d0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.579+0000 7f95927e4640 1 -- 172.21.15.94:0/4035143251 shutdown_connections 2024-04-25T16:52:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.579+0000 7f95927e4640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f957806f430 0x7f9578071940 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.579+0000 7f95927e4640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f958c15ebd0 0x7f958c15cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.579+0000 7f95927e4640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c15e7c0 0x7f958c09b9e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.579+0000 7f95927e4640 1 --2- 172.21.15.94:0/4035143251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f958c070080 0x7f958c09aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.579+0000 7f95927e4640 1 -- 172.21.15.94:0/4035143251 >> 172.21.15.94:0/4035143251 conn(0x7f958c0b47f0 msgr2=0x7f958c14fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.579+0000 7f95927e4640 1 -- 172.21.15.94:0/4035143251 shutdown_connections 2024-04-25T16:52:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.579+0000 7f95927e4640 1 -- 172.21.15.94:0/4035143251 wait complete. 2024-04-25T16:52:06.598 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:52:06.598 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:52:06.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f6416aab640 1 Processor -- start 2024-04-25T16:52:06.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f6416aab640 1 -- start start 2024-04-25T16:52:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f6416aab640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f641015da50 0x7f6410168ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f6416aab640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f641015de60 con 0x7f64100ce990 2024-04-25T16:52:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f6416aab640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64100b64c0 con 0x7f64100d2710 2024-04-25T16:52:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f6416aab640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f64100b6270 con 0x7f641015da50 2024-04-25T16:52:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f6414847640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f64100ce990 0x7f64100d2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38872/0 (socket says 172.21.15.94:38872) 2024-04-25T16:52:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f6414847640 1 -- 172.21.15.94:0/3120117030 learned_addr learned my addr 172.21.15.94:0/3120117030 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1440877549 0 0) 0x7f641015de60 con 0x7f64100ce990 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63f8003530 con 0x7f64100ce990 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f6415048640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f641015da50 0x7f6410168ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 246373219 0 0) 0x7f63f8003530 con 0x7f64100ce990 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f641015da50 msgr2=0x7f6410168ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f641015da50 0x7f6410168ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 >> v1:172.21.15.138:6789/0 conn(0x7f64100d2710 legacy=0x7f6410165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64100b67a0 con 0x7f64100ce990 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f6400001680 con 0x7f64100ce990 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f6415048640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f641015da50 0x7f6410168ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 >> v1:172.21.15.94:6789/0 conn(0x7f64100ce990 legacy=0x7f64100d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63f80039f0 0x7f63f8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63f8004810 0x7f63f8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f641015da50 0x7f63f80052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64100b6270 con 0x7f63f8004810 2024-04-25T16:52:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f63f800bc20 con 0x7f63f80039f0 2024-04-25T16:52:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.747+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f63f800be20 con 0x7f641015da50 2024-04-25T16:52:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f640ffff640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63f8004810 0x7f63f8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f6414847640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63f80039f0 0x7f63f8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f6415048640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f641015da50 0x7f63f80052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f640ffff640 1 -- 172.21.15.94:0/3120117030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f641015da50 msgr2=0x7f63f80052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f640ffff640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f641015da50 0x7f63f80052c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f640ffff640 1 -- 172.21.15.94:0/3120117030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63f80039f0 msgr2=0x7f63f8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f640ffff640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63f80039f0 0x7f63f8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f640ffff640 1 -- 172.21.15.94:0/3120117030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f8003530 con 0x7f63f8004810 2024-04-25T16:52:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f6414847640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63f80039f0 0x7f63f8003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:06.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f640ffff640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63f8004810 0x7f63f8004c20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f64100d0a80 tx=0x7f640401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b312bc93d9ed03a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:06.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040013a0 con 0x7f63f8004810 2024-04-25T16:52:06.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64040052f0 con 0x7f63f8004810 2024-04-25T16:52:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f640f7fe640 1 -- 172.21.15.94:0/3120117030 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404003840 con 0x7f63f8004810 2024-04-25T16:52:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f6416aab640 1 -- 172.21.15.94:0/3120117030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63f8004810 msgr2=0x7f63f8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f6416aab640 1 --2- 172.21.15.94:0/3120117030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63f8004810 0x7f63f8004c20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f64100d0a80 tx=0x7f640401c3a0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f6415048640 1 -- 172.21.15.94:0/3120117030 reap_dead start 2024-04-25T16:52:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f6416aab640 1 -- 172.21.15.94:0/3120117030 shutdown_connections 2024-04-25T16:52:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f6416aab640 1 -- 172.21.15.94:0/3120117030 >> 172.21.15.94:0/3120117030 conn(0x7f64100cfea0 msgr2=0x7f64100d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f6416aab640 1 -- 172.21.15.94:0/3120117030 shutdown_connections 2024-04-25T16:52:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.751+0000 7f6416aab640 1 -- 172.21.15.94:0/3120117030 wait complete. 2024-04-25T16:52:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6416aab640 1 Processor -- start 2024-04-25T16:52:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6416aab640 1 -- start start 2024-04-25T16:52:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6416aab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63f8004810 0x7f64101f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6416aab640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64100d21d0 0x7f64100b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6416aab640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64100b3a90 0x7f64100cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6416aab640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64100cf160 con 0x7f63f8004810 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6416aab640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64100b6d60 con 0x7f64100b3a90 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6416aab640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f64100cf2e0 con 0x7f64100d21d0 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6414847640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63f8004810 0x7f64101f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6414847640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63f8004810 0x7f64101f3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54990/0 (socket says 172.21.15.94:54990) 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6414847640 1 -- 172.21.15.94:0/3017816510 learned_addr learned my addr 172.21.15.94:0/3017816510 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6415048640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64100b3a90 0x7f64100cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f640ffff640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64100d21d0 0x7f64100b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6414847640 1 -- 172.21.15.94:0/3017816510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64100d21d0 msgr2=0x7f64100b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6414847640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64100d21d0 0x7f64100b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6414847640 1 -- 172.21.15.94:0/3017816510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64100b3a90 msgr2=0x7f64100cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6414847640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64100b3a90 0x7f64100cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6414847640 1 -- 172.21.15.94:0/3017816510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64100b3ea0 con 0x7f63f8004810 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6414847640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63f8004810 0x7f64101f3ae0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f640000d910 tx=0x7f640000ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f640d7fa640 1 -- 172.21.15.94:0/3017816510 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400019070 con 0x7f63f8004810 2024-04-25T16:52:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f640d7fa640 1 -- 172.21.15.94:0/3017816510 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64000074e0 con 0x7f63f8004810 2024-04-25T16:52:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6416aab640 1 -- 172.21.15.94:0/3017816510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6410156c30 con 0x7f63f8004810 2024-04-25T16:52:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f640d7fa640 1 -- 172.21.15.94:0/3017816510 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64000049f0 con 0x7f63f8004810 2024-04-25T16:52:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.755+0000 7f6416aab640 1 -- 172.21.15.94:0/3017816510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6410155e70 con 0x7f63f8004810 2024-04-25T16:52:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.759+0000 7f6416aab640 1 -- 172.21.15.94:0/3017816510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f641006a730 con 0x7f63f8004810 2024-04-25T16:52:06.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.759+0000 7f640d7fa640 1 -- 172.21.15.94:0/3017816510 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6400004b90 con 0x7f63f8004810 2024-04-25T16:52:06.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.759+0000 7f640d7fa640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f63fc06f430 0x7f63fc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:06.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.759+0000 7f640d7fa640 1 -- 172.21.15.94:0/3017816510 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f640008f300 con 0x7f63f8004810 2024-04-25T16:52:06.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.759+0000 7f640ffff640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f63fc06f430 0x7f63fc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:06.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.759+0000 7f640ffff640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f63fc06f430 0x7f63fc071940 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63f8004790 tx=0x7f64040011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:06.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.763+0000 7f640d7fa640 1 -- 172.21.15.94:0/3017816510 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f64100b3ea0 con 0x7f63f8004810 2024-04-25T16:52:06.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.923+0000 7f6416aab640 1 -- 172.21.15.94:0/3017816510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f641015cd50 con 0x7f63f8004810 2024-04-25T16:52:06.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.923+0000 7f640d7fa640 1 -- 172.21.15.94:0/3017816510 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f640005bed0 con 0x7f63f8004810 2024-04-25T16:52:06.922 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:52:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.927+0000 7f6416aab640 1 -- 172.21.15.94:0/3017816510 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f63fc06f430 msgr2=0x7f63fc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.927+0000 7f6416aab640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f63fc06f430 0x7f63fc071940 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63f8004790 tx=0x7f64040011a0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.927+0000 7f6416aab640 1 -- 172.21.15.94:0/3017816510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63f8004810 msgr2=0x7f64101f3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.927+0000 7f6416aab640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63f8004810 0x7f64101f3ae0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f640000d910 tx=0x7f640000ddd0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.927+0000 7f6416aab640 1 -- 172.21.15.94:0/3017816510 shutdown_connections 2024-04-25T16:52:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.927+0000 7f6416aab640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64100b3a90 0x7f64100cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.927+0000 7f6416aab640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f63fc06f430 0x7f63fc071940 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.927+0000 7f6416aab640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64100d21d0 0x7f64100b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.927+0000 7f6416aab640 1 --2- 172.21.15.94:0/3017816510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63f8004810 0x7f64101f3ae0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.927+0000 7f6416aab640 1 -- 172.21.15.94:0/3017816510 >> 172.21.15.94:0/3017816510 conn(0x7f64100cfea0 msgr2=0x7f6410157d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:06.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.927+0000 7f6416aab640 1 -- 172.21.15.94:0/3017816510 shutdown_connections 2024-04-25T16:52:06.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:06.927+0000 7f6416aab640 1 -- 172.21.15.94:0/3017816510 wait complete. 2024-04-25T16:52:06.943 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:52:07.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f8469a50640 1 Processor -- start 2024-04-25T16:52:07.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f8469a50640 1 -- start start 2024-04-25T16:52:07.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f8469a50640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f84640d1b40 0x7f84640d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f8469a50640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8464164510 con 0x7f84640d3450 2024-04-25T16:52:07.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f8469a50640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84641647b0 con 0x7f84640d3f70 2024-04-25T16:52:07.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f8469a50640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8464164a90 con 0x7f84640d1b40 2024-04-25T16:52:07.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f8462ffd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f84640d3450 0x7f84640d3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38874/0 (socket says 172.21.15.94:38874) 2024-04-25T16:52:07.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f8462ffd640 1 -- 172.21.15.94:0/3553944699 learned_addr learned my addr 172.21.15.94:0/3553944699 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84637fe640 1 --2- 172.21.15.94:0/3553944699 >> v2:172.21.15.138:3300/0 conn(0x7f84640d1b40 0x7f84640d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1997211050 0 0) 0x7f8464164510 con 0x7f84640d3450 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8450003530 con 0x7f84640d3450 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2704113117 0 0) 0x7f8450003530 con 0x7f84640d3450 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 >> v2:172.21.15.138:3300/0 conn(0x7f84640d1b40 msgr2=0x7f84640d1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 --2- 172.21.15.94:0/3553944699 >> v2:172.21.15.138:3300/0 conn(0x7f84640d1b40 0x7f84640d1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 >> v1:172.21.15.138:6789/0 conn(0x7f84640d3f70 legacy=0x7f84640d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84640d4b60 con 0x7f84640d3450 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f8454001680 con 0x7f84640d3450 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 >> v1:172.21.15.94:6789/0 conn(0x7f84640d3450 legacy=0x7f84640d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 --2- 172.21.15.94:0/3553944699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84500039c0 0x7f8450003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 --2- 172.21.15.94:0/3553944699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8450004810 0x7f8450004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 --2- 172.21.15.94:0/3553944699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8450005260 0x7f845000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8464164a90 con 0x7f8450004810 2024-04-25T16:52:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f845000c1f0 con 0x7f8450005260 2024-04-25T16:52:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f845000c3f0 con 0x7f84500039c0 2024-04-25T16:52:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f845bfff640 1 --2- 172.21.15.94:0/3553944699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8450004810 0x7f8450004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f845bfff640 1 -- 172.21.15.94:0/3553944699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84500039c0 msgr2=0x7f8450003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f845bfff640 1 --2- 172.21.15.94:0/3553944699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84500039c0 0x7f8450003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f845bfff640 1 -- 172.21.15.94:0/3553944699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8450005260 msgr2=0x7f845000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f845bfff640 1 --2- 172.21.15.94:0/3553944699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8450005260 0x7f845000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f845bfff640 1 -- 172.21.15.94:0/3553944699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8450003530 con 0x7f8450004810 2024-04-25T16:52:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.091+0000 7f84637fe640 1 -- 172.21.15.94:0/3553944699 reap_dead start 2024-04-25T16:52:07.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.095+0000 7f845bfff640 1 --2- 172.21.15.94:0/3553944699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8450004810 0x7f8450004c20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f844c001780 tx=0x7f844c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b1b82b749fd6d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:07.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.095+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c0013a0 con 0x7f8450004810 2024-04-25T16:52:07.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.095+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f844c0052f0 con 0x7f8450004810 2024-04-25T16:52:07.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.095+0000 7f84627fc640 1 -- 172.21.15.94:0/3553944699 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c003840 con 0x7f8450004810 2024-04-25T16:52:07.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.095+0000 7f8469a50640 1 -- 172.21.15.94:0/3553944699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8450004810 msgr2=0x7f8450004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.095+0000 7f8469a50640 1 --2- 172.21.15.94:0/3553944699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8450004810 0x7f8450004c20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f844c001780 tx=0x7f844c01c420 comp rx=0 tx=0).stop 2024-04-25T16:52:07.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.095+0000 7f8469a50640 1 -- 172.21.15.94:0/3553944699 shutdown_connections 2024-04-25T16:52:07.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.095+0000 7f8469a50640 1 --2- 172.21.15.94:0/3553944699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8450004810 0x7f8450004c20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.095+0000 7f8469a50640 1 -- 172.21.15.94:0/3553944699 >> 172.21.15.94:0/3553944699 conn(0x7f846409e8d0 msgr2=0x7f84640b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:07.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.095+0000 7f8469a50640 1 -- 172.21.15.94:0/3553944699 shutdown_connections 2024-04-25T16:52:07.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.095+0000 7f8469a50640 1 -- 172.21.15.94:0/3553944699 wait complete. 2024-04-25T16:52:07.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8469a50640 1 Processor -- start 2024-04-25T16:52:07.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8469a50640 1 -- start start 2024-04-25T16:52:07.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8469a50640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8450004810 0x7f84640ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8469a50640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f846415ba00 0x7f84640ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8469a50640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f846415be10 0x7f84640cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8469a50640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84640d1840 con 0x7f846415ba00 2024-04-25T16:52:07.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8469a50640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84640d2760 con 0x7f846415be10 2024-04-25T16:52:07.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8469a50640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f84640d19c0 con 0x7f8450004810 2024-04-25T16:52:07.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f845bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f846415ba00 0x7f84640ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f845bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f846415ba00 0x7f84640ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55010/0 (socket says 172.21.15.94:55010) 2024-04-25T16:52:07.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f845bfff640 1 -- 172.21.15.94:0/1595820787 learned_addr learned my addr 172.21.15.94:0/1595820787 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8462ffd640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8450004810 0x7f84640ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f84637fe640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f846415be10 0x7f84640cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f845bfff640 1 -- 172.21.15.94:0/1595820787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8450004810 msgr2=0x7f84640ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f845bfff640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8450004810 0x7f84640ccf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f845bfff640 1 -- 172.21.15.94:0/1595820787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f846415be10 msgr2=0x7f84640cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f845bfff640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f846415be10 0x7f84640cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f845bfff640 1 -- 172.21.15.94:0/1595820787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84640c96d0 con 0x7f846415ba00 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f84637fe640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f846415be10 0x7f84640cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f845bfff640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f846415ba00 0x7f84640ceac0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f844c01c250 tx=0x7f8450005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8460ff9640 1 -- 172.21.15.94:0/1595820787 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c007a40 con 0x7f846415ba00 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8460ff9640 1 -- 172.21.15.94:0/1595820787 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f844c0054a0 con 0x7f846415ba00 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8469a50640 1 -- 172.21.15.94:0/1595820787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84640c99b0 con 0x7f846415ba00 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8460ff9640 1 -- 172.21.15.94:0/1595820787 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c003d00 con 0x7f846415ba00 2024-04-25T16:52:07.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8469a50640 1 -- 172.21.15.94:0/1595820787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84640c9f70 con 0x7f846415ba00 2024-04-25T16:52:07.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.099+0000 7f8469a50640 1 -- 172.21.15.94:0/1595820787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84640c9420 con 0x7f846415ba00 2024-04-25T16:52:07.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.103+0000 7f8460ff9640 1 -- 172.21.15.94:0/1595820787 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f844c0bd020 con 0x7f846415ba00 2024-04-25T16:52:07.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.103+0000 7f8460ff9640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f844406f400 0x7f8444071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.103+0000 7f8460ff9640 1 -- 172.21.15.94:0/1595820787 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f844c128db0 con 0x7f846415ba00 2024-04-25T16:52:07.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.103+0000 7f8462ffd640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f844406f400 0x7f8444071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.103+0000 7f8462ffd640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f844406f400 0x7f8444071910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f84640b8c60 tx=0x7f8454009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:07.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.107+0000 7f8460ff9640 1 -- 172.21.15.94:0/1595820787 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f844c0c4020 con 0x7f846415ba00 2024-04-25T16:52:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8469a50640 1 -- 172.21.15.94:0/1595820787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8464076830 con 0x7f846415ba00 2024-04-25T16:52:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8460ff9640 1 -- 172.21.15.94:0/1595820787 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f844c016130 con 0x7f846415ba00 2024-04-25T16:52:07.253 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:52:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8469a50640 1 -- 172.21.15.94:0/1595820787 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f844406f400 msgr2=0x7f8444071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8469a50640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f844406f400 0x7f8444071910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f84640b8c60 tx=0x7f8454009040 comp rx=0 tx=0).stop 2024-04-25T16:52:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8469a50640 1 -- 172.21.15.94:0/1595820787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f846415ba00 msgr2=0x7f84640ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8469a50640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f846415ba00 0x7f84640ceac0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f844c01c250 tx=0x7f8450005840 comp rx=0 tx=0).stop 2024-04-25T16:52:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8469a50640 1 -- 172.21.15.94:0/1595820787 shutdown_connections 2024-04-25T16:52:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8469a50640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f846415be10 0x7f84640cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8469a50640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f846415ba00 0x7f84640ceac0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8469a50640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f844406f400 0x7f8444071910 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8469a50640 1 --2- 172.21.15.94:0/1595820787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8450004810 0x7f84640ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8469a50640 1 -- 172.21.15.94:0/1595820787 >> 172.21.15.94:0/1595820787 conn(0x7f846409e8d0 msgr2=0x7f84640b66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:07.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8469a50640 1 -- 172.21.15.94:0/1595820787 shutdown_connections 2024-04-25T16:52:07.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.255+0000 7f8469a50640 1 -- 172.21.15.94:0/1595820787 wait complete. 2024-04-25T16:52:07.273 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:52:07.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34e11e5640 1 Processor -- start 2024-04-25T16:52:07.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34e11e5640 1 -- start start 2024-04-25T16:52:07.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34e11e5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f34dc06f470 0x7f34dc15dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34e11e5640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34dc070630 con 0x7f34dc15e160 2024-04-25T16:52:07.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34e11e5640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34dc0ae170 con 0x7f34dc0b0420 2024-04-25T16:52:07.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34e11e5640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f34dc0aeaa0 con 0x7f34dc06f470 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34db59e640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f34dc15e160 0x7f34dc15e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38882/0 (socket says 172.21.15.94:38882) 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34db59e640 1 -- 172.21.15.94:0/1477739708 learned_addr learned my addr 172.21.15.94:0/1477739708 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1481147712 0 0) 0x7f34dc070630 con 0x7f34dc15e160 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34da59c640 1 --2- 172.21.15.94:0/1477739708 >> v2:172.21.15.138:3300/0 conn(0x7f34dc06f470 0x7f34dc15dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34c8003530 con 0x7f34dc15e160 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3244514487 0 0) 0x7f34c8003530 con 0x7f34dc15e160 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 >> v2:172.21.15.138:3300/0 conn(0x7f34dc06f470 msgr2=0x7f34dc15dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 --2- 172.21.15.94:0/1477739708 >> v2:172.21.15.138:3300/0 conn(0x7f34dc06f470 0x7f34dc15dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 >> v1:172.21.15.138:6789/0 conn(0x7f34dc0b0420 legacy=0x7f34dc06ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc0ae450 con 0x7f34dc15e160 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f34d4001a00 con 0x7f34dc15e160 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 >> v1:172.21.15.94:6789/0 conn(0x7f34dc15e160 legacy=0x7f34dc15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 --2- 172.21.15.94:0/1477739708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c80039c0 0x7f34c8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 --2- 172.21.15.94:0/1477739708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34c8004810 0x7f34c8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 --2- 172.21.15.94:0/1477739708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34c8005280 0x7f34c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc0aeaa0 con 0x7f34c80039c0 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34c800c210 con 0x7f34c8004810 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34c800c410 con 0x7f34c8005280 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34dad9d640 1 --2- 172.21.15.94:0/1477739708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c80039c0 0x7f34c8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34da59c640 1 --2- 172.21.15.94:0/1477739708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34c8004810 0x7f34c8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34db59e640 1 --2- 172.21.15.94:0/1477739708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34c8005280 0x7f34c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34dad9d640 1 -- 172.21.15.94:0/1477739708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34c8005280 msgr2=0x7f34c800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34dad9d640 1 --2- 172.21.15.94:0/1477739708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34c8005280 0x7f34c800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34dad9d640 1 -- 172.21.15.94:0/1477739708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34c8004810 msgr2=0x7f34c8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34dad9d640 1 --2- 172.21.15.94:0/1477739708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34c8004810 0x7f34c8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34db59e640 1 -- 172.21.15.94:0/1477739708 reap_dead start 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34dad9d640 1 -- 172.21.15.94:0/1477739708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c8003530 con 0x7f34c80039c0 2024-04-25T16:52:07.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.415+0000 7f34da59c640 1 --2- 172.21.15.94:0/1477739708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34c8004810 0x7f34c8004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:07.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.419+0000 7f34dad9d640 1 --2- 172.21.15.94:0/1477739708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c80039c0 0x7f34c8003e30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f34cc001750 tx=0x7f34cc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88535c89107531fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:07.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.419+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc0013a0 con 0x7f34c80039c0 2024-04-25T16:52:07.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.419+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34cc0052f0 con 0x7f34c80039c0 2024-04-25T16:52:07.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.419+0000 7f34d9d9b640 1 -- 172.21.15.94:0/1477739708 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc003840 con 0x7f34c80039c0 2024-04-25T16:52:07.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.419+0000 7f34e11e5640 1 -- 172.21.15.94:0/1477739708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c80039c0 msgr2=0x7f34c8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.419+0000 7f34e11e5640 1 --2- 172.21.15.94:0/1477739708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c80039c0 0x7f34c8003e30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f34cc001750 tx=0x7f34cc01c3c0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.419+0000 7f34e11e5640 1 -- 172.21.15.94:0/1477739708 shutdown_connections 2024-04-25T16:52:07.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.419+0000 7f34e11e5640 1 --2- 172.21.15.94:0/1477739708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c80039c0 0x7f34c8003e30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.419+0000 7f34e11e5640 1 -- 172.21.15.94:0/1477739708 >> 172.21.15.94:0/1477739708 conn(0x7f34dc0b47f0 msgr2=0x7f34dc0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:07.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.419+0000 7f34e11e5640 1 -- 172.21.15.94:0/1477739708 shutdown_connections 2024-04-25T16:52:07.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.419+0000 7f34e11e5640 1 -- 172.21.15.94:0/1477739708 wait complete. 2024-04-25T16:52:07.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34e11e5640 1 Processor -- start 2024-04-25T16:52:07.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34e11e5640 1 -- start start 2024-04-25T16:52:07.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34e11e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c80039c0 0x7f34dc150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34e11e5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc1faa20 0x7f34dc1f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34e11e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34dc153330 0x7f34dc153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34e11e5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc15e570 con 0x7f34c80039c0 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34e11e5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc0b0a40 con 0x7f34dc153330 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34e11e5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34dc15e6f0 con 0x7f34dc1faa20 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34dad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c80039c0 0x7f34dc150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34dad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c80039c0 0x7f34dc150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55032/0 (socket says 172.21.15.94:55032) 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34dad9d640 1 -- 172.21.15.94:0/1347934904 learned_addr learned my addr 172.21.15.94:0/1347934904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34db59e640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34dc153330 0x7f34dc153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34da59c640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc1faa20 0x7f34dc1f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34dad9d640 1 -- 172.21.15.94:0/1347934904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc1faa20 msgr2=0x7f34dc1f7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34dad9d640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc1faa20 0x7f34dc1f7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34dad9d640 1 -- 172.21.15.94:0/1347934904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34dc153330 msgr2=0x7f34dc153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34dad9d640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34dc153330 0x7f34dc153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34dad9d640 1 -- 172.21.15.94:0/1347934904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc153c80 con 0x7f34c80039c0 2024-04-25T16:52:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34dad9d640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c80039c0 0x7f34dc150860 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f34cc01c1f0 tx=0x7f34c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:07.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34d37fe640 1 -- 172.21.15.94:0/1347934904 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc007a40 con 0x7f34c80039c0 2024-04-25T16:52:07.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34d37fe640 1 -- 172.21.15.94:0/1347934904 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34cc0ba070 con 0x7f34c80039c0 2024-04-25T16:52:07.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34e11e5640 1 -- 172.21.15.94:0/1347934904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34dc1fafc0 con 0x7f34c80039c0 2024-04-25T16:52:07.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34d37fe640 1 -- 172.21.15.94:0/1347934904 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc0054a0 con 0x7f34c80039c0 2024-04-25T16:52:07.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34e11e5640 1 -- 172.21.15.94:0/1347934904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34dc1fb580 con 0x7f34c80039c0 2024-04-25T16:52:07.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.423+0000 7f34e11e5640 1 -- 172.21.15.94:0/1347934904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34dc076830 con 0x7f34c80039c0 2024-04-25T16:52:07.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.427+0000 7f34d37fe640 1 -- 172.21.15.94:0/1347934904 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f34cc007030 con 0x7f34c80039c0 2024-04-25T16:52:07.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.427+0000 7f34d37fe640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f34bc06f430 0x7f34bc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.427+0000 7f34da59c640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f34bc06f430 0x7f34bc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.427+0000 7f34d37fe640 1 -- 172.21.15.94:0/1347934904 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f34cc129690 con 0x7f34c80039c0 2024-04-25T16:52:07.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.427+0000 7f34da59c640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f34bc06f430 0x7f34bc071940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34c4004660 tx=0x7f34c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:07.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.431+0000 7f34d37fe640 1 -- 172.21.15.94:0/1347934904 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34dc153c80 con 0x7f34c80039c0 2024-04-25T16:52:07.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.591+0000 7f34e11e5640 1 -- 172.21.15.94:0/1347934904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f34dc155050 con 0x7f34c80039c0 2024-04-25T16:52:07.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.591+0000 7f34d37fe640 1 -- 172.21.15.94:0/1347934904 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f34cc0fe020 con 0x7f34c80039c0 2024-04-25T16:52:07.590 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:52:07.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.595+0000 7f34e11e5640 1 -- 172.21.15.94:0/1347934904 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f34bc06f430 msgr2=0x7f34bc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.595+0000 7f34e11e5640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f34bc06f430 0x7f34bc071940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34c4004660 tx=0x7f34c4008040 comp rx=0 tx=0).stop 2024-04-25T16:52:07.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.595+0000 7f34e11e5640 1 -- 172.21.15.94:0/1347934904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c80039c0 msgr2=0x7f34dc150860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.595+0000 7f34e11e5640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c80039c0 0x7f34dc150860 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f34cc01c1f0 tx=0x7f34c8005840 comp rx=0 tx=0).stop 2024-04-25T16:52:07.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.595+0000 7f34e11e5640 1 -- 172.21.15.94:0/1347934904 shutdown_connections 2024-04-25T16:52:07.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.595+0000 7f34e11e5640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34dc153330 0x7f34dc153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.595+0000 7f34e11e5640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f34bc06f430 0x7f34bc071940 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.595+0000 7f34e11e5640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc1faa20 0x7f34dc1f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.595+0000 7f34e11e5640 1 --2- 172.21.15.94:0/1347934904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c80039c0 0x7f34dc150860 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.595+0000 7f34e11e5640 1 -- 172.21.15.94:0/1347934904 >> 172.21.15.94:0/1347934904 conn(0x7f34dc0b47f0 msgr2=0x7f34dc150f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:07.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.595+0000 7f34e11e5640 1 -- 172.21.15.94:0/1347934904 shutdown_connections 2024-04-25T16:52:07.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.595+0000 7f34e11e5640 1 -- 172.21.15.94:0/1347934904 wait complete. 2024-04-25T16:52:07.611 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:52:07.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe25ddbb640 1 Processor -- start 2024-04-25T16:52:07.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe25ddbb640 1 -- start start 2024-04-25T16:52:07.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe25ddbb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe25815c7e0 0x7fe2580afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe25ddbb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2580b0330 0x7fe25806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe25ddbb640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe258074380 con 0x7fe25806dc50 2024-04-25T16:52:07.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe25ddbb640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe25806e060 con 0x7fe25815c7e0 2024-04-25T16:52:07.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe25ddbb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe2580727e0 con 0x7fe2580b0330 2024-04-25T16:52:07.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe2577fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe25815c7e0 0x7fe2580afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe2577fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe25815c7e0 0x7fe2580afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55044/0 (socket says 172.21.15.94:55044) 2024-04-25T16:52:07.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe2577fe640 1 -- 172.21.15.94:0/687464804 learned_addr learned my addr 172.21.15.94:0/687464804 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:07.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe256ffd640 1 --2- 172.21.15.94:0/687464804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2580b0330 0x7fe25806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe2577fe640 1 -- 172.21.15.94:0/687464804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2580b0330 msgr2=0x7fe25806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:07.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe2577fe640 1 --2- 172.21.15.94:0/687464804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2580b0330 0x7fe25806d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe2577fe640 1 -- 172.21.15.94:0/687464804 >> v1:172.21.15.138:6789/0 conn(0x7fe25806dc50 legacy=0x7fe25806e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:07.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.763+0000 7fe2577fe640 1 -- 172.21.15.94:0/687464804 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe258072ac0 con 0x7fe25815c7e0 2024-04-25T16:52:07.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.767+0000 7fe2577fe640 1 --2- 172.21.15.94:0/687464804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe25815c7e0 0x7fe2580afdf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe248009870 tx=0x7fe24801c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56ad75b9d18da3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:07.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.767+0000 7fe2567fc640 1 -- 172.21.15.94:0/687464804 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2480b0070 con 0x7fe25815c7e0 2024-04-25T16:52:07.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.767+0000 7fe2567fc640 1 -- 172.21.15.94:0/687464804 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2480ab600 con 0x7fe25815c7e0 2024-04-25T16:52:07.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.767+0000 7fe2567fc640 1 -- 172.21.15.94:0/687464804 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2480ab9d0 con 0x7fe25815c7e0 2024-04-25T16:52:07.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.767+0000 7fe25ddbb640 1 -- 172.21.15.94:0/687464804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe25815c7e0 msgr2=0x7fe2580afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.767+0000 7fe25ddbb640 1 --2- 172.21.15.94:0/687464804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe25815c7e0 0x7fe2580afdf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe248009870 tx=0x7fe24801c280 comp rx=0 tx=0).stop 2024-04-25T16:52:07.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.767+0000 7fe25ddbb640 1 -- 172.21.15.94:0/687464804 shutdown_connections 2024-04-25T16:52:07.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.767+0000 7fe25ddbb640 1 --2- 172.21.15.94:0/687464804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2580b0330 0x7fe25806d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.767+0000 7fe25ddbb640 1 --2- 172.21.15.94:0/687464804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe25815c7e0 0x7fe2580afdf0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.767+0000 7fe25ddbb640 1 -- 172.21.15.94:0/687464804 >> 172.21.15.94:0/687464804 conn(0x7fe2580b47f0 msgr2=0x7fe2580b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:07.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.767+0000 7fe25ddbb640 1 -- 172.21.15.94:0/687464804 shutdown_connections 2024-04-25T16:52:07.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.767+0000 7fe25ddbb640 1 -- 172.21.15.94:0/687464804 wait complete. 2024-04-25T16:52:07.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe25ddbb640 1 Processor -- start 2024-04-25T16:52:07.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe25ddbb640 1 -- start start 2024-04-25T16:52:07.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe25ddbb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe25806dc50 0x7fe25815f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe25ddbb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2580b0330 0x7fe2581fac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe25ddbb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe25815c7e0 0x7fe2581fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe25ddbb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe25806f2c0 con 0x7fe2580b0330 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe25ddbb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe258073ca0 con 0x7fe25815c7e0 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe25ddbb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe25815cbf0 con 0x7fe25806dc50 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe256ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2580b0330 0x7fe2581fac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe256ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2580b0330 0x7fe2581fac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55046/0 (socket says 172.21.15.94:55046) 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe256ffd640 1 -- 172.21.15.94:0/457316795 learned_addr learned my addr 172.21.15.94:0/457316795 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe2577fe640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe25806dc50 0x7fe25815f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe257fff640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe25815c7e0 0x7fe2581fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe256ffd640 1 -- 172.21.15.94:0/457316795 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe25806dc50 msgr2=0x7fe25815f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe256ffd640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe25806dc50 0x7fe25815f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe256ffd640 1 -- 172.21.15.94:0/457316795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe25815c7e0 msgr2=0x7fe2581fb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe256ffd640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe25815c7e0 0x7fe2581fb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe256ffd640 1 -- 172.21.15.94:0/457316795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25815fc00 con 0x7fe2580b0330 2024-04-25T16:52:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe256ffd640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2580b0330 0x7fe2581fac70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00c5e0 tx=0x7fe24c00caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:07.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe23ffff640 1 -- 172.21.15.94:0/457316795 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c015070 con 0x7fe2580b0330 2024-04-25T16:52:07.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe23ffff640 1 -- 172.21.15.94:0/457316795 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe24c004220 con 0x7fe2580b0330 2024-04-25T16:52:07.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe25ddbb640 1 -- 172.21.15.94:0/457316795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25814ef80 con 0x7fe2580b0330 2024-04-25T16:52:07.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe23ffff640 1 -- 172.21.15.94:0/457316795 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c003620 con 0x7fe2580b0330 2024-04-25T16:52:07.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe25ddbb640 1 -- 172.21.15.94:0/457316795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25814e4a0 con 0x7fe2580b0330 2024-04-25T16:52:07.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.771+0000 7fe25ddbb640 1 -- 172.21.15.94:0/457316795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe258076830 con 0x7fe2580b0330 2024-04-25T16:52:07.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.775+0000 7fe23ffff640 1 -- 172.21.15.94:0/457316795 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe24c0037c0 con 0x7fe2580b0330 2024-04-25T16:52:07.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.775+0000 7fe23ffff640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe24406f400 0x7fe244071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:07.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.775+0000 7fe23ffff640 1 -- 172.21.15.94:0/457316795 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe24c08f900 con 0x7fe2580b0330 2024-04-25T16:52:07.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.775+0000 7fe2577fe640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe24406f400 0x7fe244071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:07.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.775+0000 7fe2577fe640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe24406f400 0x7fe244071910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe2480ad000 tx=0x7fe2480a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:07.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.779+0000 7fe23ffff640 1 -- 172.21.15.94:0/457316795 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe24c05c4d0 con 0x7fe2580b0330 2024-04-25T16:52:07.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe25ddbb640 1 -- 172.21.15.94:0/457316795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe2580748a0 con 0x7fe2580b0330 2024-04-25T16:52:07.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe23ffff640 1 -- 172.21.15.94:0/457316795 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fe24c05bdd0 con 0x7fe2580b0330 2024-04-25T16:52:07.924 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:52:07.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe25ddbb640 1 -- 172.21.15.94:0/457316795 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe24406f400 msgr2=0x7fe244071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe25ddbb640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe24406f400 0x7fe244071910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe2480ad000 tx=0x7fe2480a6000 comp rx=0 tx=0).stop 2024-04-25T16:52:07.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe25ddbb640 1 -- 172.21.15.94:0/457316795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2580b0330 msgr2=0x7fe2581fac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:07.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe25ddbb640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2580b0330 0x7fe2581fac70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00c5e0 tx=0x7fe24c00caa0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe25ddbb640 1 -- 172.21.15.94:0/457316795 shutdown_connections 2024-04-25T16:52:07.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe25ddbb640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe25815c7e0 0x7fe2581fb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe25ddbb640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2580b0330 0x7fe2581fac70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe25ddbb640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe24406f400 0x7fe244071910 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe25ddbb640 1 --2- 172.21.15.94:0/457316795 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe25806dc50 0x7fe25815f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:07.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe25ddbb640 1 -- 172.21.15.94:0/457316795 >> 172.21.15.94:0/457316795 conn(0x7fe2580b47f0 msgr2=0x7fe2580b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:07.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe25ddbb640 1 -- 172.21.15.94:0/457316795 shutdown_connections 2024-04-25T16:52:07.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:07.927+0000 7fe25ddbb640 1 -- 172.21.15.94:0/457316795 wait complete. 2024-04-25T16:52:07.945 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:52:08.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.095+0000 7fa9dd273640 1 Processor -- start 2024-04-25T16:52:08.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.095+0000 7fa9dd273640 1 -- start start 2024-04-25T16:52:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.095+0000 7fa9dd273640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d806fe90 0x7fa9d80702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.095+0000 7fa9dd273640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa9d8163720 0x7fa9d8163b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.095+0000 7fa9dd273640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d806f0d0 con 0x7fa9d80cca50 2024-04-25T16:52:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.095+0000 7fa9dd273640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa9d80d03a0 con 0x7fa9d8163720 2024-04-25T16:52:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.095+0000 7fa9dd273640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d806f3b0 con 0x7fa9d806fe90 2024-04-25T16:52:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.095+0000 7fa9d67fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d806fe90 0x7fa9d80702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.095+0000 7fa9d77fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa9d8163720 0x7fa9d8163b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.099+0000 7fa9d67fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d806fe90 0x7fa9d80702a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:35970/0 (socket says 172.21.15.94:35970) 2024-04-25T16:52:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.099+0000 7fa9d67fc640 1 -- 172.21.15.94:0/2889054730 learned_addr learned my addr 172.21.15.94:0/2889054730 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.099+0000 7fa9d67fc640 1 -- 172.21.15.94:0/2889054730 >> v2:172.21.15.138:3300/0 conn(0x7fa9d8163720 msgr2=0x7fa9d8163b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.099+0000 7fa9d67fc640 1 --2- 172.21.15.94:0/2889054730 >> v2:172.21.15.138:3300/0 conn(0x7fa9d8163720 0x7fa9d8163b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.099+0000 7fa9d67fc640 1 -- 172.21.15.94:0/2889054730 >> v1:172.21.15.138:6789/0 conn(0x7fa9d80cca50 legacy=0x7fa9d80cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.099+0000 7fa9d5ffb640 1 -- 172.21.15.94:0/2889054730 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2521004506 0 0) 0x7fa9d806f0d0 con 0x7fa9d80cca50 2024-04-25T16:52:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.099+0000 7fa9d67fc640 1 -- 172.21.15.94:0/2889054730 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d806f690 con 0x7fa9d806fe90 2024-04-25T16:52:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.099+0000 7fa9d77fe640 1 --2- 172.21.15.94:0/2889054730 >> v2:172.21.15.138:3300/0 conn(0x7fa9d8163720 0x7fa9d8163b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.099+0000 7fa9d67fc640 1 --2- 172.21.15.94:0/2889054730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d806fe90 0x7fa9d80702a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009870 tx=0x7fa9cc01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2490704c7ba22e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.099+0000 7fa9d5ffb640 1 -- 172.21.15.94:0/2889054730 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0b0070 con 0x7fa9d806fe90 2024-04-25T16:52:08.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.099+0000 7fa9d5ffb640 1 -- 172.21.15.94:0/2889054730 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0ab600 con 0x7fa9d806fe90 2024-04-25T16:52:08.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.099+0000 7fa9d5ffb640 1 -- 172.21.15.94:0/2889054730 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0ab9d0 con 0x7fa9d806fe90 2024-04-25T16:52:08.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 -- 172.21.15.94:0/2889054730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d806fe90 msgr2=0x7fa9d80702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 --2- 172.21.15.94:0/2889054730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d806fe90 0x7fa9d80702a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009870 tx=0x7fa9cc01c4d0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 -- 172.21.15.94:0/2889054730 shutdown_connections 2024-04-25T16:52:08.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 --2- 172.21.15.94:0/2889054730 >> v2:172.21.15.138:3300/0 conn(0x7fa9d8163720 0x7fa9d8163b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 --2- 172.21.15.94:0/2889054730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d806fe90 0x7fa9d80702a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 -- 172.21.15.94:0/2889054730 >> 172.21.15.94:0/2889054730 conn(0x7fa9d80ce490 msgr2=0x7fa9d80ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:08.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 -- 172.21.15.94:0/2889054730 shutdown_connections 2024-04-25T16:52:08.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 -- 172.21.15.94:0/2889054730 wait complete. 2024-04-25T16:52:08.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 Processor -- start 2024-04-25T16:52:08.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 -- start start 2024-04-25T16:52:08.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d806fe90 0x7fa9d80c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9d80cca50 0x7fa9d815ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d8163720 0x7fa9d815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8070960 con 0x7fa9d8163720 2024-04-25T16:52:08.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d80707e0 con 0x7fa9d80cca50 2024-04-25T16:52:08.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9dd273640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d8164480 con 0x7fa9d806fe90 2024-04-25T16:52:08.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9d77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d8163720 0x7fa9d815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9d77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d8163720 0x7fa9d815b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48024/0 (socket says 172.21.15.94:48024) 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9d77fe640 1 -- 172.21.15.94:0/3793862504 learned_addr learned my addr 172.21.15.94:0/3793862504 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9d6ffd640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d806fe90 0x7fa9d80c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9d67fc640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9d80cca50 0x7fa9d815ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9d77fe640 1 -- 172.21.15.94:0/3793862504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d806fe90 msgr2=0x7fa9d80c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9d77fe640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d806fe90 0x7fa9d80c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9d77fe640 1 -- 172.21.15.94:0/3793862504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9d80cca50 msgr2=0x7fa9d815ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9d77fe640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9d80cca50 0x7fa9d815ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9d77fe640 1 -- 172.21.15.94:0/3793862504 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d80cbc90 con 0x7fa9d8163720 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.103+0000 7fa9d6ffd640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d806fe90 0x7fa9d80c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.107+0000 7fa9d77fe640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d8163720 0x7fa9d815b240 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000efc0 tx=0x7fa9d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.107+0000 7fa9bf7fe640 1 -- 172.21.15.94:0/3793862504 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0018030 con 0x7fa9d8163720 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.107+0000 7fa9bf7fe640 1 -- 172.21.15.94:0/3793862504 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d000cdf0 con 0x7fa9d8163720 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.107+0000 7fa9bf7fe640 1 -- 172.21.15.94:0/3793862504 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0003bc0 con 0x7fa9d8163720 2024-04-25T16:52:08.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.107+0000 7fa9dd273640 1 -- 172.21.15.94:0/3793862504 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d80c7630 con 0x7fa9d8163720 2024-04-25T16:52:08.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.107+0000 7fa9dd273640 1 -- 172.21.15.94:0/3793862504 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d80c7bf0 con 0x7fa9d8163720 2024-04-25T16:52:08.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.107+0000 7fa9dd273640 1 -- 172.21.15.94:0/3793862504 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d80c73a0 con 0x7fa9d8163720 2024-04-25T16:52:08.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.107+0000 7fa9bf7fe640 1 -- 172.21.15.94:0/3793862504 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa9d001b070 con 0x7fa9d8163720 2024-04-25T16:52:08.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.107+0000 7fa9bf7fe640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa9c406f430 0x7fa9c4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.107+0000 7fa9bf7fe640 1 -- 172.21.15.94:0/3793862504 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa9d0090690 con 0x7fa9d8163720 2024-04-25T16:52:08.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.107+0000 7fa9d6ffd640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa9c406f430 0x7fa9c4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.107+0000 7fa9d6ffd640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa9c406f430 0x7fa9c4071940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8000f70 tx=0x7fa9c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.111+0000 7fa9bf7fe640 1 -- 172.21.15.94:0/3793862504 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa9d0022070 con 0x7fa9d8163720 2024-04-25T16:52:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9dd273640 1 -- 172.21.15.94:0/3793862504 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa9d80cfa70 con 0x7fa9d8163720 2024-04-25T16:52:08.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9bf7fe640 1 -- 172.21.15.94:0/3793862504 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fa9d80cbc90 con 0x7fa9d8163720 2024-04-25T16:52:08.288 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:52:08.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9dd273640 1 -- 172.21.15.94:0/3793862504 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa9c406f430 msgr2=0x7fa9c4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9dd273640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa9c406f430 0x7fa9c4071940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8000f70 tx=0x7fa9c8009040 comp rx=0 tx=0).stop 2024-04-25T16:52:08.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9dd273640 1 -- 172.21.15.94:0/3793862504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d8163720 msgr2=0x7fa9d815b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9dd273640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d8163720 0x7fa9d815b240 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000efc0 tx=0x7fa9d000c490 comp rx=0 tx=0).stop 2024-04-25T16:52:08.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9dd273640 1 -- 172.21.15.94:0/3793862504 shutdown_connections 2024-04-25T16:52:08.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9dd273640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d8163720 0x7fa9d815b240 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9dd273640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9d80cca50 0x7fa9d815ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9dd273640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa9c406f430 0x7fa9c4071940 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9dd273640 1 --2- 172.21.15.94:0/3793862504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa9d806fe90 0x7fa9d80c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9dd273640 1 -- 172.21.15.94:0/3793862504 >> 172.21.15.94:0/3793862504 conn(0x7fa9d80ce490 msgr2=0x7fa9d806c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:08.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9dd273640 1 -- 172.21.15.94:0/3793862504 shutdown_connections 2024-04-25T16:52:08.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.291+0000 7fa9dd273640 1 -- 172.21.15.94:0/3793862504 wait complete. 2024-04-25T16:52:08.310 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T16:52:08.310 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:52:08.317 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:52:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.455+0000 7fd65aa91640 1 Processor -- start 2024-04-25T16:52:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.455+0000 7fd65aa91640 1 -- start start 2024-04-25T16:52:08.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.455+0000 7fd65aa91640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65415e990 0x7fd654165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.455+0000 7fd65aa91640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd654165b90 0x7fd654167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.455+0000 7fd65aa91640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd65406d610 con 0x7fd654070080 2024-04-25T16:52:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.455+0000 7fd65aa91640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd654158550 con 0x7fd654165b90 2024-04-25T16:52:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.455+0000 7fd65aa91640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd65406f7d0 con 0x7fd65415e990 2024-04-25T16:52:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.455+0000 7fd65902e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd654165b90 0x7fd654167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.455+0000 7fd653fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65415e990 0x7fd654165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.455+0000 7fd65902e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd654165b90 0x7fd654167fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40878/0 (socket says 172.21.15.94:40878) 2024-04-25T16:52:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.455+0000 7fd65902e640 1 -- 172.21.15.94:0/2070095975 learned_addr learned my addr 172.21.15.94:0/2070095975 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.455+0000 7fd65882d640 1 --1- 172.21.15.94:0/2070095975 >> v1:172.21.15.138:6789/0 conn(0x7fd654070080 0x7fd65415e280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:48620/0 (socket says 172.21.15.94:48620) 2024-04-25T16:52:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd653fff640 1 -- 172.21.15.94:0/2070095975 >> v2:172.21.15.138:3300/0 conn(0x7fd654165b90 msgr2=0x7fd654167fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd653fff640 1 --2- 172.21.15.94:0/2070095975 >> v2:172.21.15.138:3300/0 conn(0x7fd654165b90 0x7fd654167fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd653fff640 1 -- 172.21.15.94:0/2070095975 >> v1:172.21.15.138:6789/0 conn(0x7fd654070080 legacy=0x7fd65415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd653fff640 1 -- 172.21.15.94:0/2070095975 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65406fc60 con 0x7fd65415e990 2024-04-25T16:52:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd65902e640 1 --2- 172.21.15.94:0/2070095975 >> v2:172.21.15.138:3300/0 conn(0x7fd654165b90 0x7fd654167fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:08.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd6537fe640 1 -- 172.21.15.94:0/2070095975 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3750179925 0 0) 0x7fd648001120 con 0x7fd654070080 2024-04-25T16:52:08.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd653fff640 1 --2- 172.21.15.94:0/2070095975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65415e990 0x7fd654165650 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6541586d0 tx=0x7fd64401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59b4a4ac5e7d41fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd6537fe640 1 -- 172.21.15.94:0/2070095975 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6440ab680 con 0x7fd65415e990 2024-04-25T16:52:08.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd6537fe640 1 -- 172.21.15.94:0/2070095975 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6440b0070 con 0x7fd65415e990 2024-04-25T16:52:08.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd6537fe640 1 -- 172.21.15.94:0/2070095975 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6440aba40 con 0x7fd65415e990 2024-04-25T16:52:08.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd65aa91640 1 -- 172.21.15.94:0/2070095975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65415e990 msgr2=0x7fd654165650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd65aa91640 1 --2- 172.21.15.94:0/2070095975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65415e990 0x7fd654165650 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6541586d0 tx=0x7fd64401c480 comp rx=0 tx=0).stop 2024-04-25T16:52:08.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd65aa91640 1 -- 172.21.15.94:0/2070095975 shutdown_connections 2024-04-25T16:52:08.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd65aa91640 1 --2- 172.21.15.94:0/2070095975 >> v2:172.21.15.138:3300/0 conn(0x7fd654165b90 0x7fd654167fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd65aa91640 1 --2- 172.21.15.94:0/2070095975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65415e990 0x7fd654165650 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd65aa91640 1 -- 172.21.15.94:0/2070095975 >> 172.21.15.94:0/2070095975 conn(0x7fd6540b47f0 msgr2=0x7fd6540b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:08.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd65aa91640 1 -- 172.21.15.94:0/2070095975 shutdown_connections 2024-04-25T16:52:08.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.459+0000 7fd65aa91640 1 -- 172.21.15.94:0/2070095975 wait complete. 2024-04-25T16:52:08.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65aa91640 1 Processor -- start 2024-04-25T16:52:08.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65aa91640 1 -- start start 2024-04-25T16:52:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65aa91640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd654070080 0x7fd654150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65aa91640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65415e990 0x7fd65414fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65aa91640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd654165b90 0x7fd6540cf840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65aa91640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6540ae6c0 con 0x7fd654165b90 2024-04-25T16:52:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65aa91640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd65406f460 con 0x7fd654070080 2024-04-25T16:52:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65aa91640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd654070490 con 0x7fd65415e990 2024-04-25T16:52:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65902e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd654165b90 0x7fd6540cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65902e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd654165b90 0x7fd6540cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48040/0 (socket says 172.21.15.94:48040) 2024-04-25T16:52:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65902e640 1 -- 172.21.15.94:0/1258515479 learned_addr learned my addr 172.21.15.94:0/1258515479 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65882d640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd654070080 0x7fd654150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd653fff640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65415e990 0x7fd65414fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65902e640 1 -- 172.21.15.94:0/1258515479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65415e990 msgr2=0x7fd65414fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65902e640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65415e990 0x7fd65414fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65902e640 1 -- 172.21.15.94:0/1258515479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd654070080 msgr2=0x7fd654150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65902e640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd654070080 0x7fd654150440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65902e640 1 -- 172.21.15.94:0/1258515479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd654150e20 con 0x7fd654165b90 2024-04-25T16:52:08.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65902e640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd654165b90 0x7fd6540cf840 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00ef90 tx=0x7fd64c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd6517fa640 1 -- 172.21.15.94:0/1258515479 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c00cbe0 con 0x7fd654165b90 2024-04-25T16:52:08.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd6517fa640 1 -- 172.21.15.94:0/1258515479 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd64c00cd80 con 0x7fd654165b90 2024-04-25T16:52:08.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65aa91640 1 -- 172.21.15.94:0/1258515479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd654151100 con 0x7fd654165b90 2024-04-25T16:52:08.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd6517fa640 1 -- 172.21.15.94:0/1258515479 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c003be0 con 0x7fd654165b90 2024-04-25T16:52:08.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.463+0000 7fd65aa91640 1 -- 172.21.15.94:0/1258515479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6541516c0 con 0x7fd654165b90 2024-04-25T16:52:08.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.467+0000 7fd65aa91640 1 -- 172.21.15.94:0/1258515479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd654150b90 con 0x7fd654165b90 2024-04-25T16:52:08.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.467+0000 7fd6517fa640 1 -- 172.21.15.94:0/1258515479 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd64c01b030 con 0x7fd654165b90 2024-04-25T16:52:08.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.467+0000 7fd6517fa640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd64006f400 0x7fd640071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.467+0000 7fd6517fa640 1 -- 172.21.15.94:0/1258515479 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd64c090750 con 0x7fd654165b90 2024-04-25T16:52:08.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.467+0000 7fd65882d640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd64006f400 0x7fd640071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.467+0000 7fd65882d640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd64006f400 0x7fd640071910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd648000f70 tx=0x7fd648007d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.471+0000 7fd6517fa640 1 -- 172.21.15.94:0/1258515479 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd64c022020 con 0x7fd654165b90 2024-04-25T16:52:08.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.619+0000 7fd65aa91640 1 -- 172.21.15.94:0/1258515479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd654076830 con 0x7fd654165b90 2024-04-25T16:52:08.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.623+0000 7fd6517fa640 1 -- 172.21.15.94:0/1258515479 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7fd64c010040 con 0x7fd654165b90 2024-04-25T16:52:08.621 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:08.621 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:52:08.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.623+0000 7fd65aa91640 1 -- 172.21.15.94:0/1258515479 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd64006f400 msgr2=0x7fd640071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.623+0000 7fd65aa91640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd64006f400 0x7fd640071910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd648000f70 tx=0x7fd648007d10 comp rx=0 tx=0).stop 2024-04-25T16:52:08.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.623+0000 7fd65aa91640 1 -- 172.21.15.94:0/1258515479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd654165b90 msgr2=0x7fd6540cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.623+0000 7fd65aa91640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd654165b90 0x7fd6540cf840 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00ef90 tx=0x7fd64c00c490 comp rx=0 tx=0).stop 2024-04-25T16:52:08.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.623+0000 7fd65aa91640 1 -- 172.21.15.94:0/1258515479 shutdown_connections 2024-04-25T16:52:08.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.623+0000 7fd65aa91640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd654165b90 0x7fd6540cf840 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.623+0000 7fd65aa91640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd64006f400 0x7fd640071910 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.623+0000 7fd65aa91640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65415e990 0x7fd65414fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.623+0000 7fd65aa91640 1 --2- 172.21.15.94:0/1258515479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd654070080 0x7fd654150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.623+0000 7fd65aa91640 1 -- 172.21.15.94:0/1258515479 >> 172.21.15.94:0/1258515479 conn(0x7fd6540b47f0 msgr2=0x7fd654165fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:08.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.623+0000 7fd65aa91640 1 -- 172.21.15.94:0/1258515479 shutdown_connections 2024-04-25T16:52:08.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.623+0000 7fd65aa91640 1 -- 172.21.15.94:0/1258515479 wait complete. 2024-04-25T16:52:08.642 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:52:08.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:52:08.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.787+0000 7f59879ea640 1 Processor -- start 2024-04-25T16:52:08.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f59879ea640 1 -- start start 2024-04-25T16:52:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f59879ea640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59800b0420 0x7f598006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f59879ea640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f598015ec30 0x7f598006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f59879ea640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5980070630 con 0x7f598006f2a0 2024-04-25T16:52:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f59879ea640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f59800aeaa0 con 0x7f598015ec30 2024-04-25T16:52:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f59879ea640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f59800703e0 con 0x7f59800b0420 2024-04-25T16:52:08.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5984f85640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f598006f2a0 0x7f598006c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:43064/0 (socket says 172.21.15.94:43064) 2024-04-25T16:52:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5984f85640 1 -- 172.21.15.94:0/4247688462 learned_addr learned my addr 172.21.15.94:0/4247688462 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5985f87640 1 --2- 172.21.15.94:0/4247688462 >> v2:172.21.15.138:3300/0 conn(0x7f598015ec30 0x7f598006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5985786640 1 --2- 172.21.15.94:0/4247688462 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59800b0420 0x7f598006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1730044815 0 0) 0x7f5980070630 con 0x7f598006f2a0 2024-04-25T16:52:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f596c003530 con 0x7f598006f2a0 2024-04-25T16:52:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 721129826 0 0) 0x7f596c003530 con 0x7f598006f2a0 2024-04-25T16:52:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59800b0420 msgr2=0x7f598006ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 --2- 172.21.15.94:0/4247688462 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59800b0420 0x7f598006ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 >> v2:172.21.15.138:3300/0 conn(0x7f598015ec30 msgr2=0x7f598006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 --2- 172.21.15.94:0/4247688462 >> v2:172.21.15.138:3300/0 conn(0x7f598015ec30 0x7f598006cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598006f6b0 con 0x7f598006f2a0 2024-04-25T16:52:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f5970001680 con 0x7f598006f2a0 2024-04-25T16:52:08.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 >> v1:172.21.15.94:6789/0 conn(0x7f598006f2a0 legacy=0x7f598006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 --2- 172.21.15.94:0/4247688462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f596c0039c0 0x7f596c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 --2- 172.21.15.94:0/4247688462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f596c004810 0x7f596c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 --2- 172.21.15.94:0/4247688462 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59800b0420 0x7f596c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59800aeaa0 con 0x7f596c004810 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59800703e0 con 0x7f596c0039c0 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f596c00b9d0 con 0x7f59800b0420 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5984f85640 1 --2- 172.21.15.94:0/4247688462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f596c004810 0x7f596c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5985786640 1 --2- 172.21.15.94:0/4247688462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f596c0039c0 0x7f596c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5985f87640 1 --2- 172.21.15.94:0/4247688462 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59800b0420 0x7f596c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5984f85640 1 -- 172.21.15.94:0/4247688462 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59800b0420 msgr2=0x7f596c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5984f85640 1 --2- 172.21.15.94:0/4247688462 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59800b0420 0x7f596c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5984f85640 1 -- 172.21.15.94:0/4247688462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f596c0039c0 msgr2=0x7f596c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5984f85640 1 --2- 172.21.15.94:0/4247688462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f596c0039c0 0x7f596c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.791+0000 7f5984f85640 1 -- 172.21.15.94:0/4247688462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c003530 con 0x7f596c004810 2024-04-25T16:52:08.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.795+0000 7f5984f85640 1 --2- 172.21.15.94:0/4247688462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f596c004810 0x7f596c004c20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f59800afeb0 tx=0x7f597001f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf1bcc031798e16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.795+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970007d60 con 0x7f596c004810 2024-04-25T16:52:08.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.795+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59700ac020 con 0x7f596c004810 2024-04-25T16:52:08.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.795+0000 7f5977fff640 1 -- 172.21.15.94:0/4247688462 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59700a8e10 con 0x7f596c004810 2024-04-25T16:52:08.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.795+0000 7f59879ea640 1 -- 172.21.15.94:0/4247688462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f596c004810 msgr2=0x7f596c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.795+0000 7f59879ea640 1 --2- 172.21.15.94:0/4247688462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f596c004810 0x7f596c004c20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f59800afeb0 tx=0x7f597001f1b0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.795+0000 7f5985f87640 1 -- 172.21.15.94:0/4247688462 reap_dead start 2024-04-25T16:52:08.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.795+0000 7f59879ea640 1 -- 172.21.15.94:0/4247688462 shutdown_connections 2024-04-25T16:52:08.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.795+0000 7f59879ea640 1 -- 172.21.15.94:0/4247688462 >> 172.21.15.94:0/4247688462 conn(0x7f59800b47f0 msgr2=0x7f59800b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:08.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.795+0000 7f59879ea640 1 -- 172.21.15.94:0/4247688462 shutdown_connections 2024-04-25T16:52:08.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.795+0000 7f59879ea640 1 -- 172.21.15.94:0/4247688462 wait complete. 2024-04-25T16:52:08.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f59879ea640 1 Processor -- start 2024-04-25T16:52:08.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f59879ea640 1 -- start start 2024-04-25T16:52:08.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f59879ea640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f596c004810 0x7f59801f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f59879ea640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59800742e0 0x7f59800ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f59879ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5980160260 0x7f5980160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f59879ea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59800ae730 con 0x7f5980160260 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f59879ea640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59800ae450 con 0x7f59800742e0 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f59879ea640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f59800aef30 con 0x7f596c004810 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5985f87640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5980160260 0x7f5980160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5985f87640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5980160260 0x7f5980160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48058/0 (socket says 172.21.15.94:48058) 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5985f87640 1 -- 172.21.15.94:0/1292734938 learned_addr learned my addr 172.21.15.94:0/1292734938 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5984f85640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59800742e0 0x7f59800ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5985786640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f596c004810 0x7f59801f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5985f87640 1 -- 172.21.15.94:0/1292734938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f596c004810 msgr2=0x7f59801f9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5985f87640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f596c004810 0x7f59801f9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5985f87640 1 -- 172.21.15.94:0/1292734938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59800742e0 msgr2=0x7f59800ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5985f87640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59800742e0 0x7f59800ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5985f87640 1 -- 172.21.15.94:0/1292734938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59801fa4b0 con 0x7f5980160260 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5985f87640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5980160260 0x7f5980160670 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f59800726f0 tx=0x7f59780115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5975ffb640 1 -- 172.21.15.94:0/1292734938 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5978011e40 con 0x7f5980160260 2024-04-25T16:52:08.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5975ffb640 1 -- 172.21.15.94:0/1292734938 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f597801b070 con 0x7f5980160260 2024-04-25T16:52:08.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f59879ea640 1 -- 172.21.15.94:0/1292734938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59801fa790 con 0x7f5980160260 2024-04-25T16:52:08.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5975ffb640 1 -- 172.21.15.94:0/1292734938 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59780039a0 con 0x7f5980160260 2024-04-25T16:52:08.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f59879ea640 1 -- 172.21.15.94:0/1292734938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59800ca510 con 0x7f5980160260 2024-04-25T16:52:08.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f59879ea640 1 -- 172.21.15.94:0/1292734938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5980076830 con 0x7f5980160260 2024-04-25T16:52:08.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.799+0000 7f5975ffb640 1 -- 172.21.15.94:0/1292734938 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5978003b40 con 0x7f5980160260 2024-04-25T16:52:08.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.803+0000 7f5975ffb640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f596406f400 0x7f5964071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.803+0000 7f5975ffb640 1 -- 172.21.15.94:0/1292734938 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5978090070 con 0x7f5980160260 2024-04-25T16:52:08.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.803+0000 7f5985786640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f596406f400 0x7f5964071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.803+0000 7f5985786640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f596406f400 0x7f5964071910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f597c009160 tx=0x7f597c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.807+0000 7f5975ffb640 1 -- 172.21.15.94:0/1292734938 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5978025020 con 0x7f5980160260 2024-04-25T16:52:08.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.951+0000 7f59879ea640 1 -- 172.21.15.94:0/1292734938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f598006fb20 con 0x7f5980160260 2024-04-25T16:52:08.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.955+0000 7f5975ffb640 1 -- 172.21.15.94:0/1292734938 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f597805cb60 con 0x7f5980160260 2024-04-25T16:52:08.952 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:52:08.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.955+0000 7f59879ea640 1 -- 172.21.15.94:0/1292734938 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f596406f400 msgr2=0x7f5964071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.955+0000 7f59879ea640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f596406f400 0x7f5964071910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f597c009160 tx=0x7f597c008040 comp rx=0 tx=0).stop 2024-04-25T16:52:08.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.955+0000 7f59879ea640 1 -- 172.21.15.94:0/1292734938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5980160260 msgr2=0x7f5980160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.955+0000 7f59879ea640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5980160260 0x7f5980160670 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f59800726f0 tx=0x7f59780115a0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.955+0000 7f59879ea640 1 -- 172.21.15.94:0/1292734938 shutdown_connections 2024-04-25T16:52:08.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.955+0000 7f59879ea640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5980160260 0x7f5980160670 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.955+0000 7f59879ea640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59800742e0 0x7f59800ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.955+0000 7f59879ea640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f596406f400 0x7f5964071910 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.955+0000 7f59879ea640 1 --2- 172.21.15.94:0/1292734938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f596c004810 0x7f59801f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.955+0000 7f59879ea640 1 -- 172.21.15.94:0/1292734938 >> 172.21.15.94:0/1292734938 conn(0x7f59800b47f0 msgr2=0x7f598015b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:08.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.955+0000 7f59879ea640 1 -- 172.21.15.94:0/1292734938 shutdown_connections 2024-04-25T16:52:08.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:08.955+0000 7f59879ea640 1 -- 172.21.15.94:0/1292734938 wait complete. 2024-04-25T16:52:08.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:52:09.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a9e39640 1 Processor -- start 2024-04-25T16:52:09.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a9e39640 1 -- start start 2024-04-25T16:52:09.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a9e39640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f60a406f470 0x7f60a415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a9e39640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a415e160 0x7f60a415e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a9e39640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a4070630 con 0x7f60a40b0420 2024-04-25T16:52:09.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a9e39640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f60a40aeaa0 con 0x7f60a406f470 2024-04-25T16:52:09.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a9e39640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f60a40703e0 con 0x7f60a415e160 2024-04-25T16:52:09.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a37fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f60a40b0420 0x7f60a406ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:43078/0 (socket says 172.21.15.94:43078) 2024-04-25T16:52:09.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a37fe640 1 -- 172.21.15.94:0/565440048 learned_addr learned my addr 172.21.15.94:0/565440048 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:09.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a2ffd640 1 --2- 172.21.15.94:0/565440048 >> v2:172.21.15.94:3300/0 conn(0x7f60a406f470 0x7f60a415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a2ffd640 1 -- 172.21.15.94:0/565440048 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a415e160 msgr2=0x7f60a415e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:09.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a2ffd640 1 --2- 172.21.15.94:0/565440048 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a415e160 0x7f60a415e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a2ffd640 1 -- 172.21.15.94:0/565440048 >> v1:172.21.15.94:6789/0 conn(0x7f60a40b0420 legacy=0x7f60a406ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.127+0000 7f60a2ffd640 1 -- 172.21.15.94:0/565440048 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a40ae170 con 0x7f60a406f470 2024-04-25T16:52:09.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.131+0000 7f60a2ffd640 1 --2- 172.21.15.94:0/565440048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a406f470 0x7f60a415dc20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f60980097c0 tx=0x7f609801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e62eda328341f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:09.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.131+0000 7f60a27fc640 1 -- 172.21.15.94:0/565440048 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60980a7070 con 0x7f60a406f470 2024-04-25T16:52:09.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.131+0000 7f60a27fc640 1 -- 172.21.15.94:0/565440048 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60980b0070 con 0x7f60a406f470 2024-04-25T16:52:09.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.131+0000 7f60a27fc640 1 -- 172.21.15.94:0/565440048 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60980ab890 con 0x7f60a406f470 2024-04-25T16:52:09.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.131+0000 7f60a9e39640 1 -- 172.21.15.94:0/565440048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a406f470 msgr2=0x7f60a415dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.131+0000 7f60a9e39640 1 --2- 172.21.15.94:0/565440048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a406f470 0x7f60a415dc20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f60980097c0 tx=0x7f609801c470 comp rx=0 tx=0).stop 2024-04-25T16:52:09.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.131+0000 7f60a9e39640 1 -- 172.21.15.94:0/565440048 shutdown_connections 2024-04-25T16:52:09.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.131+0000 7f60a9e39640 1 --2- 172.21.15.94:0/565440048 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a415e160 0x7f60a415e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.131+0000 7f60a9e39640 1 --2- 172.21.15.94:0/565440048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a406f470 0x7f60a415dc20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.131+0000 7f60a9e39640 1 -- 172.21.15.94:0/565440048 >> 172.21.15.94:0/565440048 conn(0x7f60a40b47f0 msgr2=0x7f60a40b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:09.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.131+0000 7f60a9e39640 1 -- 172.21.15.94:0/565440048 shutdown_connections 2024-04-25T16:52:09.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.131+0000 7f60a9e39640 1 -- 172.21.15.94:0/565440048 wait complete. 2024-04-25T16:52:09.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a9e39640 1 Processor -- start 2024-04-25T16:52:09.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a9e39640 1 -- start start 2024-04-25T16:52:09.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a9e39640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a40b0420 0x7f60a414b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a9e39640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a415e160 0x7f60a41490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a9e39640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60a4157ad0 0x7f60a4157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a9e39640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60a40b09b0 con 0x7f60a40b0420 2024-04-25T16:52:09.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a9e39640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60a40b0830 con 0x7f60a4157ad0 2024-04-25T16:52:09.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a9e39640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f60a416a6f0 con 0x7f60a415e160 2024-04-25T16:52:09.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a40b0420 0x7f60a414b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a40b0420 0x7f60a414b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48072/0 (socket says 172.21.15.94:48072) 2024-04-25T16:52:09.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a37fe640 1 -- 172.21.15.94:0/2351412629 learned_addr learned my addr 172.21.15.94:0/2351412629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:09.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a2ffd640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a415e160 0x7f60a41490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a3fff640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60a4157ad0 0x7f60a4157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a37fe640 1 -- 172.21.15.94:0/2351412629 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a415e160 msgr2=0x7f60a41490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a37fe640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a415e160 0x7f60a41490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a37fe640 1 -- 172.21.15.94:0/2351412629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60a4157ad0 msgr2=0x7f60a4157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a37fe640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60a4157ad0 0x7f60a4157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a37fe640 1 -- 172.21.15.94:0/2351412629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a4158420 con 0x7f60a40b0420 2024-04-25T16:52:09.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a37fe640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a40b0420 0x7f60a414b9d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f609400c370 tx=0x7f609400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:09.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f608bfff640 1 -- 172.21.15.94:0/2351412629 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094017070 con 0x7f60a40b0420 2024-04-25T16:52:09.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f608bfff640 1 -- 172.21.15.94:0/2351412629 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6094005cf0 con 0x7f60a40b0420 2024-04-25T16:52:09.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a9e39640 1 -- 172.21.15.94:0/2351412629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a4150150 con 0x7f60a40b0420 2024-04-25T16:52:09.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f608bfff640 1 -- 172.21.15.94:0/2351412629 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940072f0 con 0x7f60a40b0420 2024-04-25T16:52:09.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.135+0000 7f60a9e39640 1 -- 172.21.15.94:0/2351412629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a414d760 con 0x7f60a40b0420 2024-04-25T16:52:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.139+0000 7f60a9e39640 1 -- 172.21.15.94:0/2351412629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a4076830 con 0x7f60a40b0420 2024-04-25T16:52:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.139+0000 7f608bfff640 1 -- 172.21.15.94:0/2351412629 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f609401a070 con 0x7f60a40b0420 2024-04-25T16:52:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.139+0000 7f608bfff640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f609006f430 0x7f6090071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.139+0000 7f608bfff640 1 -- 172.21.15.94:0/2351412629 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f609408e9b0 con 0x7f60a40b0420 2024-04-25T16:52:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.139+0000 7f60a2ffd640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f609006f430 0x7f6090071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.139+0000 7f60a2ffd640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f609006f430 0x7f6090071940 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f60980af000 tx=0x7f6098008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:09.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.143+0000 7f608bfff640 1 -- 172.21.15.94:0/2351412629 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f609405b530 con 0x7f60a40b0420 2024-04-25T16:52:09.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.299+0000 7f60a9e39640 1 -- 172.21.15.94:0/2351412629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f60a415d330 con 0x7f60a40b0420 2024-04-25T16:52:09.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.303+0000 7f608bfff640 1 -- 172.21.15.94:0/2351412629 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f609405ae30 con 0x7f60a40b0420 2024-04-25T16:52:09.300 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:52:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.303+0000 7f60a9e39640 1 -- 172.21.15.94:0/2351412629 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f609006f430 msgr2=0x7f6090071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.303+0000 7f60a9e39640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f609006f430 0x7f6090071940 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f60980af000 tx=0x7f6098008040 comp rx=0 tx=0).stop 2024-04-25T16:52:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.303+0000 7f60a9e39640 1 -- 172.21.15.94:0/2351412629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a40b0420 msgr2=0x7f60a414b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.303+0000 7f60a9e39640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a40b0420 0x7f60a414b9d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f609400c370 tx=0x7f609400c830 comp rx=0 tx=0).stop 2024-04-25T16:52:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.303+0000 7f60a9e39640 1 -- 172.21.15.94:0/2351412629 shutdown_connections 2024-04-25T16:52:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.303+0000 7f60a9e39640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60a4157ad0 0x7f60a4157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.303+0000 7f60a9e39640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f609006f430 0x7f6090071940 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.303+0000 7f60a9e39640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a415e160 0x7f60a41490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.303+0000 7f60a9e39640 1 --2- 172.21.15.94:0/2351412629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a40b0420 0x7f60a414b9d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.303+0000 7f60a9e39640 1 -- 172.21.15.94:0/2351412629 >> 172.21.15.94:0/2351412629 conn(0x7f60a40b47f0 msgr2=0x7f60a415f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.303+0000 7f60a9e39640 1 -- 172.21.15.94:0/2351412629 shutdown_connections 2024-04-25T16:52:09.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.303+0000 7f60a9e39640 1 -- 172.21.15.94:0/2351412629 wait complete. 2024-04-25T16:52:09.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:52:09.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5496ca9640 1 Processor -- start 2024-04-25T16:52:09.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5496ca9640 1 -- start start 2024-04-25T16:52:09.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5496ca9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f54900cca50 0x7f54900cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5496ca9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f549006fe90 0x7f54900702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5496ca9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f549006f0d0 con 0x7f549006c7e0 2024-04-25T16:52:09.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5496ca9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f549006f3b0 con 0x7f54900cca50 2024-04-25T16:52:09.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5496ca9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f549006f690 con 0x7f549006fe90 2024-04-25T16:52:09.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5494a45640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f54900cca50 0x7f54900cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5494a45640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f54900cca50 0x7f54900cfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48076/0 (socket says 172.21.15.94:48076) 2024-04-25T16:52:09.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5494a45640 1 -- 172.21.15.94:0/478538631 learned_addr learned my addr 172.21.15.94:0/478538631 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:09.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5494a45640 1 -- 172.21.15.94:0/478538631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f549006fe90 msgr2=0x7f54900702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:09.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5494a45640 1 --2- 172.21.15.94:0/478538631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f549006fe90 0x7f54900702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5494a45640 1 -- 172.21.15.94:0/478538631 >> v1:172.21.15.94:6789/0 conn(0x7f549006c7e0 legacy=0x7f549006cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.471+0000 7f5494a45640 1 -- 172.21.15.94:0/478538631 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549006f970 con 0x7f54900cca50 2024-04-25T16:52:09.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.475+0000 7f5494a45640 1 --2- 172.21.15.94:0/478538631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54900cca50 0x7f54900cfc90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5480009870 tx=0x7f548001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5ba9199a099b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:09.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.475+0000 7f548f7fe640 1 -- 172.21.15.94:0/478538631 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54800b0070 con 0x7f54900cca50 2024-04-25T16:52:09.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.475+0000 7f548f7fe640 1 -- 172.21.15.94:0/478538631 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54800ab600 con 0x7f54900cca50 2024-04-25T16:52:09.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.475+0000 7f548f7fe640 1 -- 172.21.15.94:0/478538631 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54800ab9d0 con 0x7f54900cca50 2024-04-25T16:52:09.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.475+0000 7f5496ca9640 1 -- 172.21.15.94:0/478538631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54900cca50 msgr2=0x7f54900cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.475+0000 7f5496ca9640 1 --2- 172.21.15.94:0/478538631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54900cca50 0x7f54900cfc90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5480009870 tx=0x7f548001c4d0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.475+0000 7f5496ca9640 1 -- 172.21.15.94:0/478538631 shutdown_connections 2024-04-25T16:52:09.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.475+0000 7f5496ca9640 1 --2- 172.21.15.94:0/478538631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f549006fe90 0x7f54900702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.475+0000 7f5496ca9640 1 --2- 172.21.15.94:0/478538631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54900cca50 0x7f54900cfc90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.475+0000 7f5496ca9640 1 -- 172.21.15.94:0/478538631 >> 172.21.15.94:0/478538631 conn(0x7f54900ce490 msgr2=0x7f54900ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:09.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.475+0000 7f5496ca9640 1 -- 172.21.15.94:0/478538631 shutdown_connections 2024-04-25T16:52:09.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.475+0000 7f5496ca9640 1 -- 172.21.15.94:0/478538631 wait complete. 2024-04-25T16:52:09.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5496ca9640 1 Processor -- start 2024-04-25T16:52:09.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5496ca9640 1 -- start start 2024-04-25T16:52:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5496ca9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f549006c7e0 0x7f54900c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5496ca9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f549006fe90 0x7f5490158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5496ca9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54900cca50 0x7f549015a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5496ca9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f54900d01d0 con 0x7f549006c7e0 2024-04-25T16:52:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5496ca9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f549006d710 con 0x7f549006fe90 2024-04-25T16:52:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5496ca9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f54900d0350 con 0x7f54900cca50 2024-04-25T16:52:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5494a45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f549006c7e0 0x7f54900c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5494a45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f549006c7e0 0x7f54900c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48086/0 (socket says 172.21.15.94:48086) 2024-04-25T16:52:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5494a45640 1 -- 172.21.15.94:0/4087674771 learned_addr learned my addr 172.21.15.94:0/4087674771 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5495246640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54900cca50 0x7f549015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f548ffff640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f549006fe90 0x7f5490158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5494a45640 1 -- 172.21.15.94:0/4087674771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54900cca50 msgr2=0x7f549015a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5494a45640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54900cca50 0x7f549015a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5494a45640 1 -- 172.21.15.94:0/4087674771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f549006fe90 msgr2=0x7f5490158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5494a45640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f549006fe90 0x7f5490158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5494a45640 1 -- 172.21.15.94:0/4087674771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5490160890 con 0x7f549006c7e0 2024-04-25T16:52:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f548ffff640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f549006fe90 0x7f5490158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5494a45640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f549006c7e0 0x7f54900c7570 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f548001a100 tx=0x7f548001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f548d7fa640 1 -- 172.21.15.94:0/4087674771 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54800b0040 con 0x7f549006c7e0 2024-04-25T16:52:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f548d7fa640 1 -- 172.21.15.94:0/4087674771 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54800bc070 con 0x7f549006c7e0 2024-04-25T16:52:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5496ca9640 1 -- 172.21.15.94:0/4087674771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5490160b70 con 0x7f549006c7e0 2024-04-25T16:52:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f548d7fa640 1 -- 172.21.15.94:0/4087674771 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54800b6880 con 0x7f549006c7e0 2024-04-25T16:52:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5496ca9640 1 -- 172.21.15.94:0/4087674771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5490161130 con 0x7f549006c7e0 2024-04-25T16:52:09.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.479+0000 7f5496ca9640 1 -- 172.21.15.94:0/4087674771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f549006a730 con 0x7f549006c7e0 2024-04-25T16:52:09.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.483+0000 7f548d7fa640 1 -- 172.21.15.94:0/4087674771 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f54800ab480 con 0x7f549006c7e0 2024-04-25T16:52:09.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.483+0000 7f548d7fa640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f547c06f400 0x7f547c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.483+0000 7f548ffff640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f547c06f400 0x7f547c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.483+0000 7f548d7fa640 1 -- 172.21.15.94:0/4087674771 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f548012a4b0 con 0x7f549006c7e0 2024-04-25T16:52:09.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.483+0000 7f548ffff640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f547c06f400 0x7f547c071910 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f54901628e0 tx=0x7f5484008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:09.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.487+0000 7f548d7fa640 1 -- 172.21.15.94:0/4087674771 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5480016150 con 0x7f549006c7e0 2024-04-25T16:52:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.647+0000 7f5496ca9640 1 -- 172.21.15.94:0/4087674771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f54900cfa70 con 0x7f549006c7e0 2024-04-25T16:52:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.647+0000 7f548d7fa640 1 -- 172.21.15.94:0/4087674771 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f54800f7080 con 0x7f549006c7e0 2024-04-25T16:52:09.645 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:52:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.651+0000 7f5496ca9640 1 -- 172.21.15.94:0/4087674771 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f547c06f400 msgr2=0x7f547c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.651+0000 7f5496ca9640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f547c06f400 0x7f547c071910 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f54901628e0 tx=0x7f5484008040 comp rx=0 tx=0).stop 2024-04-25T16:52:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.651+0000 7f5496ca9640 1 -- 172.21.15.94:0/4087674771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f549006c7e0 msgr2=0x7f54900c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.651+0000 7f5496ca9640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f549006c7e0 0x7f54900c7570 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f548001a100 tx=0x7f548001ad10 comp rx=0 tx=0).stop 2024-04-25T16:52:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.651+0000 7f5496ca9640 1 -- 172.21.15.94:0/4087674771 shutdown_connections 2024-04-25T16:52:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.651+0000 7f5496ca9640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f547c06f400 0x7f547c071910 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.651+0000 7f5496ca9640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54900cca50 0x7f549015a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.651+0000 7f5496ca9640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f549006fe90 0x7f5490158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.651+0000 7f5496ca9640 1 --2- 172.21.15.94:0/4087674771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f549006c7e0 0x7f54900c7570 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.651+0000 7f5496ca9640 1 -- 172.21.15.94:0/4087674771 >> 172.21.15.94:0/4087674771 conn(0x7f54900ce490 msgr2=0x7f54900cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.651+0000 7f5496ca9640 1 -- 172.21.15.94:0/4087674771 shutdown_connections 2024-04-25T16:52:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.651+0000 7f5496ca9640 1 -- 172.21.15.94:0/4087674771 wait complete. 2024-04-25T16:52:09.666 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:52:09.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.819+0000 7f0835043640 1 Processor -- start 2024-04-25T16:52:09.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.819+0000 7f0835043640 1 -- start start 2024-04-25T16:52:09.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.819+0000 7f0835043640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f08300d0960 0x7f0830160b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.819+0000 7f0835043640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0830161050 con 0x7f08300b73d0 2024-04-25T16:52:09.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.819+0000 7f0835043640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08300d0d70 con 0x7f08300cea70 2024-04-25T16:52:09.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.819+0000 7f0835043640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f08301463c0 con 0x7f08300d0960 2024-04-25T16:52:09.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.819+0000 7f082ed9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f08300d0960 0x7f0830160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.819+0000 7f082ed9d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f08300d0960 0x7f0830160b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48092/0 (socket says 172.21.15.94:48092) 2024-04-25T16:52:09.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.819+0000 7f082ed9d640 1 -- 172.21.15.94:0/2668658761 learned_addr learned my addr 172.21.15.94:0/2668658761 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:09.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.819+0000 7f082ed9d640 1 -- 172.21.15.94:0/2668658761 >> v1:172.21.15.138:6789/0 conn(0x7f08300cea70 legacy=0x7f08300cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.819+0000 7f082ed9d640 1 -- 172.21.15.94:0/2668658761 >> v1:172.21.15.94:6789/0 conn(0x7f08300b73d0 legacy=0x7f08300b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.819+0000 7f082ed9d640 1 -- 172.21.15.94:0/2668658761 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08300cd1d0 con 0x7f08300d0960 2024-04-25T16:52:09.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.823+0000 7f082ed9d640 1 --2- 172.21.15.94:0/2668658761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08300d0960 0x7f0830160b10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0828009e50 tx=0x7f082801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446edb6702526d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:09.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.823+0000 7f082dd9b640 1 -- 172.21.15.94:0/2668658761 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08280a5660 con 0x7f08300d0960 2024-04-25T16:52:09.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.823+0000 7f082dd9b640 1 -- 172.21.15.94:0/2668658761 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08280a5800 con 0x7f08300d0960 2024-04-25T16:52:09.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.823+0000 7f082dd9b640 1 -- 172.21.15.94:0/2668658761 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08280a5bd0 con 0x7f08300d0960 2024-04-25T16:52:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 -- 172.21.15.94:0/2668658761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08300d0960 msgr2=0x7f0830160b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 --2- 172.21.15.94:0/2668658761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08300d0960 0x7f0830160b10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0828009e50 tx=0x7f082801c9a0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 -- 172.21.15.94:0/2668658761 shutdown_connections 2024-04-25T16:52:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 --2- 172.21.15.94:0/2668658761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08300d0960 0x7f0830160b10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 -- 172.21.15.94:0/2668658761 >> 172.21.15.94:0/2668658761 conn(0x7f08300d33d0 msgr2=0x7f08300d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 -- 172.21.15.94:0/2668658761 shutdown_connections 2024-04-25T16:52:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 -- 172.21.15.94:0/2668658761 wait complete. 2024-04-25T16:52:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 Processor -- start 2024-04-25T16:52:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 -- start start 2024-04-25T16:52:09.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08300b73d0 0x7f0830141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08300cea70 0x7f0830156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08300d0960 0x7f0830156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f08300cce50 con 0x7f08300b73d0 2024-04-25T16:52:09.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08300cc870 con 0x7f08300d0960 2024-04-25T16:52:09.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f0835043640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f083016e650 con 0x7f08300cea70 2024-04-25T16:52:09.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f082ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08300b73d0 0x7f0830141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f082ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08300b73d0 0x7f0830141ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48106/0 (socket says 172.21.15.94:48106) 2024-04-25T16:52:09.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.827+0000 7f082ed9d640 1 -- 172.21.15.94:0/2910975636 learned_addr learned my addr 172.21.15.94:0/2910975636 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:09.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f082e59c640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08300cea70 0x7f0830156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f082f59e640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08300d0960 0x7f0830156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f082ed9d640 1 -- 172.21.15.94:0/2910975636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08300cea70 msgr2=0x7f0830156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f082ed9d640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08300cea70 0x7f0830156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f082ed9d640 1 -- 172.21.15.94:0/2910975636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08300d0960 msgr2=0x7f0830156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f082ed9d640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08300d0960 0x7f0830156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f082ed9d640 1 -- 172.21.15.94:0/2910975636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0830159a30 con 0x7f08300b73d0 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f082f59e640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08300d0960 0x7f0830156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f082ed9d640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08300b73d0 0x7f0830141ea0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f08280af040 tx=0x7f0828019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f08177fe640 1 -- 172.21.15.94:0/2910975636 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08280bc020 con 0x7f08300b73d0 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f08177fe640 1 -- 172.21.15.94:0/2910975636 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08280b8670 con 0x7f08300b73d0 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f08177fe640 1 -- 172.21.15.94:0/2910975636 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08280b8a40 con 0x7f08300b73d0 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f0835043640 1 -- 172.21.15.94:0/2910975636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0830159d10 con 0x7f08300b73d0 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f0835043640 1 -- 172.21.15.94:0/2910975636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083015a2d0 con 0x7f08300b73d0 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f0835043640 1 -- 172.21.15.94:0/2910975636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083006a730 con 0x7f08300b73d0 2024-04-25T16:52:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f08177fe640 1 -- 172.21.15.94:0/2910975636 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f08280bf020 con 0x7f08300b73d0 2024-04-25T16:52:09.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f08177fe640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f081c06f400 0x7f081c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:09.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f08177fe640 1 -- 172.21.15.94:0/2910975636 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f082812a800 con 0x7f08300b73d0 2024-04-25T16:52:09.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f082e59c640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f081c06f400 0x7f081c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:09.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.831+0000 7f082e59c640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f081c06f400 0x7f081c071910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0830158a90 tx=0x7f0824009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:09.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.835+0000 7f08177fe640 1 -- 172.21.15.94:0/2910975636 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f08280c6030 con 0x7f08300b73d0 2024-04-25T16:52:09.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.995+0000 7f0835043640 1 -- 172.21.15.94:0/2910975636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f08300d1290 con 0x7f08300b73d0 2024-04-25T16:52:09.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.995+0000 7f08177fe640 1 -- 172.21.15.94:0/2910975636 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f0828016620 con 0x7f08300b73d0 2024-04-25T16:52:09.994 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:52:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.999+0000 7f0835043640 1 -- 172.21.15.94:0/2910975636 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f081c06f400 msgr2=0x7f081c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.999+0000 7f0835043640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f081c06f400 0x7f081c071910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0830158a90 tx=0x7f0824009040 comp rx=0 tx=0).stop 2024-04-25T16:52:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.999+0000 7f0835043640 1 -- 172.21.15.94:0/2910975636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08300b73d0 msgr2=0x7f0830141ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.999+0000 7f0835043640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08300b73d0 0x7f0830141ea0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f08280af040 tx=0x7f0828019740 comp rx=0 tx=0).stop 2024-04-25T16:52:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.999+0000 7f0835043640 1 -- 172.21.15.94:0/2910975636 shutdown_connections 2024-04-25T16:52:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.999+0000 7f0835043640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08300d0960 0x7f0830156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.999+0000 7f0835043640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f081c06f400 0x7f081c071910 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.999+0000 7f0835043640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08300cea70 0x7f0830156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.999+0000 7f0835043640 1 --2- 172.21.15.94:0/2910975636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08300b73d0 0x7f0830141ea0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:09.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.999+0000 7f0835043640 1 -- 172.21.15.94:0/2910975636 >> 172.21.15.94:0/2910975636 conn(0x7f08300d33d0 msgr2=0x7f0830157ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:09.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.999+0000 7f0835043640 1 -- 172.21.15.94:0/2910975636 shutdown_connections 2024-04-25T16:52:09.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:09.999+0000 7f0835043640 1 -- 172.21.15.94:0/2910975636 wait complete. 2024-04-25T16:52:10.017 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:52:10.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0fb5c26640 1 Processor -- start 2024-04-25T16:52:10.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0fb5c26640 1 -- start start 2024-04-25T16:52:10.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0fb5c26640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0fb0161900 0x7f0fb0161d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0fb5c26640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fb015f6e0 0x7f0fb015faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0fb5c26640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fb015ef20 con 0x7f0fb00701b0 2024-04-25T16:52:10.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0fb5c26640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f0fb015e680 con 0x7f0fb0161900 2024-04-25T16:52:10.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0fb5c26640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb015e960 con 0x7f0fb015f6e0 2024-04-25T16:52:10.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0faeffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0fb0161900 0x7f0fb0161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0faeffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0fb0161900 0x7f0fb0161d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48112/0 (socket says 172.21.15.94:48112) 2024-04-25T16:52:10.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0faeffd640 1 -- 172.21.15.94:0/1794277483 learned_addr learned my addr 172.21.15.94:0/1794277483 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:10.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0faeffd640 1 -- 172.21.15.94:0/1794277483 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fb015f6e0 msgr2=0x7f0fb015faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:10.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0faeffd640 1 --2- 172.21.15.94:0/1794277483 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fb015f6e0 0x7f0fb015faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0faeffd640 1 -- 172.21.15.94:0/1794277483 >> v1:172.21.15.138:6789/0 conn(0x7f0fb00701b0 legacy=0x7f0fb00705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:10.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.155+0000 7f0faeffd640 1 -- 172.21.15.94:0/1794277483 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb015ec40 con 0x7f0fb0161900 2024-04-25T16:52:10.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.159+0000 7f0faeffd640 1 --2- 172.21.15.94:0/1794277483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb0161900 0x7f0fb0161d10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009870 tx=0x7f0fa401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332fb03232f33c41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:10.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.159+0000 7f0fae7fc640 1 -- 172.21.15.94:0/1794277483 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40aa650 con 0x7f0fb0161900 2024-04-25T16:52:10.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.159+0000 7f0fae7fc640 1 -- 172.21.15.94:0/1794277483 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa40af070 con 0x7f0fb0161900 2024-04-25T16:52:10.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.159+0000 7f0fae7fc640 1 -- 172.21.15.94:0/1794277483 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40aaa10 con 0x7f0fb0161900 2024-04-25T16:52:10.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.159+0000 7f0fb5c26640 1 -- 172.21.15.94:0/1794277483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb0161900 msgr2=0x7f0fb0161d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.159+0000 7f0fb5c26640 1 --2- 172.21.15.94:0/1794277483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb0161900 0x7f0fb0161d10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009870 tx=0x7f0fa401c510 comp rx=0 tx=0).stop 2024-04-25T16:52:10.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.159+0000 7f0fb5c26640 1 -- 172.21.15.94:0/1794277483 shutdown_connections 2024-04-25T16:52:10.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.159+0000 7f0fb5c26640 1 --2- 172.21.15.94:0/1794277483 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fb015f6e0 0x7f0fb015faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.159+0000 7f0fb5c26640 1 --2- 172.21.15.94:0/1794277483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb0161900 0x7f0fb0161d10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.159+0000 7f0fb5c26640 1 -- 172.21.15.94:0/1794277483 >> 172.21.15.94:0/1794277483 conn(0x7f0fb0071f00 msgr2=0x7f0fb0073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:10.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.159+0000 7f0fb5c26640 1 -- 172.21.15.94:0/1794277483 shutdown_connections 2024-04-25T16:52:10.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.159+0000 7f0fb5c26640 1 -- 172.21.15.94:0/1794277483 wait complete. 2024-04-25T16:52:10.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0fb5c26640 1 Processor -- start 2024-04-25T16:52:10.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0fb5c26640 1 -- start start 2024-04-25T16:52:10.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0fb5c26640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fb00701b0 0x7f0fb013ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0fb5c26640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fb015f6e0 0x7f0fb013f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0fb5c26640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb0161900 0x7f0fb012e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0fb5c26640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb016aa20 con 0x7f0fb0161900 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0fb5c26640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb016a8a0 con 0x7f0fb015f6e0 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0fb5c26640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb016aba0 con 0x7f0fb00701b0 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0faffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb0161900 0x7f0fb012e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0faffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb0161900 0x7f0fb012e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48124/0 (socket says 172.21.15.94:48124) 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0faffff640 1 -- 172.21.15.94:0/2232093996 learned_addr learned my addr 172.21.15.94:0/2232093996 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0faf7fe640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fb00701b0 0x7f0fb013ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0faeffd640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fb015f6e0 0x7f0fb013f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0faffff640 1 -- 172.21.15.94:0/2232093996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fb00701b0 msgr2=0x7f0fb013ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0faffff640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fb00701b0 0x7f0fb013ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0faffff640 1 -- 172.21.15.94:0/2232093996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fb015f6e0 msgr2=0x7f0fb013f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0faffff640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fb015f6e0 0x7f0fb013f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0faffff640 1 -- 172.21.15.94:0/2232093996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb0144690 con 0x7f0fb0161900 2024-04-25T16:52:10.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0faffff640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb0161900 0x7f0fb012e270 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000b490 tx=0x7f0fa000b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:10.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0f97fff640 1 -- 172.21.15.94:0/2232093996 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0015020 con 0x7f0fb0161900 2024-04-25T16:52:10.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0f97fff640 1 -- 172.21.15.94:0/2232093996 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa0011830 con 0x7f0fb0161900 2024-04-25T16:52:10.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0fb5c26640 1 -- 172.21.15.94:0/2232093996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb013da50 con 0x7f0fb0161900 2024-04-25T16:52:10.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0f97fff640 1 -- 172.21.15.94:0/2232093996 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0011c20 con 0x7f0fb0161900 2024-04-25T16:52:10.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0fb5c26640 1 -- 172.21.15.94:0/2232093996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb0142e10 con 0x7f0fb0161900 2024-04-25T16:52:10.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.163+0000 7f0fb5c26640 1 -- 172.21.15.94:0/2232093996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb013d7c0 con 0x7f0fb0161900 2024-04-25T16:52:10.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.167+0000 7f0f97fff640 1 -- 172.21.15.94:0/2232093996 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0fa0018020 con 0x7f0fb0161900 2024-04-25T16:52:10.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.167+0000 7f0f97fff640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0f9c06f430 0x7f0f9c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.167+0000 7f0faf7fe640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0f9c06f430 0x7f0f9c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.167+0000 7f0f97fff640 1 -- 172.21.15.94:0/2232093996 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0fa008f5f0 con 0x7f0fb0161900 2024-04-25T16:52:10.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.167+0000 7f0faf7fe640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0f9c06f430 0x7f0f9c071940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0074b90 tx=0x7f0fa80094d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:10.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.171+0000 7f0f97fff640 1 -- 172.21.15.94:0/2232093996 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0fa001f020 con 0x7f0fb0161900 2024-04-25T16:52:10.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.335+0000 7f0fb5c26640 1 -- 172.21.15.94:0/2232093996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0fb006a730 con 0x7f0fb0161900 2024-04-25T16:52:10.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.335+0000 7f0f97fff640 1 -- 172.21.15.94:0/2232093996 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f0fa005c040 con 0x7f0fb0161900 2024-04-25T16:52:10.335 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-04-25T16:52:10.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.339+0000 7f0fb5c26640 1 -- 172.21.15.94:0/2232093996 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0f9c06f430 msgr2=0x7f0f9c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.339+0000 7f0fb5c26640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0f9c06f430 0x7f0f9c071940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0074b90 tx=0x7f0fa80094d0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.339+0000 7f0fb5c26640 1 -- 172.21.15.94:0/2232093996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb0161900 msgr2=0x7f0fb012e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.339+0000 7f0fb5c26640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb0161900 0x7f0fb012e270 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000b490 tx=0x7f0fa000b950 comp rx=0 tx=0).stop 2024-04-25T16:52:10.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.339+0000 7f0fb5c26640 1 -- 172.21.15.94:0/2232093996 shutdown_connections 2024-04-25T16:52:10.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.339+0000 7f0fb5c26640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb0161900 0x7f0fb012e270 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.339+0000 7f0fb5c26640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fb015f6e0 0x7f0fb013f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.339+0000 7f0fb5c26640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0f9c06f430 0x7f0f9c071940 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.339+0000 7f0fb5c26640 1 --2- 172.21.15.94:0/2232093996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fb00701b0 0x7f0fb013ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.339+0000 7f0fb5c26640 1 -- 172.21.15.94:0/2232093996 >> 172.21.15.94:0/2232093996 conn(0x7f0fb0071f00 msgr2=0x7f0fb00ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:10.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.339+0000 7f0fb5c26640 1 -- 172.21.15.94:0/2232093996 shutdown_connections 2024-04-25T16:52:10.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.339+0000 7f0fb5c26640 1 -- 172.21.15.94:0/2232093996 wait complete. 2024-04-25T16:52:10.356 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-04-25T16:52:10.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:10.356 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T16:52:10.371 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:52:10.371 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T16:52:10.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:10.372 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T16:52:10.419 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T16:52:10.419 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:52:10.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7fecf1f20640 1 Processor -- start 2024-04-25T16:52:10.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7fecf1f20640 1 -- start start 2024-04-25T16:52:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7fecf1f20640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fecec070080 0x7fecec15e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7fecf1f20640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecec15ebd0 0x7fecec06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7fecf1f20640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecec06fc60 con 0x7fecec15e7c0 2024-04-25T16:52:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7fecf1f20640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fecec158550 con 0x7fecec070080 2024-04-25T16:52:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7fecf1f20640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fecec06f7d0 con 0x7fecec15ebd0 2024-04-25T16:52:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7feceb7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fecec070080 0x7fecec15e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7feceb7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fecec070080 0x7fecec15e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48126/0 (socket says 172.21.15.94:48126) 2024-04-25T16:52:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7feceb7fe640 1 -- 172.21.15.94:0/3473562527 learned_addr learned my addr 172.21.15.94:0/3473562527 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7feceb7fe640 1 -- 172.21.15.94:0/3473562527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecec15ebd0 msgr2=0x7fecec06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7feceb7fe640 1 --2- 172.21.15.94:0/3473562527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecec15ebd0 0x7fecec06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7feceb7fe640 1 -- 172.21.15.94:0/3473562527 >> v1:172.21.15.94:6789/0 conn(0x7fecec15e7c0 legacy=0x7fecec06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.559+0000 7feceb7fe640 1 -- 172.21.15.94:0/3473562527 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec0aef70 con 0x7fecec070080 2024-04-25T16:52:10.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.563+0000 7feceb7fe640 1 --2- 172.21.15.94:0/3473562527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec070080 0x7fecec15e280 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fece4009e50 tx=0x7fece401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74a4ec801cde703 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:10.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.563+0000 7fecea7fc640 1 -- 172.21.15.94:0/3473562527 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece40a57e0 con 0x7fecec070080 2024-04-25T16:52:10.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.563+0000 7fecea7fc640 1 -- 172.21.15.94:0/3473562527 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece40a5980 con 0x7fecec070080 2024-04-25T16:52:10.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.563+0000 7fecea7fc640 1 -- 172.21.15.94:0/3473562527 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece40a5d50 con 0x7fecec070080 2024-04-25T16:52:10.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.563+0000 7fecf1f20640 1 -- 172.21.15.94:0/3473562527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec070080 msgr2=0x7fecec15e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.563+0000 7fecf1f20640 1 --2- 172.21.15.94:0/3473562527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec070080 0x7fecec15e280 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fece4009e50 tx=0x7fece401ca00 comp rx=0 tx=0).stop 2024-04-25T16:52:10.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.563+0000 7fecf1f20640 1 -- 172.21.15.94:0/3473562527 shutdown_connections 2024-04-25T16:52:10.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.563+0000 7fecf1f20640 1 --2- 172.21.15.94:0/3473562527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecec15ebd0 0x7fecec06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.563+0000 7fecf1f20640 1 --2- 172.21.15.94:0/3473562527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec070080 0x7fecec15e280 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.563+0000 7fecf1f20640 1 -- 172.21.15.94:0/3473562527 >> 172.21.15.94:0/3473562527 conn(0x7fecec0b47f0 msgr2=0x7fecec0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:10.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.563+0000 7fecf1f20640 1 -- 172.21.15.94:0/3473562527 shutdown_connections 2024-04-25T16:52:10.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.563+0000 7fecf1f20640 1 -- 172.21.15.94:0/3473562527 wait complete. 2024-04-25T16:52:10.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fecf1f20640 1 Processor -- start 2024-04-25T16:52:10.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fecf1f20640 1 -- start start 2024-04-25T16:52:10.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fecf1f20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec070080 0x7fecec14f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fecf1f20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecec15e7c0 0x7fecec0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fecf1f20640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecec15ebd0 0x7fecec0cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fecf1f20640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fecec070490 con 0x7fecec070080 2024-04-25T16:52:10.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fecf1f20640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fecec0ae6c0 con 0x7fecec15e7c0 2024-04-25T16:52:10.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fecf1f20640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fecec16ae90 con 0x7fecec15ebd0 2024-04-25T16:52:10.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7feceb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec070080 0x7fecec14f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7feceb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec070080 0x7fecec14f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48132/0 (socket says 172.21.15.94:48132) 2024-04-25T16:52:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7feceb7fe640 1 -- 172.21.15.94:0/4009820466 learned_addr learned my addr 172.21.15.94:0/4009820466 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fecebfff640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecec15ebd0 0x7fecec0cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7feceaffd640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecec15e7c0 0x7fecec0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7feceb7fe640 1 -- 172.21.15.94:0/4009820466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecec15ebd0 msgr2=0x7fecec0cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7feceb7fe640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecec15ebd0 0x7fecec0cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7feceb7fe640 1 -- 172.21.15.94:0/4009820466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecec15e7c0 msgr2=0x7fecec0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7feceb7fe640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecec15e7c0 0x7fecec0ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7feceb7fe640 1 -- 172.21.15.94:0/4009820466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec09d940 con 0x7fecec070080 2024-04-25T16:52:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7feceb7fe640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec070080 0x7fecec14f370 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fece401a630 tx=0x7fece40197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fece3fff640 1 -- 172.21.15.94:0/4009820466 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece40bc020 con 0x7fecec070080 2024-04-25T16:52:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fece3fff640 1 -- 172.21.15.94:0/4009820466 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece40b8670 con 0x7fecec070080 2024-04-25T16:52:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fecf1f20640 1 -- 172.21.15.94:0/4009820466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecec14fe30 con 0x7fecec070080 2024-04-25T16:52:10.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fece3fff640 1 -- 172.21.15.94:0/4009820466 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece40b8a60 con 0x7fecec070080 2024-04-25T16:52:10.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fecf1f20640 1 -- 172.21.15.94:0/4009820466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecec1503f0 con 0x7fecec070080 2024-04-25T16:52:10.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.567+0000 7fecf1f20640 1 -- 172.21.15.94:0/4009820466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecec09db10 con 0x7fecec070080 2024-04-25T16:52:10.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.571+0000 7fece3fff640 1 -- 172.21.15.94:0/4009820466 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fece40bf020 con 0x7fecec070080 2024-04-25T16:52:10.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.571+0000 7fece3fff640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feccc06f400 0x7feccc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.571+0000 7fece3fff640 1 -- 172.21.15.94:0/4009820466 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fece412a7f0 con 0x7fecec070080 2024-04-25T16:52:10.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.571+0000 7feceaffd640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feccc06f400 0x7feccc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.571+0000 7feceaffd640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feccc06f400 0x7feccc071910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fecec0cd580 tx=0x7fecd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:10.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.575+0000 7fece3fff640 1 -- 172.21.15.94:0/4009820466 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fece40c6030 con 0x7fecec070080 2024-04-25T16:52:10.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.771+0000 7fecf1f20640 1 -- 172.21.15.94:0/4009820466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecec076830 con 0x7fecec070080 2024-04-25T16:52:10.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.771+0000 7fece3fff640 1 -- 172.21.15.94:0/4009820466 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fece4016620 con 0x7fecec070080 2024-04-25T16:52:10.770 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:10.770 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2352186041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2352186041},{"type":"v1","addr":"172.21.15.138:6833","nonce":2352186041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3210927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3210927873},{"type":"v1","addr":"172.21.15.94:6835","nonce":3210927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2237413225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2237413225},{"type":"v1","addr":"172.21.15.94:6837","nonce":2237413225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:50:54.403299+0000","modified":"2024-04-25T16:51:18.139471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/3652545132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3652545132},{"type":"v1","addr":"172.21.15.138:6835","nonce":3652545132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T16:52:10.770 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:52:10.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.775+0000 7fecf1f20640 1 -- 172.21.15.94:0/4009820466 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feccc06f400 msgr2=0x7feccc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.775+0000 7fecf1f20640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feccc06f400 0x7feccc071910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fecec0cd580 tx=0x7fecd4009040 comp rx=0 tx=0).stop 2024-04-25T16:52:10.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.775+0000 7fecf1f20640 1 -- 172.21.15.94:0/4009820466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec070080 msgr2=0x7fecec14f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.775+0000 7fecf1f20640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec070080 0x7fecec14f370 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fece401a630 tx=0x7fece40197a0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.775+0000 7fecf1f20640 1 -- 172.21.15.94:0/4009820466 shutdown_connections 2024-04-25T16:52:10.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.775+0000 7fecf1f20640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7feccc06f400 0x7feccc071910 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.775+0000 7fecf1f20640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecec15ebd0 0x7fecec0cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.775+0000 7fecf1f20640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecec15e7c0 0x7fecec0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.775+0000 7fecf1f20640 1 --2- 172.21.15.94:0/4009820466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec070080 0x7fecec14f370 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.775+0000 7fecf1f20640 1 -- 172.21.15.94:0/4009820466 >> 172.21.15.94:0/4009820466 conn(0x7fecec0b47f0 msgr2=0x7fecec06c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:10.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.775+0000 7fecf1f20640 1 -- 172.21.15.94:0/4009820466 shutdown_connections 2024-04-25T16:52:10.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.775+0000 7fecf1f20640 1 -- 172.21.15.94:0/4009820466 wait complete. 2024-04-25T16:52:10.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:52:10.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60e152b640 1 Processor -- start 2024-04-25T16:52:10.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60e152b640 1 -- start start 2024-04-25T16:52:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60e152b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60dc0cdcb0 0x7f60dc072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60e152b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f60dc0b7df0 0x7f60dc161c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60e152b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60dc073090 con 0x7f60dc072c80 2024-04-25T16:52:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60e152b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f60dc0d2120 con 0x7f60dc0b7df0 2024-04-25T16:52:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60e152b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f60dc0d2400 con 0x7f60dc0cdcb0 2024-04-25T16:52:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60db7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f60dc0b7df0 0x7f60dc161c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60db7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f60dc0b7df0 0x7f60dc161c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48146/0 (socket says 172.21.15.94:48146) 2024-04-25T16:52:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60db7fe640 1 -- 172.21.15.94:0/2206753613 learned_addr learned my addr 172.21.15.94:0/2206753613 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60daffd640 1 --2- 172.21.15.94:0/2206753613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60dc0cdcb0 0x7f60dc072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60db7fe640 1 -- 172.21.15.94:0/2206753613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60dc0cdcb0 msgr2=0x7f60dc072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60db7fe640 1 --2- 172.21.15.94:0/2206753613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60dc0cdcb0 0x7f60dc072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60db7fe640 1 -- 172.21.15.94:0/2206753613 >> v1:172.21.15.138:6789/0 conn(0x7f60dc072c80 legacy=0x7f60dc161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60db7fe640 1 -- 172.21.15.94:0/2206753613 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc0d26e0 con 0x7f60dc0b7df0 2024-04-25T16:52:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.931+0000 7f60daffd640 1 --2- 172.21.15.94:0/2206753613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60dc0cdcb0 0x7f60dc072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.935+0000 7f60db7fe640 1 --2- 172.21.15.94:0/2206753613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0b7df0 0x7f60dc161c90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009e50 tx=0x7f60d401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379e510c973d8f76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.935+0000 7f60d9ffb640 1 -- 172.21.15.94:0/2206753613 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d40a5810 con 0x7f60dc0b7df0 2024-04-25T16:52:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.935+0000 7f60d9ffb640 1 -- 172.21.15.94:0/2206753613 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d40a59b0 con 0x7f60dc0b7df0 2024-04-25T16:52:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.935+0000 7f60d9ffb640 1 -- 172.21.15.94:0/2206753613 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d40a5d80 con 0x7f60dc0b7df0 2024-04-25T16:52:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.935+0000 7f60e152b640 1 -- 172.21.15.94:0/2206753613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0b7df0 msgr2=0x7f60dc161c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.935+0000 7f60e152b640 1 --2- 172.21.15.94:0/2206753613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0b7df0 0x7f60dc161c90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009e50 tx=0x7f60d401c970 comp rx=0 tx=0).stop 2024-04-25T16:52:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.935+0000 7f60e152b640 1 -- 172.21.15.94:0/2206753613 shutdown_connections 2024-04-25T16:52:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.935+0000 7f60e152b640 1 --2- 172.21.15.94:0/2206753613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0b7df0 0x7f60dc161c90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.935+0000 7f60e152b640 1 --2- 172.21.15.94:0/2206753613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60dc0cdcb0 0x7f60dc072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.935+0000 7f60e152b640 1 -- 172.21.15.94:0/2206753613 >> 172.21.15.94:0/2206753613 conn(0x7f60dc0d3850 msgr2=0x7f60dc0d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.935+0000 7f60e152b640 1 -- 172.21.15.94:0/2206753613 shutdown_connections 2024-04-25T16:52:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.935+0000 7f60e152b640 1 -- 172.21.15.94:0/2206753613 wait complete. 2024-04-25T16:52:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60e152b640 1 Processor -- start 2024-04-25T16:52:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60e152b640 1 -- start start 2024-04-25T16:52:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60e152b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc072c80 0x7f60dc160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60e152b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60dc0b7df0 0x7f60dc15ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60e152b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60dc0cdcb0 0x7f60dc157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60e152b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc0b6680 con 0x7f60dc072c80 2024-04-25T16:52:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60e152b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc0b6500 con 0x7f60dc0b7df0 2024-04-25T16:52:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60e152b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f60dc0ce0c0 con 0x7f60dc0cdcb0 2024-04-25T16:52:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60daffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc072c80 0x7f60dc160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60daffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc072c80 0x7f60dc160b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48148/0 (socket says 172.21.15.94:48148) 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60daffd640 1 -- 172.21.15.94:0/4255880352 learned_addr learned my addr 172.21.15.94:0/4255880352 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60db7fe640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60dc0cdcb0 0x7f60dc157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60da7fc640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60dc0b7df0 0x7f60dc15ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60daffd640 1 -- 172.21.15.94:0/4255880352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60dc0cdcb0 msgr2=0x7f60dc157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60daffd640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60dc0cdcb0 0x7f60dc157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60daffd640 1 -- 172.21.15.94:0/4255880352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60dc0b7df0 msgr2=0x7f60dc15ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60daffd640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60dc0b7df0 0x7f60dc15ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60daffd640 1 -- 172.21.15.94:0/4255880352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc1f6f40 con 0x7f60dc072c80 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60db7fe640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60dc0cdcb0 0x7f60dc157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60da7fc640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60dc0b7df0 0x7f60dc15ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60daffd640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc072c80 0x7f60dc160b70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00c7f0 tx=0x7f60cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60d37fe640 1 -- 172.21.15.94:0/4255880352 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc015070 con 0x7f60dc072c80 2024-04-25T16:52:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60d37fe640 1 -- 172.21.15.94:0/4255880352 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60cc007820 con 0x7f60dc072c80 2024-04-25T16:52:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60d37fe640 1 -- 172.21.15.94:0/4255880352 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc007c10 con 0x7f60dc072c80 2024-04-25T16:52:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60e152b640 1 -- 172.21.15.94:0/4255880352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60dc15f290 con 0x7f60dc072c80 2024-04-25T16:52:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60e152b640 1 -- 172.21.15.94:0/4255880352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60dc15f850 con 0x7f60dc072c80 2024-04-25T16:52:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.939+0000 7f60e152b640 1 -- 172.21.15.94:0/4255880352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60dc06a730 con 0x7f60dc072c80 2024-04-25T16:52:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.943+0000 7f60d37fe640 1 -- 172.21.15.94:0/4255880352 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f60cc018070 con 0x7f60dc072c80 2024-04-25T16:52:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.943+0000 7f60d37fe640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f60bc06f400 0x7f60bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.943+0000 7f60d37fe640 1 -- 172.21.15.94:0/4255880352 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f60cc08f7e0 con 0x7f60dc072c80 2024-04-25T16:52:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.943+0000 7f60da7fc640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f60bc06f400 0x7f60bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.943+0000 7f60da7fc640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f60bc06f400 0x7f60bc071910 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60dc162840 tx=0x7f60c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:10.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:10.947+0000 7f60d37fe640 1 -- 172.21.15.94:0/4255880352 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f60cc05c2d0 con 0x7f60dc072c80 2024-04-25T16:52:11.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.107+0000 7f60e152b640 1 -- 172.21.15.94:0/4255880352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60dc0d2b30 con 0x7f60dc072c80 2024-04-25T16:52:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.111+0000 7f60d37fe640 1 -- 172.21.15.94:0/4255880352 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7f60cc05bbd0 con 0x7f60dc072c80 2024-04-25T16:52:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:11.109 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:52:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.111+0000 7f60e152b640 1 -- 172.21.15.94:0/4255880352 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f60bc06f400 msgr2=0x7f60bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.111+0000 7f60e152b640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f60bc06f400 0x7f60bc071910 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60dc162840 tx=0x7f60c4009040 comp rx=0 tx=0).stop 2024-04-25T16:52:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.111+0000 7f60e152b640 1 -- 172.21.15.94:0/4255880352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc072c80 msgr2=0x7f60dc160b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.111+0000 7f60e152b640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc072c80 0x7f60dc160b70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00c7f0 tx=0x7f60cc00ccb0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.111+0000 7f60e152b640 1 -- 172.21.15.94:0/4255880352 shutdown_connections 2024-04-25T16:52:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.111+0000 7f60e152b640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f60bc06f400 0x7f60bc071910 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.111+0000 7f60e152b640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60dc0cdcb0 0x7f60dc157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.111+0000 7f60e152b640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60dc0b7df0 0x7f60dc15ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.111+0000 7f60e152b640 1 --2- 172.21.15.94:0/4255880352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc072c80 0x7f60dc160b70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.111+0000 7f60e152b640 1 -- 172.21.15.94:0/4255880352 >> 172.21.15.94:0/4255880352 conn(0x7f60dc0d3850 msgr2=0x7f60dc15aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.111+0000 7f60e152b640 1 -- 172.21.15.94:0/4255880352 shutdown_connections 2024-04-25T16:52:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.111+0000 7f60e152b640 1 -- 172.21.15.94:0/4255880352 wait complete. 2024-04-25T16:52:11.130 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:50:52.026933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:50:53.368977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T16:50:57.106043+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T16:52:11.131 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:52:11.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd879d640 1 Processor -- start 2024-04-25T16:52:11.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd879d640 1 -- start start 2024-04-25T16:52:11.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd879d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cd00d3450 0x7f2cd00d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd879d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2cd00d3da0 0x7f2cd00b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd879d640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2cd00b7510 0x7f2cd00b7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd879d640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f2cd00b7e60 con 0x7f2cd00b7510 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd879d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2cd0164a90 con 0x7f2cd00d3da0 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd879d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd00d51b0 con 0x7f2cd00d3450 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd6d3a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2cd00b7510 0x7f2cd00b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd6d3a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2cd00b7510 0x7f2cd00b7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48152/0 (socket says 172.21.15.94:48152) 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd6d3a640 1 -- 172.21.15.94:0/3279863163 learned_addr learned my addr 172.21.15.94:0/3279863163 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd5d38640 1 --2- 172.21.15.94:0/3279863163 >> v2:172.21.15.138:3300/0 conn(0x7f2cd00d3da0 0x7f2cd00b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd6539640 1 --2- 172.21.15.94:0/3279863163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cd00d3450 0x7f2cd00d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd6d3a640 1 -- 172.21.15.94:0/3279863163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cd00d3450 msgr2=0x7f2cd00d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd6d3a640 1 --2- 172.21.15.94:0/3279863163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cd00d3450 0x7f2cd00d3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd6d3a640 1 -- 172.21.15.94:0/3279863163 >> v2:172.21.15.138:3300/0 conn(0x7f2cd00d3da0 msgr2=0x7f2cd00b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd6d3a640 1 --2- 172.21.15.94:0/3279863163 >> v2:172.21.15.138:3300/0 conn(0x7f2cd00d3da0 0x7f2cd00b6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd6d3a640 1 -- 172.21.15.94:0/3279863163 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd0164510 con 0x7f2cd00b7510 2024-04-25T16:52:11.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.279+0000 7f2cd6539640 1 --2- 172.21.15.94:0/3279863163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cd00d3450 0x7f2cd00d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:11.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.283+0000 7f2cd6d3a640 1 --2- 172.21.15.94:0/3279863163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd00b7510 0x7f2cd00b7920 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2cd008df00 tx=0x7f2cc801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d96f499ed09716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:11.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.283+0000 7f2cd5537640 1 -- 172.21.15.94:0/3279863163 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc80a57b0 con 0x7f2cd00b7510 2024-04-25T16:52:11.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.283+0000 7f2cd5537640 1 -- 172.21.15.94:0/3279863163 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc80a5950 con 0x7f2cd00b7510 2024-04-25T16:52:11.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.283+0000 7f2cd5537640 1 -- 172.21.15.94:0/3279863163 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc80a5d20 con 0x7f2cd00b7510 2024-04-25T16:52:11.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.283+0000 7f2cd879d640 1 -- 172.21.15.94:0/3279863163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd00b7510 msgr2=0x7f2cd00b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.283+0000 7f2cd879d640 1 --2- 172.21.15.94:0/3279863163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd00b7510 0x7f2cd00b7920 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2cd008df00 tx=0x7f2cc801c930 comp rx=0 tx=0).stop 2024-04-25T16:52:11.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.283+0000 7f2cd879d640 1 -- 172.21.15.94:0/3279863163 shutdown_connections 2024-04-25T16:52:11.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.283+0000 7f2cd879d640 1 --2- 172.21.15.94:0/3279863163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd00b7510 0x7f2cd00b7920 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.283+0000 7f2cd879d640 1 --2- 172.21.15.94:0/3279863163 >> v2:172.21.15.138:3300/0 conn(0x7f2cd00d3da0 0x7f2cd00b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.283+0000 7f2cd879d640 1 --2- 172.21.15.94:0/3279863163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cd00d3450 0x7f2cd00d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.283+0000 7f2cd879d640 1 -- 172.21.15.94:0/3279863163 >> 172.21.15.94:0/3279863163 conn(0x7f2cd009e8d0 msgr2=0x7f2cd00b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:11.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.283+0000 7f2cd879d640 1 -- 172.21.15.94:0/3279863163 shutdown_connections 2024-04-25T16:52:11.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.283+0000 7f2cd879d640 1 -- 172.21.15.94:0/3279863163 wait complete. 2024-04-25T16:52:11.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd879d640 1 Processor -- start 2024-04-25T16:52:11.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd879d640 1 -- start start 2024-04-25T16:52:11.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd879d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd00b7510 0x7f2cd01595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd879d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cd00d3450 0x7f2cd0159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd879d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cd00d3da0 0x7f2cd015ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd879d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd00d41b0 con 0x7f2cd00b7510 2024-04-25T16:52:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd879d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd00cfc10 con 0x7f2cd00d3450 2024-04-25T16:52:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd879d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd00b8cd0 con 0x7f2cd00d3da0 2024-04-25T16:52:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd6539640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd00b7510 0x7f2cd01595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd6539640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd00b7510 0x7f2cd01595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48164/0 (socket says 172.21.15.94:48164) 2024-04-25T16:52:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd6539640 1 -- 172.21.15.94:0/1551722820 learned_addr learned my addr 172.21.15.94:0/1551722820 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd5d38640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cd00d3450 0x7f2cd0159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd6d3a640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cd00d3da0 0x7f2cd015ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd6539640 1 -- 172.21.15.94:0/1551722820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cd00d3da0 msgr2=0x7f2cd015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd6539640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cd00d3da0 0x7f2cd015ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd6539640 1 -- 172.21.15.94:0/1551722820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cd00d3450 msgr2=0x7f2cd0159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd6539640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cd00d3450 0x7f2cd0159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd6539640 1 -- 172.21.15.94:0/1551722820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd00cb330 con 0x7f2cd00b7510 2024-04-25T16:52:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd6539640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd00b7510 0x7f2cd01595d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00c7f0 tx=0x7f2ccc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cc6ffd640 1 -- 172.21.15.94:0/1551722820 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc015070 con 0x7f2cd00b7510 2024-04-25T16:52:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cc6ffd640 1 -- 172.21.15.94:0/1551722820 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ccc007820 con 0x7f2cd00b7510 2024-04-25T16:52:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd879d640 1 -- 172.21.15.94:0/1551722820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd00ca870 con 0x7f2cd00b7510 2024-04-25T16:52:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cc6ffd640 1 -- 172.21.15.94:0/1551722820 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc007c10 con 0x7f2cd00b7510 2024-04-25T16:52:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd879d640 1 -- 172.21.15.94:0/1551722820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd015ac30 con 0x7f2cd00b7510 2024-04-25T16:52:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.287+0000 7f2cd879d640 1 -- 172.21.15.94:0/1551722820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd0076830 con 0x7f2cd00b7510 2024-04-25T16:52:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.291+0000 7f2cc6ffd640 1 -- 172.21.15.94:0/1551722820 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2ccc018070 con 0x7f2cd00b7510 2024-04-25T16:52:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.291+0000 7f2cc6ffd640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2cb406f400 0x7f2cb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.291+0000 7f2cc6ffd640 1 -- 172.21.15.94:0/1551722820 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2ccc08f780 con 0x7f2cd00b7510 2024-04-25T16:52:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.291+0000 7f2cd5d38640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2cb406f400 0x7f2cb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.291+0000 7f2cd5d38640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2cb406f400 0x7f2cb4071910 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0163190 tx=0x7f2cc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.295+0000 7f2cc6ffd640 1 -- 172.21.15.94:0/1551722820 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2cd00cb330 con 0x7f2cd00b7510 2024-04-25T16:52:11.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.451+0000 7f2cd879d640 1 -- 172.21.15.94:0/1551722820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2cd00d26e0 con 0x7f2cd00b7510 2024-04-25T16:52:11.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.451+0000 7f2cc6ffd640 1 -- 172.21.15.94:0/1551722820 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f2ccc05c270 con 0x7f2cd00b7510 2024-04-25T16:52:11.450 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:52:11.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.455+0000 7f2cd879d640 1 -- 172.21.15.94:0/1551722820 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2cb406f400 msgr2=0x7f2cb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.455+0000 7f2cd879d640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2cb406f400 0x7f2cb4071910 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0163190 tx=0x7f2cc0008040 comp rx=0 tx=0).stop 2024-04-25T16:52:11.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.455+0000 7f2cd879d640 1 -- 172.21.15.94:0/1551722820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd00b7510 msgr2=0x7f2cd01595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.455+0000 7f2cd879d640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd00b7510 0x7f2cd01595d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00c7f0 tx=0x7f2ccc00ccb0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.455+0000 7f2cd879d640 1 -- 172.21.15.94:0/1551722820 shutdown_connections 2024-04-25T16:52:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.455+0000 7f2cd879d640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2cb406f400 0x7f2cb4071910 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.455+0000 7f2cd879d640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cd00d3da0 0x7f2cd015ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.455+0000 7f2cd879d640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cd00d3450 0x7f2cd0159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.455+0000 7f2cd879d640 1 --2- 172.21.15.94:0/1551722820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd00b7510 0x7f2cd01595d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.455+0000 7f2cd879d640 1 -- 172.21.15.94:0/1551722820 >> 172.21.15.94:0/1551722820 conn(0x7f2cd009e8d0 msgr2=0x7f2cd015fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.455+0000 7f2cd879d640 1 -- 172.21.15.94:0/1551722820 shutdown_connections 2024-04-25T16:52:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.455+0000 7f2cd879d640 1 -- 172.21.15.94:0/1551722820 wait complete. 2024-04-25T16:52:11.470 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:52:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb4ee6f640 1 Processor -- start 2024-04-25T16:52:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb4ee6f640 1 -- start start 2024-04-25T16:52:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb4ee6f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbb480d2710 0x7fbb480b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb4ee6f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb480cdc10 con 0x7fbb480b6270 2024-04-25T16:52:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb4ee6f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb480cdef0 con 0x7fbb480b78c0 2024-04-25T16:52:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb4ee6f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbb480cfce0 con 0x7fbb480d2710 2024-04-25T16:52:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb4d40c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fbb480b6270 0x7fbb480b6680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:43108/0 (socket says 172.21.15.94:43108) 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb4d40c640 1 -- 172.21.15.94:0/4249094185 learned_addr learned my addr 172.21.15.94:0/4249094185 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb47fff640 1 --2- 172.21.15.94:0/4249094185 >> v2:172.21.15.138:3300/0 conn(0x7fbb480d2710 0x7fbb480b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2389733058 0 0) 0x7fbb480cdc10 con 0x7fbb480b6270 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb30003530 con 0x7fbb480b6270 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 612366690 0 0) 0x7fbb30003530 con 0x7fbb480b6270 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 >> v2:172.21.15.138:3300/0 conn(0x7fbb480d2710 msgr2=0x7fbb480b7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 --2- 172.21.15.94:0/4249094185 >> v2:172.21.15.138:3300/0 conn(0x7fbb480d2710 0x7fbb480b7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 >> v1:172.21.15.138:6789/0 conn(0x7fbb480b78c0 legacy=0x7fbb480d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb480ce1d0 con 0x7fbb480b6270 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fbb40001680 con 0x7fbb480b6270 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 >> v1:172.21.15.94:6789/0 conn(0x7fbb480b6270 legacy=0x7fbb480b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 --2- 172.21.15.94:0/4249094185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb300039c0 0x7fbb30003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 --2- 172.21.15.94:0/4249094185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb30004810 0x7fbb30004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 --2- 172.21.15.94:0/4249094185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb300051d0 0x7fbb3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3000c130 con 0x7fbb30004810 2024-04-25T16:52:11.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3000c2e0 con 0x7fbb300051d0 2024-04-25T16:52:11.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.619+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3000c4e0 con 0x7fbb300039c0 2024-04-25T16:52:11.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb47fff640 1 --2- 172.21.15.94:0/4249094185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb30004810 0x7fbb30004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb4cc0b640 1 --2- 172.21.15.94:0/4249094185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb300039c0 0x7fbb30003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb47fff640 1 -- 172.21.15.94:0/4249094185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb300039c0 msgr2=0x7fbb30003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:11.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb47fff640 1 --2- 172.21.15.94:0/4249094185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb300039c0 0x7fbb30003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb4d40c640 1 --2- 172.21.15.94:0/4249094185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb300051d0 0x7fbb3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb47fff640 1 -- 172.21.15.94:0/4249094185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb300051d0 msgr2=0x7fbb3000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:11.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb47fff640 1 --2- 172.21.15.94:0/4249094185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb300051d0 0x7fbb3000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb47fff640 1 -- 172.21.15.94:0/4249094185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb30003530 con 0x7fbb30004810 2024-04-25T16:52:11.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb4d40c640 1 --2- 172.21.15.94:0/4249094185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb300051d0 0x7fbb3000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:11.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb4d40c640 1 -- 172.21.15.94:0/4249094185 reap_dead start 2024-04-25T16:52:11.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb47fff640 1 --2- 172.21.15.94:0/4249094185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb30004810 0x7fbb30004c20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009ae0 tx=0x7fbb3c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280f5111eaea5198 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:11.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c0b2070 con 0x7fbb30004810 2024-04-25T16:52:11.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb3c0a5660 con 0x7fbb30004810 2024-04-25T16:52:11.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb477fe640 1 -- 172.21.15.94:0/4249094185 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c0096b0 con 0x7fbb30004810 2024-04-25T16:52:11.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/4249094185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb30004810 msgr2=0x7fbb30004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb4ee6f640 1 --2- 172.21.15.94:0/4249094185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb30004810 0x7fbb30004c20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009ae0 tx=0x7fbb3c01ca70 comp rx=0 tx=0).stop 2024-04-25T16:52:11.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/4249094185 shutdown_connections 2024-04-25T16:52:11.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb4ee6f640 1 --2- 172.21.15.94:0/4249094185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb30004810 0x7fbb30004c20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/4249094185 >> 172.21.15.94:0/4249094185 conn(0x7fbb480cfea0 msgr2=0x7fbb480d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:11.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/4249094185 shutdown_connections 2024-04-25T16:52:11.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.623+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/4249094185 wait complete. 2024-04-25T16:52:11.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4ee6f640 1 Processor -- start 2024-04-25T16:52:11.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4ee6f640 1 -- start start 2024-04-25T16:52:11.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4ee6f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb30004810 0x7fbb48158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4ee6f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb48148bc0 0x7fbb4815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4ee6f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb481f9350 0x7fbb481f9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4ee6f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbb480cecb0 con 0x7fbb481f9350 2024-04-25T16:52:11.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4ee6f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb480ce820 con 0x7fbb30004810 2024-04-25T16:52:11.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4ee6f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbb480b7cd0 con 0x7fbb48148bc0 2024-04-25T16:52:11.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4d40c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb481f9350 0x7fbb481f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4d40c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb481f9350 0x7fbb481f9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48188/0 (socket says 172.21.15.94:48188) 2024-04-25T16:52:11.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4d40c640 1 -- 172.21.15.94:0/3493318647 learned_addr learned my addr 172.21.15.94:0/3493318647 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:11.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4d40c640 1 -- 172.21.15.94:0/3493318647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb48148bc0 msgr2=0x7fbb4815bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb47fff640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb48148bc0 0x7fbb4815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4cc0b640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb30004810 0x7fbb48158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4d40c640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb48148bc0 0x7fbb4815bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4d40c640 1 -- 172.21.15.94:0/3493318647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb30004810 msgr2=0x7fbb48158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4d40c640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb30004810 0x7fbb48158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4d40c640 1 -- 172.21.15.94:0/3493318647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb481f9ca0 con 0x7fbb481f9350 2024-04-25T16:52:11.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4d40c640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb481f9350 0x7fbb481f9760 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbb4806d630 tx=0x7fbb4000d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:11.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb457fa640 1 -- 172.21.15.94:0/3493318647 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40007440 con 0x7fbb481f9350 2024-04-25T16:52:11.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb457fa640 1 -- 172.21.15.94:0/3493318647 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb400075e0 con 0x7fbb481f9350 2024-04-25T16:52:11.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/3493318647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb480c7c50 con 0x7fbb481f9350 2024-04-25T16:52:11.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb457fa640 1 -- 172.21.15.94:0/3493318647 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40004a30 con 0x7fbb481f9350 2024-04-25T16:52:11.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.627+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/3493318647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb480c8210 con 0x7fbb481f9350 2024-04-25T16:52:11.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.631+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/3493318647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb4806a730 con 0x7fbb481f9350 2024-04-25T16:52:11.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.631+0000 7fbb457fa640 1 -- 172.21.15.94:0/3493318647 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbb4001b020 con 0x7fbb481f9350 2024-04-25T16:52:11.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.631+0000 7fbb457fa640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbb3406f400 0x7fbb34071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.631+0000 7fbb4cc0b640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbb3406f400 0x7fbb34071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.631+0000 7fbb457fa640 1 -- 172.21.15.94:0/3493318647 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fbb4008f0d0 con 0x7fbb481f9350 2024-04-25T16:52:11.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.631+0000 7fbb4cc0b640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbb3406f400 0x7fbb34071910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbb38001c70 tx=0x7fbb38009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:11.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.635+0000 7fbb457fa640 1 -- 172.21.15.94:0/3493318647 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbb481f9ca0 con 0x7fbb481f9350 2024-04-25T16:52:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.783+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/3493318647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbb4815c490 con 0x7fbb481f9350 2024-04-25T16:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.783+0000 7fbb457fa640 1 -- 172.21.15.94:0/3493318647 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fbb4005bc50 con 0x7fbb481f9350 2024-04-25T16:52:11.796 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.787+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/3493318647 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbb3406f400 msgr2=0x7fbb34071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.787+0000 7fbb4ee6f640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbb3406f400 0x7fbb34071910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbb38001c70 tx=0x7fbb38009040 comp rx=0 tx=0).stop 2024-04-25T16:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.787+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/3493318647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb481f9350 msgr2=0x7fbb481f9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.787+0000 7fbb4ee6f640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb481f9350 0x7fbb481f9760 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbb4806d630 tx=0x7fbb4000d950 comp rx=0 tx=0).stop 2024-04-25T16:52:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.787+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/3493318647 shutdown_connections 2024-04-25T16:52:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.787+0000 7fbb4ee6f640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb481f9350 0x7fbb481f9760 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.787+0000 7fbb4ee6f640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbb3406f400 0x7fbb34071910 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.787+0000 7fbb4ee6f640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb48148bc0 0x7fbb4815bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.787+0000 7fbb4ee6f640 1 --2- 172.21.15.94:0/3493318647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb30004810 0x7fbb48158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.787+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/3493318647 >> 172.21.15.94:0/3493318647 conn(0x7fbb480cfea0 msgr2=0x7fbb480b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.787+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/3493318647 shutdown_connections 2024-04-25T16:52:11.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.787+0000 7fbb4ee6f640 1 -- 172.21.15.94:0/3493318647 wait complete. 2024-04-25T16:52:11.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:52:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7bfff640 1 Processor -- start 2024-04-25T16:52:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7bfff640 1 -- start start 2024-04-25T16:52:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fde7c0ce490 0x7fde7c15e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7bfff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fde7c0cff50 0x7fde7c0d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7bfff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde7c1613a0 0x7fde7c1617b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7bfff640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fde7c073800 con 0x7fde7c0ce490 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7bfff640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fde7c072bf0 con 0x7fde7c0cff50 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7bfff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c072ed0 con 0x7fde7c1613a0 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7affd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fde7c0ce490 0x7fde7c15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7affd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fde7c0ce490 0x7fde7c15e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48192/0 (socket says 172.21.15.94:48192) 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7affd640 1 -- 172.21.15.94:0/113339808 learned_addr learned my addr 172.21.15.94:0/113339808 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7a7fc640 1 --2- 172.21.15.94:0/113339808 >> v2:172.21.15.138:3300/0 conn(0x7fde7c0cff50 0x7fde7c0d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7b7fe640 1 --2- 172.21.15.94:0/113339808 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde7c1613a0 0x7fde7c1617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7affd640 1 -- 172.21.15.94:0/113339808 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde7c1613a0 msgr2=0x7fde7c1617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7affd640 1 --2- 172.21.15.94:0/113339808 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde7c1613a0 0x7fde7c1617b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7affd640 1 -- 172.21.15.94:0/113339808 >> v2:172.21.15.138:3300/0 conn(0x7fde7c0cff50 msgr2=0x7fde7c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7affd640 1 --2- 172.21.15.94:0/113339808 >> v2:172.21.15.138:3300/0 conn(0x7fde7c0cff50 0x7fde7c0d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7affd640 1 -- 172.21.15.94:0/113339808 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c0731b0 con 0x7fde7c0ce490 2024-04-25T16:52:11.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.955+0000 7fde7a7fc640 1 --2- 172.21.15.94:0/113339808 >> v2:172.21.15.138:3300/0 conn(0x7fde7c0cff50 0x7fde7c0d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:11.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.959+0000 7fde7affd640 1 --2- 172.21.15.94:0/113339808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c0ce490 0x7fde7c15e670 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fde7c072320 tx=0x7fde6c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0f22f379468ca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:11.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.959+0000 7fde79ffb640 1 -- 172.21.15.94:0/113339808 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0aa430 con 0x7fde7c0ce490 2024-04-25T16:52:11.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.959+0000 7fde79ffb640 1 -- 172.21.15.94:0/113339808 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde6c0af070 con 0x7fde7c0ce490 2024-04-25T16:52:11.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.959+0000 7fde79ffb640 1 -- 172.21.15.94:0/113339808 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0aa7f0 con 0x7fde7c0ce490 2024-04-25T16:52:11.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.959+0000 7fde7bfff640 1 -- 172.21.15.94:0/113339808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c0ce490 msgr2=0x7fde7c15e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.959+0000 7fde7bfff640 1 --2- 172.21.15.94:0/113339808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c0ce490 0x7fde7c15e670 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fde7c072320 tx=0x7fde6c01c4f0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.959+0000 7fde7bfff640 1 -- 172.21.15.94:0/113339808 shutdown_connections 2024-04-25T16:52:11.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.959+0000 7fde7bfff640 1 --2- 172.21.15.94:0/113339808 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde7c1613a0 0x7fde7c1617b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.959+0000 7fde7bfff640 1 --2- 172.21.15.94:0/113339808 >> v2:172.21.15.138:3300/0 conn(0x7fde7c0cff50 0x7fde7c0d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.959+0000 7fde7bfff640 1 --2- 172.21.15.94:0/113339808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c0ce490 0x7fde7c15e670 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.959+0000 7fde7bfff640 1 -- 172.21.15.94:0/113339808 >> 172.21.15.94:0/113339808 conn(0x7fde7c0d1020 msgr2=0x7fde7c0d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:11.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.959+0000 7fde7bfff640 1 -- 172.21.15.94:0/113339808 shutdown_connections 2024-04-25T16:52:11.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.959+0000 7fde7bfff640 1 -- 172.21.15.94:0/113339808 wait complete. 2024-04-25T16:52:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7bfff640 1 Processor -- start 2024-04-25T16:52:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7bfff640 1 -- start start 2024-04-25T16:52:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde7c0ce490 0x7fde7c140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7bfff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde7c0cff50 0x7fde7c157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c1613a0 0x7fde7c13d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7bfff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c0ce8a0 con 0x7fde7c1613a0 2024-04-25T16:52:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7bfff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c15ebb0 con 0x7fde7c0ce490 2024-04-25T16:52:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7bfff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c15a0b0 con 0x7fde7c0cff50 2024-04-25T16:52:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c1613a0 0x7fde7c13d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c1613a0 0x7fde7c13d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48208/0 (socket says 172.21.15.94:48208) 2024-04-25T16:52:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7b7fe640 1 -- 172.21.15.94:0/780259406 learned_addr learned my addr 172.21.15.94:0/780259406 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7affd640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde7c0ce490 0x7fde7c140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7a7fc640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde7c0cff50 0x7fde7c157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7b7fe640 1 -- 172.21.15.94:0/780259406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde7c0cff50 msgr2=0x7fde7c157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7b7fe640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde7c0cff50 0x7fde7c157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7b7fe640 1 -- 172.21.15.94:0/780259406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde7c0ce490 msgr2=0x7fde7c140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7b7fe640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde7c0ce490 0x7fde7c140e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7b7fe640 1 -- 172.21.15.94:0/780259406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c141340 con 0x7fde7c1613a0 2024-04-25T16:52:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde7b7fe640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c1613a0 0x7fde7c13d230 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fde7400ccb0 tx=0x7fde7400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.963+0000 7fde637fe640 1 -- 172.21.15.94:0/780259406 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7400ce60 con 0x7fde7c1613a0 2024-04-25T16:52:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.967+0000 7fde637fe640 1 -- 172.21.15.94:0/780259406 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde740039a0 con 0x7fde7c1613a0 2024-04-25T16:52:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.967+0000 7fde7bfff640 1 -- 172.21.15.94:0/780259406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c1578a0 con 0x7fde7c1613a0 2024-04-25T16:52:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.967+0000 7fde637fe640 1 -- 172.21.15.94:0/780259406 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74003d70 con 0x7fde7c1613a0 2024-04-25T16:52:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.967+0000 7fde7bfff640 1 -- 172.21.15.94:0/780259406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c141af0 con 0x7fde7c1613a0 2024-04-25T16:52:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.967+0000 7fde7bfff640 1 -- 172.21.15.94:0/780259406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7c06a310 con 0x7fde7c1613a0 2024-04-25T16:52:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.967+0000 7fde637fe640 1 -- 172.21.15.94:0/780259406 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fde74003f10 con 0x7fde7c1613a0 2024-04-25T16:52:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.967+0000 7fde637fe640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fde6806f430 0x7fde68071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.967+0000 7fde637fe640 1 -- 172.21.15.94:0/780259406 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fde7408fb20 con 0x7fde7c1613a0 2024-04-25T16:52:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.967+0000 7fde7affd640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fde6806f430 0x7fde68071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.967+0000 7fde7affd640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fde6806f430 0x7fde68071940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0ae000 tx=0x7fde6c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:11.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:11.971+0000 7fde637fe640 1 -- 172.21.15.94:0/780259406 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fde74014030 con 0x7fde7c1613a0 2024-04-25T16:52:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.135+0000 7fde7bfff640 1 -- 172.21.15.94:0/780259406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fde7c072930 con 0x7fde7c1613a0 2024-04-25T16:52:12.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.135+0000 7fde637fe640 1 -- 172.21.15.94:0/780259406 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fde7405c590 con 0x7fde7c1613a0 2024-04-25T16:52:12.133 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T16:52:12.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.135+0000 7fde7bfff640 1 -- 172.21.15.94:0/780259406 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fde6806f430 msgr2=0x7fde68071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.135+0000 7fde7bfff640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fde6806f430 0x7fde68071940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0ae000 tx=0x7fde6c0a7000 comp rx=0 tx=0).stop 2024-04-25T16:52:12.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.135+0000 7fde7bfff640 1 -- 172.21.15.94:0/780259406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c1613a0 msgr2=0x7fde7c13d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.135+0000 7fde7bfff640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c1613a0 0x7fde7c13d230 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fde7400ccb0 tx=0x7fde7400a5c0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.135+0000 7fde7bfff640 1 -- 172.21.15.94:0/780259406 shutdown_connections 2024-04-25T16:52:12.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.135+0000 7fde7bfff640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde7c1613a0 0x7fde7c13d230 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.135+0000 7fde7bfff640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fde6806f430 0x7fde68071940 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.135+0000 7fde7bfff640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde7c0cff50 0x7fde7c157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.135+0000 7fde7bfff640 1 --2- 172.21.15.94:0/780259406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde7c0ce490 0x7fde7c140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.139+0000 7fde7bfff640 1 -- 172.21.15.94:0/780259406 >> 172.21.15.94:0/780259406 conn(0x7fde7c0d1020 msgr2=0x7fde7c0b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:12.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.139+0000 7fde7bfff640 1 -- 172.21.15.94:0/780259406 shutdown_connections 2024-04-25T16:52:12.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.139+0000 7fde7bfff640 1 -- 172.21.15.94:0/780259406 wait complete. 2024-04-25T16:52:12.154 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T16:52:12.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.307+0000 7f108dd0a640 1 Processor -- start 2024-04-25T16:52:12.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.307+0000 7f108dd0a640 1 -- start start 2024-04-25T16:52:12.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f108dd0a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10880b73d0 0x7f10880b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:12.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f108dd0a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10880701b0 0x7f10880705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:12.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f108dd0a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10880b7d20 con 0x7f1088143f20 2024-04-25T16:52:12.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f108dd0a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f108815e980 con 0x7f10880b73d0 2024-04-25T16:52:12.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f108dd0a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1088144330 con 0x7f10880701b0 2024-04-25T16:52:12.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f1086ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10880b73d0 0x7f10880b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:12.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f1087fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10880701b0 0x7f10880705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:12.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f1086ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10880b73d0 0x7f10880b77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41010/0 (socket says 172.21.15.94:41010) 2024-04-25T16:52:12.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f1086ffd640 1 -- 172.21.15.94:0/3814474312 learned_addr learned my addr 172.21.15.94:0/3814474312 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:12.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f1087fff640 1 -- 172.21.15.94:0/3814474312 >> v2:172.21.15.138:3300/0 conn(0x7f10880b73d0 msgr2=0x7f10880b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:12.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f1087fff640 1 --2- 172.21.15.94:0/3814474312 >> v2:172.21.15.138:3300/0 conn(0x7f10880b73d0 0x7f10880b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f1087fff640 1 -- 172.21.15.94:0/3814474312 >> v1:172.21.15.138:6789/0 conn(0x7f1088143f20 legacy=0x7f10880d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f1087fff640 1 -- 172.21.15.94:0/3814474312 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108815ee10 con 0x7f10880701b0 2024-04-25T16:52:12.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.311+0000 7f1086ffd640 1 --2- 172.21.15.94:0/3814474312 >> v2:172.21.15.138:3300/0 conn(0x7f10880b73d0 0x7f10880b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:12.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.315+0000 7f1087fff640 1 --2- 172.21.15.94:0/3814474312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10880701b0 0x7f10880705c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1080009dc0 tx=0x7f108001c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ece624fbbf772855 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:12.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.315+0000 7f10867fc640 1 -- 172.21.15.94:0/3814474312 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10800b2070 con 0x7f10880701b0 2024-04-25T16:52:12.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.315+0000 7f10867fc640 1 -- 172.21.15.94:0/3814474312 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10800a59c0 con 0x7f10880701b0 2024-04-25T16:52:12.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.315+0000 7f10867fc640 1 -- 172.21.15.94:0/3814474312 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10800a5d90 con 0x7f10880701b0 2024-04-25T16:52:12.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.315+0000 7f108dd0a640 1 -- 172.21.15.94:0/3814474312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10880701b0 msgr2=0x7f10880705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.315+0000 7f108dd0a640 1 --2- 172.21.15.94:0/3814474312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10880701b0 0x7f10880705c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1080009dc0 tx=0x7f108001c900 comp rx=0 tx=0).stop 2024-04-25T16:52:12.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.315+0000 7f108dd0a640 1 -- 172.21.15.94:0/3814474312 shutdown_connections 2024-04-25T16:52:12.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.315+0000 7f108dd0a640 1 --2- 172.21.15.94:0/3814474312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10880701b0 0x7f10880705c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.315+0000 7f108dd0a640 1 --2- 172.21.15.94:0/3814474312 >> v2:172.21.15.138:3300/0 conn(0x7f10880b73d0 0x7f10880b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.315+0000 7f108dd0a640 1 -- 172.21.15.94:0/3814474312 >> 172.21.15.94:0/3814474312 conn(0x7f10880d33d0 msgr2=0x7f10880d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:12.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.315+0000 7f108dd0a640 1 -- 172.21.15.94:0/3814474312 shutdown_connections 2024-04-25T16:52:12.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.315+0000 7f108dd0a640 1 -- 172.21.15.94:0/3814474312 wait complete. 2024-04-25T16:52:12.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f108dd0a640 1 Processor -- start 2024-04-25T16:52:12.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f108dd0a640 1 -- start start 2024-04-25T16:52:12.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f108dd0a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10880701b0 0x7f108815d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:12.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f108dd0a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10880b73d0 0x7f108815c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:12.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f108dd0a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1088143f20 0x7f1088148a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:12.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f108dd0a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f108816a630 con 0x7f10880701b0 2024-04-25T16:52:12.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f108dd0a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10880d0fb0 con 0x7f1088143f20 2024-04-25T16:52:12.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f108dd0a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f108816a7b0 con 0x7f10880b73d0 2024-04-25T16:52:12.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f10877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10880701b0 0x7f108815d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:12.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f10877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10880701b0 0x7f108815d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48224/0 (socket says 172.21.15.94:48224) 2024-04-25T16:52:12.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f10877fe640 1 -- 172.21.15.94:0/2602275922 learned_addr learned my addr 172.21.15.94:0/2602275922 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:12.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f1087fff640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1088143f20 0x7f1088148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:12.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f1086ffd640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10880b73d0 0x7f108815c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:12.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f10877fe640 1 -- 172.21.15.94:0/2602275922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10880b73d0 msgr2=0x7f108815c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f10877fe640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10880b73d0 0x7f108815c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f10877fe640 1 -- 172.21.15.94:0/2602275922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1088143f20 msgr2=0x7f1088148a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f10877fe640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1088143f20 0x7f1088148a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f10877fe640 1 -- 172.21.15.94:0/2602275922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108813c330 con 0x7f10880701b0 2024-04-25T16:52:12.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f1087fff640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1088143f20 0x7f1088148a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:12.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f10877fe640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10880701b0 0x7f108815d2a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f107800c370 tx=0x7f107800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:12.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f106ffff640 1 -- 172.21.15.94:0/2602275922 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10780052d0 con 0x7f10880701b0 2024-04-25T16:52:12.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f106ffff640 1 -- 172.21.15.94:0/2602275922 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1078005470 con 0x7f10880701b0 2024-04-25T16:52:12.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f108dd0a640 1 -- 172.21.15.94:0/2602275922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108813c610 con 0x7f10880701b0 2024-04-25T16:52:12.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f106ffff640 1 -- 172.21.15.94:0/2602275922 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078007260 con 0x7f10880701b0 2024-04-25T16:52:12.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f108dd0a640 1 -- 172.21.15.94:0/2602275922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f108813cbd0 con 0x7f10880701b0 2024-04-25T16:52:12.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f108dd0a640 1 -- 172.21.15.94:0/2602275922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108806a730 con 0x7f10880701b0 2024-04-25T16:52:12.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.319+0000 7f106ffff640 1 -- 172.21.15.94:0/2602275922 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1078007400 con 0x7f10880701b0 2024-04-25T16:52:12.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.323+0000 7f106ffff640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f107406f400 0x7f1074071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:12.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.323+0000 7f1086ffd640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f107406f400 0x7f1074071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:12.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.323+0000 7f106ffff640 1 -- 172.21.15.94:0/2602275922 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f107808eab0 con 0x7f10880701b0 2024-04-25T16:52:12.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.323+0000 7f1086ffd640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f107406f400 0x7f1074071910 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f10881fac90 tx=0x7f107c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:12.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.327+0000 7f106ffff640 1 -- 172.21.15.94:0/2602275922 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f10880b7d20 con 0x7f10880701b0 2024-04-25T16:52:12.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f108dd0a640 1 -- 172.21.15.94:0/2602275922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f108815d8f0 con 0x7f10880701b0 2024-04-25T16:52:12.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f106ffff640 1 -- 172.21.15.94:0/2602275922 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f107805b630 con 0x7f10880701b0 2024-04-25T16:52:12.472 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T16:52:12.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f108dd0a640 1 -- 172.21.15.94:0/2602275922 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f107406f400 msgr2=0x7f1074071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f108dd0a640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f107406f400 0x7f1074071910 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f10881fac90 tx=0x7f107c008040 comp rx=0 tx=0).stop 2024-04-25T16:52:12.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f108dd0a640 1 -- 172.21.15.94:0/2602275922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10880701b0 msgr2=0x7f108815d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f108dd0a640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10880701b0 0x7f108815d2a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f107800c370 tx=0x7f107800c830 comp rx=0 tx=0).stop 2024-04-25T16:52:12.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f108dd0a640 1 -- 172.21.15.94:0/2602275922 shutdown_connections 2024-04-25T16:52:12.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f108dd0a640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1088143f20 0x7f1088148a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f108dd0a640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f107406f400 0x7f1074071910 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f108dd0a640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10880b73d0 0x7f108815c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f108dd0a640 1 --2- 172.21.15.94:0/2602275922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10880701b0 0x7f108815d2a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f108dd0a640 1 -- 172.21.15.94:0/2602275922 >> 172.21.15.94:0/2602275922 conn(0x7f10880d33d0 msgr2=0x7f10880b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:12.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f108dd0a640 1 -- 172.21.15.94:0/2602275922 shutdown_connections 2024-04-25T16:52:12.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.475+0000 7f108dd0a640 1 -- 172.21.15.94:0/2602275922 wait complete. 2024-04-25T16:52:12.494 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:52:12.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2ae2c1640 1 Processor -- start 2024-04-25T16:52:12.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2ae2c1640 1 -- start start 2024-04-25T16:52:12.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2ae2c1640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd2a80b5e70 0x7fd2a80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:12.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2ae2c1640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd2a8070010 0x7fd2a8070420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:12.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2ae2c1640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a80b67c0 con 0x7fd2a806ce90 2024-04-25T16:52:12.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2ae2c1640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a81426a0 con 0x7fd2a80b5e70 2024-04-25T16:52:12.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2ae2c1640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a815fd60 con 0x7fd2a8070010 2024-04-25T16:52:12.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2a7fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd2a806ce90 0x7fd2a8141f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:43118/0 (socket says 172.21.15.94:43118) 2024-04-25T16:52:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2a7fff640 1 -- 172.21.15.94:0/759038646 learned_addr learned my addr 172.21.15.94:0/759038646 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2a77fe640 1 --2- 172.21.15.94:0/759038646 >> v2:172.21.15.94:3300/0 conn(0x7fd2a80b5e70 0x7fd2a80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2ac85e640 1 --2- 172.21.15.94:0/759038646 >> v2:172.21.15.138:3300/0 conn(0x7fd2a8070010 0x7fd2a8070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2a6ffd640 1 -- 172.21.15.94:0/759038646 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1702246877 0 0) 0x7fd2a0001120 con 0x7fd2a806ce90 2024-04-25T16:52:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2a6ffd640 1 -- 172.21.15.94:0/759038646 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd294003530 con 0x7fd2a806ce90 2024-04-25T16:52:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2a77fe640 1 -- 172.21.15.94:0/759038646 >> v2:172.21.15.138:3300/0 conn(0x7fd2a8070010 msgr2=0x7fd2a8070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2a77fe640 1 --2- 172.21.15.94:0/759038646 >> v2:172.21.15.138:3300/0 conn(0x7fd2a8070010 0x7fd2a8070420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2a77fe640 1 -- 172.21.15.94:0/759038646 >> v1:172.21.15.94:6789/0 conn(0x7fd2a806ce90 legacy=0x7fd2a8141f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.643+0000 7fd2a77fe640 1 -- 172.21.15.94:0/759038646 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a8160040 con 0x7fd2a80b5e70 2024-04-25T16:52:12.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.647+0000 7fd2a77fe640 1 --2- 172.21.15.94:0/759038646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2a80b5e70 0x7fd2a80b6280 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8074310 tx=0x7fd29001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385acfd242998774 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:12.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.647+0000 7fd2a6ffd640 1 -- 172.21.15.94:0/759038646 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2900b0070 con 0x7fd2a80b5e70 2024-04-25T16:52:12.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.647+0000 7fd2a6ffd640 1 -- 172.21.15.94:0/759038646 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2900ab600 con 0x7fd2a80b5e70 2024-04-25T16:52:12.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.647+0000 7fd2a6ffd640 1 -- 172.21.15.94:0/759038646 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2900a4e60 con 0x7fd2a80b5e70 2024-04-25T16:52:12.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.647+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/759038646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2a80b5e70 msgr2=0x7fd2a80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.647+0000 7fd2ae2c1640 1 --2- 172.21.15.94:0/759038646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2a80b5e70 0x7fd2a80b6280 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8074310 tx=0x7fd29001c360 comp rx=0 tx=0).stop 2024-04-25T16:52:12.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.647+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/759038646 shutdown_connections 2024-04-25T16:52:12.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.647+0000 7fd2ae2c1640 1 --2- 172.21.15.94:0/759038646 >> v2:172.21.15.138:3300/0 conn(0x7fd2a8070010 0x7fd2a8070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.647+0000 7fd2ae2c1640 1 --2- 172.21.15.94:0/759038646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2a80b5e70 0x7fd2a80b6280 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.647+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/759038646 >> 172.21.15.94:0/759038646 conn(0x7fd2a80d1d30 msgr2=0x7fd2a80d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:12.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.647+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/759038646 shutdown_connections 2024-04-25T16:52:12.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.647+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/759038646 wait complete. 2024-04-25T16:52:12.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2ae2c1640 1 Processor -- start 2024-04-25T16:52:12.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2ae2c1640 1 -- start start 2024-04-25T16:52:12.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2ae2c1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2a806ce90 0x7fd2a815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:12.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2ae2c1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a8070010 0x7fd2a81553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:12.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2ae2c1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2a80b5e70 0x7fd2a80b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:12.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2ae2c1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81419f0 con 0x7fd2a806ce90 2024-04-25T16:52:12.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2ae2c1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8141870 con 0x7fd2a80b5e70 2024-04-25T16:52:12.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2ae2c1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a8166fc0 con 0x7fd2a8070010 2024-04-25T16:52:12.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2a7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2a806ce90 0x7fd2a815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:12.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2a7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2a806ce90 0x7fd2a815a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48242/0 (socket says 172.21.15.94:48242) 2024-04-25T16:52:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2a7fff640 1 -- 172.21.15.94:0/1914020312 learned_addr learned my addr 172.21.15.94:0/1914020312 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2ac85e640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2a80b5e70 0x7fd2a80b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2a77fe640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a8070010 0x7fd2a81553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2a7fff640 1 -- 172.21.15.94:0/1914020312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a8070010 msgr2=0x7fd2a81553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2a7fff640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a8070010 0x7fd2a81553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2a7fff640 1 -- 172.21.15.94:0/1914020312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2a80b5e70 msgr2=0x7fd2a80b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2a7fff640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2a80b5e70 0x7fd2a80b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2a7fff640 1 -- 172.21.15.94:0/1914020312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a81393f0 con 0x7fd2a806ce90 2024-04-25T16:52:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2ac85e640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2a80b5e70 0x7fd2a80b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2a7fff640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2a806ce90 0x7fd2a815a5e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000c3f0 tx=0x7fd2a000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2a4ff9640 1 -- 172.21.15.94:0/1914020312 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0017070 con 0x7fd2a806ce90 2024-04-25T16:52:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2a4ff9640 1 -- 172.21.15.94:0/1914020312 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a0005cf0 con 0x7fd2a806ce90 2024-04-25T16:52:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/1914020312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a813a360 con 0x7fd2a806ce90 2024-04-25T16:52:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2a4ff9640 1 -- 172.21.15.94:0/1914020312 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00072f0 con 0x7fd2a806ce90 2024-04-25T16:52:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/1914020312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a813a920 con 0x7fd2a806ce90 2024-04-25T16:52:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.651+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/1914020312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a806a730 con 0x7fd2a806ce90 2024-04-25T16:52:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.655+0000 7fd2a4ff9640 1 -- 172.21.15.94:0/1914020312 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd2a001a070 con 0x7fd2a806ce90 2024-04-25T16:52:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.655+0000 7fd2a4ff9640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd28806f430 0x7fd288071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.655+0000 7fd2a4ff9640 1 -- 172.21.15.94:0/1914020312 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd2a008ed70 con 0x7fd2a806ce90 2024-04-25T16:52:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.655+0000 7fd2a77fe640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd28806f430 0x7fd288071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.655+0000 7fd2a77fe640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd28806f430 0x7fd288071940 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd2900ad000 tx=0x7fd290008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:12.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.659+0000 7fd2a4ff9640 1 -- 172.21.15.94:0/1914020312 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd2a0021020 con 0x7fd2a806ce90 2024-04-25T16:52:12.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.851+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/1914020312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2a8156960 con 0x7fd2a806ce90 2024-04-25T16:52:12.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.855+0000 7fd2a4ff9640 1 -- 172.21.15.94:0/1914020312 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fd2a80b67c0 con 0x7fd2a806ce90 2024-04-25T16:52:12.852 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:12.853 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2352186041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2352186041},{"type":"v1","addr":"172.21.15.138:6833","nonce":2352186041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3210927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3210927873},{"type":"v1","addr":"172.21.15.94:6835","nonce":3210927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2237413225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2237413225},{"type":"v1","addr":"172.21.15.94:6837","nonce":2237413225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:50:54.403299+0000","modified":"2024-04-25T16:51:18.139471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/3652545132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3652545132},{"type":"v1","addr":"172.21.15.138:6835","nonce":3652545132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T16:52:12.853 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:52:12.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.855+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/1914020312 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd28806f430 msgr2=0x7fd288071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.855+0000 7fd2ae2c1640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd28806f430 0x7fd288071940 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd2900ad000 tx=0x7fd290008040 comp rx=0 tx=0).stop 2024-04-25T16:52:12.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.855+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/1914020312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2a806ce90 msgr2=0x7fd2a815a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:12.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.855+0000 7fd2ae2c1640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2a806ce90 0x7fd2a815a5e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000c3f0 tx=0x7fd2a000c8b0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.855+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/1914020312 shutdown_connections 2024-04-25T16:52:12.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.855+0000 7fd2ae2c1640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2a80b5e70 0x7fd2a80b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.855+0000 7fd2ae2c1640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd28806f430 0x7fd288071940 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.855+0000 7fd2ae2c1640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2a8070010 0x7fd2a81553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.855+0000 7fd2ae2c1640 1 --2- 172.21.15.94:0/1914020312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2a806ce90 0x7fd2a815a5e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:12.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.855+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/1914020312 >> 172.21.15.94:0/1914020312 conn(0x7fd2a80d1d30 msgr2=0x7fd2a806d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:12.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.855+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/1914020312 shutdown_connections 2024-04-25T16:52:12.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:12.855+0000 7fd2ae2c1640 1 -- 172.21.15.94:0/1914020312 wait complete. 2024-04-25T16:52:12.873 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:52:13.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26ad224640 1 Processor -- start 2024-04-25T16:52:13.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26ad224640 1 -- start start 2024-04-25T16:52:13.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26ad224640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f26a806d950 0x7f26a815de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26ad224640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f26a8070510 0x7f26a815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26ad224640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a815e760 con 0x7f26a815e350 2024-04-25T16:52:13.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26ad224640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f26a806d4c0 con 0x7f26a806d950 2024-04-25T16:52:13.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26ad224640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f26a806ce90 con 0x7f26a8070510 2024-04-25T16:52:13.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26a659c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f26a815e350 0x7f26a806fe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:43132/0 (socket says 172.21.15.94:43132) 2024-04-25T16:52:13.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26a659c640 1 -- 172.21.15.94:0/815634792 learned_addr learned my addr 172.21.15.94:0/815634792 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:13.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26a6d9d640 1 --2- 172.21.15.94:0/815634792 >> v2:172.21.15.94:3300/0 conn(0x7f26a806d950 0x7f26a815de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26a759e640 1 --2- 172.21.15.94:0/815634792 >> v2:172.21.15.138:3300/0 conn(0x7f26a8070510 0x7f26a815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26a6d9d640 1 -- 172.21.15.94:0/815634792 >> v2:172.21.15.138:3300/0 conn(0x7f26a8070510 msgr2=0x7f26a815efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:13.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26a6d9d640 1 --2- 172.21.15.94:0/815634792 >> v2:172.21.15.138:3300/0 conn(0x7f26a8070510 0x7f26a815efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26a6d9d640 1 -- 172.21.15.94:0/815634792 >> v1:172.21.15.94:6789/0 conn(0x7f26a815e350 legacy=0x7f26a806fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.023+0000 7f26a6d9d640 1 -- 172.21.15.94:0/815634792 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a806e330 con 0x7f26a806d950 2024-04-25T16:52:13.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.027+0000 7f26a6d9d640 1 --2- 172.21.15.94:0/815634792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a806d950 0x7f26a815de10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f26a81624a0 tx=0x7f269801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0394106fcd6c719 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:13.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.027+0000 7f26a5d9b640 1 -- 172.21.15.94:0/815634792 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26980b2070 con 0x7f26a806d950 2024-04-25T16:52:13.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.027+0000 7f26a5d9b640 1 -- 172.21.15.94:0/815634792 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26980a5820 con 0x7f26a806d950 2024-04-25T16:52:13.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.027+0000 7f26a5d9b640 1 -- 172.21.15.94:0/815634792 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26980a5bf0 con 0x7f26a806d950 2024-04-25T16:52:13.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.027+0000 7f26ad224640 1 -- 172.21.15.94:0/815634792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a806d950 msgr2=0x7f26a815de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.027+0000 7f26ad224640 1 --2- 172.21.15.94:0/815634792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a806d950 0x7f26a815de10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f26a81624a0 tx=0x7f269801c870 comp rx=0 tx=0).stop 2024-04-25T16:52:13.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.027+0000 7f26ad224640 1 -- 172.21.15.94:0/815634792 shutdown_connections 2024-04-25T16:52:13.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.027+0000 7f26ad224640 1 --2- 172.21.15.94:0/815634792 >> v2:172.21.15.138:3300/0 conn(0x7f26a8070510 0x7f26a815efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.027+0000 7f26ad224640 1 --2- 172.21.15.94:0/815634792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a806d950 0x7f26a815de10 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.027+0000 7f26ad224640 1 -- 172.21.15.94:0/815634792 >> 172.21.15.94:0/815634792 conn(0x7f26a8070f40 msgr2=0x7f26a8071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:13.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.027+0000 7f26ad224640 1 -- 172.21.15.94:0/815634792 shutdown_connections 2024-04-25T16:52:13.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.027+0000 7f26ad224640 1 -- 172.21.15.94:0/815634792 wait complete. 2024-04-25T16:52:13.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26ad224640 1 Processor -- start 2024-04-25T16:52:13.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26ad224640 1 -- start start 2024-04-25T16:52:13.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26ad224640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26a806d950 0x7f26a8167f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26ad224640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a8070510 0x7f26a80b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26ad224640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26a815e350 0x7f26a80b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26ad224640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26a816a7a0 con 0x7f26a8070510 2024-04-25T16:52:13.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26ad224640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26a806dd60 con 0x7f26a806d950 2024-04-25T16:52:13.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26ad224640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f26a816a920 con 0x7f26a815e350 2024-04-25T16:52:13.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26a659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a8070510 0x7f26a80b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26a659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a8070510 0x7f26a80b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48254/0 (socket says 172.21.15.94:48254) 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26a659c640 1 -- 172.21.15.94:0/2628873042 learned_addr learned my addr 172.21.15.94:0/2628873042 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26a759e640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26a815e350 0x7f26a80b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26a6d9d640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26a806d950 0x7f26a8167f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26a659c640 1 -- 172.21.15.94:0/2628873042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26a815e350 msgr2=0x7f26a80b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26a659c640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26a815e350 0x7f26a80b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26a659c640 1 -- 172.21.15.94:0/2628873042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26a806d950 msgr2=0x7f26a8167f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26a659c640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26a806d950 0x7f26a8167f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26a659c640 1 -- 172.21.15.94:0/2628873042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a81f92e0 con 0x7f26a8070510 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26a6d9d640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26a806d950 0x7f26a8167f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.031+0000 7f26a659c640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a8070510 0x7f26a80b54f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f269c00c370 tx=0x7f269c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.035+0000 7f268f7fe640 1 -- 172.21.15.94:0/2628873042 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c017070 con 0x7f26a8070510 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.035+0000 7f268f7fe640 1 -- 172.21.15.94:0/2628873042 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f269c005cf0 con 0x7f26a8070510 2024-04-25T16:52:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.035+0000 7f268f7fe640 1 -- 172.21.15.94:0/2628873042 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c0072f0 con 0x7f26a8070510 2024-04-25T16:52:13.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.035+0000 7f26ad224640 1 -- 172.21.15.94:0/2628873042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a80b62b0 con 0x7f26a8070510 2024-04-25T16:52:13.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.035+0000 7f26ad224640 1 -- 172.21.15.94:0/2628873042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a80b6870 con 0x7f26a8070510 2024-04-25T16:52:13.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.035+0000 7f26ad224640 1 -- 172.21.15.94:0/2628873042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a806a730 con 0x7f26a8070510 2024-04-25T16:52:13.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.035+0000 7f268f7fe640 1 -- 172.21.15.94:0/2628873042 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f269c01a070 con 0x7f26a8070510 2024-04-25T16:52:13.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.035+0000 7f268f7fe640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f269406f400 0x7f2694071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.035+0000 7f268f7fe640 1 -- 172.21.15.94:0/2628873042 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f269c08ed60 con 0x7f26a8070510 2024-04-25T16:52:13.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.035+0000 7f26a6d9d640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f269406f400 0x7f2694071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.035+0000 7f26a6d9d640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f269406f400 0x7f2694071910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f26980af000 tx=0x7f26980a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:13.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.039+0000 7f268f7fe640 1 -- 172.21.15.94:0/2628873042 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f269c021020 con 0x7f26a8070510 2024-04-25T16:52:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.203+0000 7f26ad224640 1 -- 172.21.15.94:0/2628873042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26a80b6040 con 0x7f26a8070510 2024-04-25T16:52:13.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.207+0000 7f268f7fe640 1 -- 172.21.15.94:0/2628873042 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7f269c05b9c0 con 0x7f26a8070510 2024-04-25T16:52:13.204 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:13.204 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:52:13.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.207+0000 7f26ad224640 1 -- 172.21.15.94:0/2628873042 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f269406f400 msgr2=0x7f2694071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.207+0000 7f26ad224640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f269406f400 0x7f2694071910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f26980af000 tx=0x7f26980a7070 comp rx=0 tx=0).stop 2024-04-25T16:52:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.207+0000 7f26ad224640 1 -- 172.21.15.94:0/2628873042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a8070510 msgr2=0x7f26a80b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.207+0000 7f26ad224640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a8070510 0x7f26a80b54f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f269c00c370 tx=0x7f269c00c830 comp rx=0 tx=0).stop 2024-04-25T16:52:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.207+0000 7f26ad224640 1 -- 172.21.15.94:0/2628873042 shutdown_connections 2024-04-25T16:52:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.207+0000 7f26ad224640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f269406f400 0x7f2694071910 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.207+0000 7f26ad224640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26a815e350 0x7f26a80b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.207+0000 7f26ad224640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a8070510 0x7f26a80b54f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.207+0000 7f26ad224640 1 --2- 172.21.15.94:0/2628873042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26a806d950 0x7f26a8167f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.207+0000 7f26ad224640 1 -- 172.21.15.94:0/2628873042 >> 172.21.15.94:0/2628873042 conn(0x7f26a8070f40 msgr2=0x7f26a8068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.207+0000 7f26ad224640 1 -- 172.21.15.94:0/2628873042 shutdown_connections 2024-04-25T16:52:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.207+0000 7f26ad224640 1 -- 172.21.15.94:0/2628873042 wait complete. 2024-04-25T16:52:13.226 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T16:52:13.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:52:13.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31cc0ec640 1 Processor -- start 2024-04-25T16:52:13.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31cc0ec640 1 -- start start 2024-04-25T16:52:13.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31cc0ec640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31c40b5440 0x7f31c40b5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31cc0ec640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f31c4160430 0x7f31c4160840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31cc0ec640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31c4160d80 con 0x7f31c40d1c90 2024-04-25T16:52:13.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31cc0ec640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f31c40d0be0 con 0x7f31c4160430 2024-04-25T16:52:13.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31cc0ec640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f31c406f120 con 0x7f31c40b5440 2024-04-25T16:52:13.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31ca689640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f31c4160430 0x7f31c4160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31ca689640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f31c4160430 0x7f31c4160840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48266/0 (socket says 172.21.15.94:48266) 2024-04-25T16:52:13.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31ca689640 1 -- 172.21.15.94:0/3507109821 learned_addr learned my addr 172.21.15.94:0/3507109821 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:13.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31c9687640 1 --2- 172.21.15.94:0/3507109821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31c40b5440 0x7f31c40b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31ca689640 1 -- 172.21.15.94:0/3507109821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31c40b5440 msgr2=0x7f31c40b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:13.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31ca689640 1 --2- 172.21.15.94:0/3507109821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31c40b5440 0x7f31c40b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.375+0000 7f31ca689640 1 -- 172.21.15.94:0/3507109821 >> v1:172.21.15.138:6789/0 conn(0x7f31c40d1c90 legacy=0x7f31c40d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.379+0000 7f31ca689640 1 -- 172.21.15.94:0/3507109821 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c406f400 con 0x7f31c4160430 2024-04-25T16:52:13.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.379+0000 7f31c9687640 1 --2- 172.21.15.94:0/3507109821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31c40b5440 0x7f31c40b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:13.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.379+0000 7f31ca689640 1 --2- 172.21.15.94:0/3507109821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31c4160430 0x7f31c4160840 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009870 tx=0x7f31c001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3f7c925319b8e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:13.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.379+0000 7f31c8e86640 1 -- 172.21.15.94:0/3507109821 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c00b1070 con 0x7f31c4160430 2024-04-25T16:52:13.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.379+0000 7f31c8e86640 1 -- 172.21.15.94:0/3507109821 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c00ac600 con 0x7f31c4160430 2024-04-25T16:52:13.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.379+0000 7f31c8e86640 1 -- 172.21.15.94:0/3507109821 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c00ac9d0 con 0x7f31c4160430 2024-04-25T16:52:13.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.379+0000 7f31cc0ec640 1 -- 172.21.15.94:0/3507109821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31c4160430 msgr2=0x7f31c4160840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.379+0000 7f31cc0ec640 1 --2- 172.21.15.94:0/3507109821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31c4160430 0x7f31c4160840 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009870 tx=0x7f31c001c4d0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 -- 172.21.15.94:0/3507109821 shutdown_connections 2024-04-25T16:52:13.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 --2- 172.21.15.94:0/3507109821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31c4160430 0x7f31c4160840 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 --2- 172.21.15.94:0/3507109821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31c40b5440 0x7f31c40b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 -- 172.21.15.94:0/3507109821 >> 172.21.15.94:0/3507109821 conn(0x7f31c4071150 msgr2=0x7f31c4071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:13.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 -- 172.21.15.94:0/3507109821 shutdown_connections 2024-04-25T16:52:13.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 -- 172.21.15.94:0/3507109821 wait complete. 2024-04-25T16:52:13.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 Processor -- start 2024-04-25T16:52:13.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 -- start start 2024-04-25T16:52:13.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31c40b5440 0x7f31c415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31c40d1c90 0x7f31c4148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31c4160430 0x7f31c415e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f31c40d1780 con 0x7f31c40d1c90 2024-04-25T16:52:13.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31c40d20a0 con 0x7f31c4160430 2024-04-25T16:52:13.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31cc0ec640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f31c40d1900 con 0x7f31c40b5440 2024-04-25T16:52:13.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31c9687640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31c40d1c90 0x7f31c4148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31c9687640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31c40d1c90 0x7f31c4148e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48268/0 (socket says 172.21.15.94:48268) 2024-04-25T16:52:13.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31c9687640 1 -- 172.21.15.94:0/2058173598 learned_addr learned my addr 172.21.15.94:0/2058173598 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:13.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31c9e88640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31c40b5440 0x7f31c415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31ca689640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31c4160430 0x7f31c415e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31c9687640 1 -- 172.21.15.94:0/2058173598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31c40b5440 msgr2=0x7f31c415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31c9687640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31c40b5440 0x7f31c415e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31c9687640 1 -- 172.21.15.94:0/2058173598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31c4160430 msgr2=0x7f31c415e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31c9687640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31c4160430 0x7f31c415e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31c9687640 1 -- 172.21.15.94:0/2058173598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c40b3f70 con 0x7f31c40d1c90 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31c9e88640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31c40b5440 0x7f31c415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.383+0000 7f31ca689640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31c4160430 0x7f31c415e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.387+0000 7f31c9687640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31c40d1c90 0x7f31c4148e40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f31b400d800 tx=0x7f31b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.387+0000 7f31ba7fc640 1 -- 172.21.15.94:0/2058173598 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4018020 con 0x7f31c40d1c90 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.387+0000 7f31ba7fc640 1 -- 172.21.15.94:0/2058173598 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b40096c0 con 0x7f31c40d1c90 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.387+0000 7f31cc0ec640 1 -- 172.21.15.94:0/2058173598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c415d2f0 con 0x7f31c40d1c90 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.387+0000 7f31ba7fc640 1 -- 172.21.15.94:0/2058173598 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4009ab0 con 0x7f31c40d1c90 2024-04-25T16:52:13.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.387+0000 7f31cc0ec640 1 -- 172.21.15.94:0/2058173598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c415d8b0 con 0x7f31c40d1c90 2024-04-25T16:52:13.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.387+0000 7f31cc0ec640 1 -- 172.21.15.94:0/2058173598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c406a730 con 0x7f31c40d1c90 2024-04-25T16:52:13.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.387+0000 7f31ba7fc640 1 -- 172.21.15.94:0/2058173598 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f31b401b020 con 0x7f31c40d1c90 2024-04-25T16:52:13.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.387+0000 7f31ba7fc640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f31a806f400 0x7f31a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.387+0000 7f31ba7fc640 1 -- 172.21.15.94:0/2058173598 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f31b408fc80 con 0x7f31c40d1c90 2024-04-25T16:52:13.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.387+0000 7f31c9e88640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f31a806f400 0x7f31a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.387+0000 7f31c9e88640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f31a806f400 0x7f31a8071910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f31bc007fd0 tx=0x7f31bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.391+0000 7f31ba7fc640 1 -- 172.21.15.94:0/2058173598 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f31b4022020 con 0x7f31c40d1c90 2024-04-25T16:52:13.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.575+0000 7f31cc0ec640 1 -- 172.21.15.94:0/2058173598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31c40b7580 con 0x7f31c40d1c90 2024-04-25T16:52:13.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.575+0000 7f31ba7fc640 1 -- 172.21.15.94:0/2058173598 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f31b405c770 con 0x7f31c40d1c90 2024-04-25T16:52:13.574 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:13.574 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2352186041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2352186041},{"type":"v1","addr":"172.21.15.138:6833","nonce":2352186041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3210927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3210927873},{"type":"v1","addr":"172.21.15.94:6835","nonce":3210927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2237413225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2237413225},{"type":"v1","addr":"172.21.15.94:6837","nonce":2237413225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:50:54.403299+0000","modified":"2024-04-25T16:51:18.139471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/3652545132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3652545132},{"type":"v1","addr":"172.21.15.138:6835","nonce":3652545132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T16:52:13.574 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:52:13.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.579+0000 7f31cc0ec640 1 -- 172.21.15.94:0/2058173598 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f31a806f400 msgr2=0x7f31a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.579+0000 7f31cc0ec640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f31a806f400 0x7f31a8071910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f31bc007fd0 tx=0x7f31bc009040 comp rx=0 tx=0).stop 2024-04-25T16:52:13.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.579+0000 7f31cc0ec640 1 -- 172.21.15.94:0/2058173598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31c40d1c90 msgr2=0x7f31c4148e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.579+0000 7f31cc0ec640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31c40d1c90 0x7f31c4148e40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f31b400d800 tx=0x7f31b400dcc0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.579+0000 7f31cc0ec640 1 -- 172.21.15.94:0/2058173598 shutdown_connections 2024-04-25T16:52:13.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.579+0000 7f31cc0ec640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31c4160430 0x7f31c415e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.579+0000 7f31cc0ec640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31c40d1c90 0x7f31c4148e40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.579+0000 7f31cc0ec640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f31a806f400 0x7f31a8071910 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.579+0000 7f31cc0ec640 1 --2- 172.21.15.94:0/2058173598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31c40b5440 0x7f31c415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.579+0000 7f31cc0ec640 1 -- 172.21.15.94:0/2058173598 >> 172.21.15.94:0/2058173598 conn(0x7f31c4071150 msgr2=0x7f31c40b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:13.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.579+0000 7f31cc0ec640 1 -- 172.21.15.94:0/2058173598 shutdown_connections 2024-04-25T16:52:13.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.579+0000 7f31cc0ec640 1 -- 172.21.15.94:0/2058173598 wait complete. 2024-04-25T16:52:13.594 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:52:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.747+0000 7fedcd433640 1 Processor -- start 2024-04-25T16:52:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.747+0000 7fedcd433640 1 -- start start 2024-04-25T16:52:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.747+0000 7fedcd433640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fedc815f9e0 0x7fedc80d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.747+0000 7fedcd433640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedc80d1940 con 0x7fedc80cf0e0 2024-04-25T16:52:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.747+0000 7fedcd433640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedc80b5360 con 0x7fedc80d1530 2024-04-25T16:52:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.747+0000 7fedcd433640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fedc80b6250 con 0x7fedc815f9e0 2024-04-25T16:52:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.747+0000 7fedc67fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fedc815f9e0 0x7fedc80d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.747+0000 7fedc67fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fedc815f9e0 0x7fedc80d0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48278/0 (socket says 172.21.15.94:48278) 2024-04-25T16:52:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.747+0000 7fedc67fc640 1 -- 172.21.15.94:0/1929075060 learned_addr learned my addr 172.21.15.94:0/1929075060 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.747+0000 7fedc67fc640 1 -- 172.21.15.94:0/1929075060 >> v1:172.21.15.138:6789/0 conn(0x7fedc80d1530 legacy=0x7fedc80af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.747+0000 7fedc67fc640 1 -- 172.21.15.94:0/1929075060 >> v1:172.21.15.94:6789/0 conn(0x7fedc80cf0e0 legacy=0x7fedc80b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.747+0000 7fedc67fc640 1 -- 172.21.15.94:0/1929075060 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc80b5640 con 0x7fedc815f9e0 2024-04-25T16:52:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.751+0000 7fedc67fc640 1 --2- 172.21.15.94:0/1929075060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc815f9e0 0x7fedc80d0ff0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009870 tx=0x7fedbc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed64a5b69fe80fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.751+0000 7fedc5ffb640 1 -- 172.21.15.94:0/1929075060 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0aa650 con 0x7fedc815f9e0 2024-04-25T16:52:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.751+0000 7fedc5ffb640 1 -- 172.21.15.94:0/1929075060 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedbc0af070 con 0x7fedc815f9e0 2024-04-25T16:52:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.751+0000 7fedc5ffb640 1 -- 172.21.15.94:0/1929075060 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0aaa10 con 0x7fedc815f9e0 2024-04-25T16:52:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.751+0000 7fedcd433640 1 -- 172.21.15.94:0/1929075060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc815f9e0 msgr2=0x7fedc80d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.751+0000 7fedcd433640 1 --2- 172.21.15.94:0/1929075060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc815f9e0 0x7fedc80d0ff0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009870 tx=0x7fedbc01c4f0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.751+0000 7fedcd433640 1 -- 172.21.15.94:0/1929075060 shutdown_connections 2024-04-25T16:52:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.751+0000 7fedcd433640 1 --2- 172.21.15.94:0/1929075060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc815f9e0 0x7fedc80d0ff0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.751+0000 7fedcd433640 1 -- 172.21.15.94:0/1929075060 >> 172.21.15.94:0/1929075060 conn(0x7fedc8070f40 msgr2=0x7fedc8071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:13.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.751+0000 7fedcd433640 1 -- 172.21.15.94:0/1929075060 shutdown_connections 2024-04-25T16:52:13.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.751+0000 7fedcd433640 1 -- 172.21.15.94:0/1929075060 wait complete. 2024-04-25T16:52:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedcd433640 1 Processor -- start 2024-04-25T16:52:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedcd433640 1 -- start start 2024-04-25T16:52:13.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedcd433640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedc80cf0e0 0x7fedc8158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedcd433640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedc80d1530 0x7fedc8160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedcd433640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc815f9e0 0x7fedc80b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedcd433640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fedc80cffb0 con 0x7fedc815f9e0 2024-04-25T16:52:13.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedcd433640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fedc815fdf0 con 0x7fedc80cf0e0 2024-04-25T16:52:13.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedcd433640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fedc80b6df0 con 0x7fedc80d1530 2024-04-25T16:52:13.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedc77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc815f9e0 0x7fedc80b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedc77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc815f9e0 0x7fedc80b1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48286/0 (socket says 172.21.15.94:48286) 2024-04-25T16:52:13.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedc77fe640 1 -- 172.21.15.94:0/1537751036 learned_addr learned my addr 172.21.15.94:0/1537751036 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:13.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedc67fc640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedc80d1530 0x7fedc8160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedc6ffd640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedc80cf0e0 0x7fedc8158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedc77fe640 1 -- 172.21.15.94:0/1537751036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedc80d1530 msgr2=0x7fedc8160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedc77fe640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedc80d1530 0x7fedc8160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedc77fe640 1 -- 172.21.15.94:0/1537751036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedc80cf0e0 msgr2=0x7fedc8158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedc77fe640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedc80cf0e0 0x7fedc8158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedc77fe640 1 -- 172.21.15.94:0/1537751036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc81600a0 con 0x7fedc815f9e0 2024-04-25T16:52:13.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedc77fe640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc815f9e0 0x7fedc80b1140 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fedc000b490 tx=0x7fedc000b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:13.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedaf7fe640 1 -- 172.21.15.94:0/1537751036 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0015020 con 0x7fedc815f9e0 2024-04-25T16:52:13.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedaf7fe640 1 -- 172.21.15.94:0/1537751036 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc0011830 con 0x7fedc815f9e0 2024-04-25T16:52:13.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedcd433640 1 -- 172.21.15.94:0/1537751036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc8158c40 con 0x7fedc815f9e0 2024-04-25T16:52:13.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedaf7fe640 1 -- 172.21.15.94:0/1537751036 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0011c20 con 0x7fedc815f9e0 2024-04-25T16:52:13.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedcd433640 1 -- 172.21.15.94:0/1537751036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc8158f20 con 0x7fedc815f9e0 2024-04-25T16:52:13.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.755+0000 7fedcd433640 1 -- 172.21.15.94:0/1537751036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc806a730 con 0x7fedc815f9e0 2024-04-25T16:52:13.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.759+0000 7fedaf7fe640 1 -- 172.21.15.94:0/1537751036 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fedc0018020 con 0x7fedc815f9e0 2024-04-25T16:52:13.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.759+0000 7fedaf7fe640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fedb406f400 0x7fedb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:13.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.759+0000 7fedaf7fe640 1 -- 172.21.15.94:0/1537751036 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fedc008f0f0 con 0x7fedc815f9e0 2024-04-25T16:52:13.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.759+0000 7fedc6ffd640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fedb406f400 0x7fedb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:13.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.759+0000 7fedc6ffd640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fedb406f400 0x7fedb4071910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fedc8073310 tx=0x7fedb8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:13.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.763+0000 7fedaf7fe640 1 -- 172.21.15.94:0/1537751036 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fedc001f030 con 0x7fedc815f9e0 2024-04-25T16:52:13.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.943+0000 7fedcd433640 1 -- 172.21.15.94:0/1537751036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedc81589d0 con 0x7fedc815f9e0 2024-04-25T16:52:13.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.947+0000 7fedaf7fe640 1 -- 172.21.15.94:0/1537751036 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fedc005bd00 con 0x7fedc815f9e0 2024-04-25T16:52:13.944 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:13.944 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2352186041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2352186041},{"type":"v1","addr":"172.21.15.138:6833","nonce":2352186041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3210927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3210927873},{"type":"v1","addr":"172.21.15.94:6835","nonce":3210927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2237413225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2237413225},{"type":"v1","addr":"172.21.15.94:6837","nonce":2237413225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:50:54.403299+0000","modified":"2024-04-25T16:51:18.139471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/3652545132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3652545132},{"type":"v1","addr":"172.21.15.138:6835","nonce":3652545132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T16:52:13.944 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T16:52:13.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.947+0000 7fedcd433640 1 -- 172.21.15.94:0/1537751036 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fedb406f400 msgr2=0x7fedb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.947+0000 7fedcd433640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fedb406f400 0x7fedb4071910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fedc8073310 tx=0x7fedb8009040 comp rx=0 tx=0).stop 2024-04-25T16:52:13.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.947+0000 7fedcd433640 1 -- 172.21.15.94:0/1537751036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc815f9e0 msgr2=0x7fedc80b1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:13.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.947+0000 7fedcd433640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc815f9e0 0x7fedc80b1140 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fedc000b490 tx=0x7fedc000b950 comp rx=0 tx=0).stop 2024-04-25T16:52:13.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.947+0000 7fedcd433640 1 -- 172.21.15.94:0/1537751036 shutdown_connections 2024-04-25T16:52:13.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.947+0000 7fedcd433640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedc815f9e0 0x7fedc80b1140 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.947+0000 7fedcd433640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fedb406f400 0x7fedb4071910 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.947+0000 7fedcd433640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedc80d1530 0x7fedc8160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.947+0000 7fedcd433640 1 --2- 172.21.15.94:0/1537751036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedc80cf0e0 0x7fedc8158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:13.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.947+0000 7fedcd433640 1 -- 172.21.15.94:0/1537751036 >> 172.21.15.94:0/1537751036 conn(0x7fedc8070f40 msgr2=0x7fedc806f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:13.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.947+0000 7fedcd433640 1 -- 172.21.15.94:0/1537751036 shutdown_connections 2024-04-25T16:52:13.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:13.947+0000 7fedcd433640 1 -- 172.21.15.94:0/1537751036 wait complete. 2024-04-25T16:52:13.964 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:52:14.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f13072c1640 1 Processor -- start 2024-04-25T16:52:14.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f13072c1640 1 -- start start 2024-04-25T16:52:14.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f13072c1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f130006f6c0 0x7f13000d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f13072c1640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f130015ff50 0x7f13000d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f13072c1640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f13000b7050 0x7f13000b7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f13072c1640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f13000d2fd0 con 0x7f13000b7050 2024-04-25T16:52:14.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f13072c1640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f13000d2070 con 0x7f130015ff50 2024-04-25T16:52:14.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f13072c1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f13000d2aa0 con 0x7f130006f6c0 2024-04-25T16:52:14.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f130585e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f13000b7050 0x7f13000b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f130585e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f13000b7050 0x7f13000b7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48290/0 (socket says 172.21.15.94:48290) 2024-04-25T16:52:14.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f130585e640 1 -- 172.21.15.94:0/1434327842 learned_addr learned my addr 172.21.15.94:0/1434327842 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:14.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f130505d640 1 --2- 172.21.15.94:0/1434327842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f130006f6c0 0x7f13000d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f130485c640 1 --2- 172.21.15.94:0/1434327842 >> v2:172.21.15.138:3300/0 conn(0x7f130015ff50 0x7f13000d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f130585e640 1 -- 172.21.15.94:0/1434327842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f130006f6c0 msgr2=0x7f13000d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:14.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f130585e640 1 --2- 172.21.15.94:0/1434327842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f130006f6c0 0x7f13000d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f130585e640 1 -- 172.21.15.94:0/1434327842 >> v2:172.21.15.138:3300/0 conn(0x7f130015ff50 msgr2=0x7f13000d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:14.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f130585e640 1 --2- 172.21.15.94:0/1434327842 >> v2:172.21.15.138:3300/0 conn(0x7f130015ff50 0x7f13000d3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f130585e640 1 -- 172.21.15.94:0/1434327842 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13000d1d40 con 0x7f13000b7050 2024-04-25T16:52:14.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.103+0000 7f130505d640 1 --2- 172.21.15.94:0/1434327842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f130006f6c0 0x7f13000d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:14.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.107+0000 7f130585e640 1 --2- 172.21.15.94:0/1434327842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13000b7050 0x7f13000b7460 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f130009f080 tx=0x7f12f801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581461ed64023819 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:14.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.107+0000 7f12f7fff640 1 -- 172.21.15.94:0/1434327842 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80a5890 con 0x7f13000b7050 2024-04-25T16:52:14.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.107+0000 7f12f7fff640 1 -- 172.21.15.94:0/1434327842 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f80a5a30 con 0x7f13000b7050 2024-04-25T16:52:14.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.107+0000 7f12f7fff640 1 -- 172.21.15.94:0/1434327842 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80a5e00 con 0x7f13000b7050 2024-04-25T16:52:14.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.107+0000 7f13072c1640 1 -- 172.21.15.94:0/1434327842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13000b7050 msgr2=0x7f13000b7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.107+0000 7f13072c1640 1 --2- 172.21.15.94:0/1434327842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13000b7050 0x7f13000b7460 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f130009f080 tx=0x7f12f801c9f0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.107+0000 7f13072c1640 1 -- 172.21.15.94:0/1434327842 shutdown_connections 2024-04-25T16:52:14.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.107+0000 7f13072c1640 1 --2- 172.21.15.94:0/1434327842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13000b7050 0x7f13000b7460 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.107+0000 7f13072c1640 1 --2- 172.21.15.94:0/1434327842 >> v2:172.21.15.138:3300/0 conn(0x7f130015ff50 0x7f13000d3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.107+0000 7f13072c1640 1 --2- 172.21.15.94:0/1434327842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f130006f6c0 0x7f13000d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.107+0000 7f13072c1640 1 -- 172.21.15.94:0/1434327842 >> 172.21.15.94:0/1434327842 conn(0x7f130009e950 msgr2=0x7f130008d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:14.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.107+0000 7f13072c1640 1 -- 172.21.15.94:0/1434327842 shutdown_connections 2024-04-25T16:52:14.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.107+0000 7f13072c1640 1 -- 172.21.15.94:0/1434327842 wait complete. 2024-04-25T16:52:14.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f13072c1640 1 Processor -- start 2024-04-25T16:52:14.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f13072c1640 1 -- start start 2024-04-25T16:52:14.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f13072c1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f130006f6c0 0x7f130015cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f13072c1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13000b7050 0x7f13000cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f13072c1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f130015ff50 0x7f1300154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f13072c1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f130006fd90 con 0x7f130015ff50 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f13072c1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f130006fc10 con 0x7f13000b7050 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f13072c1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f13000d5600 con 0x7f130006f6c0 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f130585e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f130015ff50 0x7f1300154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f130585e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f130015ff50 0x7f1300154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48294/0 (socket says 172.21.15.94:48294) 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f130585e640 1 -- 172.21.15.94:0/1284250991 learned_addr learned my addr 172.21.15.94:0/1284250991 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f130505d640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f130006f6c0 0x7f130015cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f130485c640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13000b7050 0x7f13000cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f130585e640 1 -- 172.21.15.94:0/1284250991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f130006f6c0 msgr2=0x7f130015cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f130585e640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f130006f6c0 0x7f130015cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f130585e640 1 -- 172.21.15.94:0/1284250991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13000b7050 msgr2=0x7f13000cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f130585e640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13000b7050 0x7f13000cb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f130585e640 1 -- 172.21.15.94:0/1284250991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13000cd460 con 0x7f130015ff50 2024-04-25T16:52:14.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f130505d640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f130006f6c0 0x7f130015cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:14.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f130485c640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13000b7050 0x7f13000cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:14.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f130585e640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f130015ff50 0x7f1300154440 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f12f80af040 tx=0x7f12f8019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:14.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f12f5ffb640 1 -- 172.21.15.94:0/1284250991 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80b8880 con 0x7f130015ff50 2024-04-25T16:52:14.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f12f5ffb640 1 -- 172.21.15.94:0/1284250991 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f80be070 con 0x7f130015ff50 2024-04-25T16:52:14.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f13072c1640 1 -- 172.21.15.94:0/1284250991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13000cd740 con 0x7f130015ff50 2024-04-25T16:52:14.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f12f5ffb640 1 -- 172.21.15.94:0/1284250991 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80b8c80 con 0x7f130015ff50 2024-04-25T16:52:14.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f13072c1640 1 -- 172.21.15.94:0/1284250991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13000cdd00 con 0x7f130015ff50 2024-04-25T16:52:14.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.111+0000 7f13072c1640 1 -- 172.21.15.94:0/1284250991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f130015d3b0 con 0x7f130015ff50 2024-04-25T16:52:14.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.115+0000 7f12f5ffb640 1 -- 172.21.15.94:0/1284250991 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f12f80c3460 con 0x7f130015ff50 2024-04-25T16:52:14.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.115+0000 7f12f5ffb640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f12e406f400 0x7f12e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.115+0000 7f12f5ffb640 1 -- 172.21.15.94:0/1284250991 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f12f812a970 con 0x7f130015ff50 2024-04-25T16:52:14.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.115+0000 7f130505d640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f12e406f400 0x7f12e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.115+0000 7f130505d640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f12e406f400 0x7f12e4071910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f13000d0280 tx=0x7f12fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:14.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.119+0000 7f12f5ffb640 1 -- 172.21.15.94:0/1284250991 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f12f80165e0 con 0x7f130015ff50 2024-04-25T16:52:14.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.279+0000 7f13072c1640 1 -- 172.21.15.94:0/1284250991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1300076830 con 0x7f130015ff50 2024-04-25T16:52:14.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.279+0000 7f12f5ffb640 1 -- 172.21.15.94:0/1284250991 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18913 (secure 0 0 0) 0x7f12f8128e20 con 0x7f130015ff50 2024-04-25T16:52:14.279 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:14.279 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:51:34.863495+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:50:52.026933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:50:53.368977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T16:50:57.106043+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:52:14.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.283+0000 7f13072c1640 1 -- 172.21.15.94:0/1284250991 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f12e406f400 msgr2=0x7f12e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.283+0000 7f13072c1640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f12e406f400 0x7f12e4071910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f13000d0280 tx=0x7f12fc008040 comp rx=0 tx=0).stop 2024-04-25T16:52:14.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.283+0000 7f13072c1640 1 -- 172.21.15.94:0/1284250991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f130015ff50 msgr2=0x7f1300154440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.283+0000 7f13072c1640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f130015ff50 0x7f1300154440 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f12f80af040 tx=0x7f12f8019790 comp rx=0 tx=0).stop 2024-04-25T16:52:14.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.283+0000 7f13072c1640 1 -- 172.21.15.94:0/1284250991 shutdown_connections 2024-04-25T16:52:14.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.283+0000 7f13072c1640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f130015ff50 0x7f1300154440 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.283+0000 7f13072c1640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13000b7050 0x7f13000cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.283+0000 7f13072c1640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f12e406f400 0x7f12e4071910 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.283+0000 7f13072c1640 1 --2- 172.21.15.94:0/1284250991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f130006f6c0 0x7f130015cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.283+0000 7f13072c1640 1 -- 172.21.15.94:0/1284250991 >> 172.21.15.94:0/1284250991 conn(0x7f130009e950 msgr2=0x7f13000d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:14.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.283+0000 7f13072c1640 1 -- 172.21.15.94:0/1284250991 shutdown_connections 2024-04-25T16:52:14.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.283+0000 7f13072c1640 1 -- 172.21.15.94:0/1284250991 wait complete. 2024-04-25T16:52:14.301 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T16:52:14.383 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:52:14.385 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:52:14.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.451+0000 7fa6a80f0640 1 Processor -- start 2024-04-25T16:52:14.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.451+0000 7fa6a80f0640 1 -- start start 2024-04-25T16:52:14.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.455+0000 7fa6a80f0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a00ccf30 0x7fa6a015da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.455+0000 7fa6a80f0640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa6a015dfc0 0x7fa6a015e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.455+0000 7fa6a80f0640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a006d510 con 0x7fa6a0161480 2024-04-25T16:52:14.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.455+0000 7fa6a80f0640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a006f0d0 con 0x7fa6a015dfc0 2024-04-25T16:52:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.455+0000 7fa6a80f0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a00cd700 con 0x7fa6a00ccf30 2024-04-25T16:52:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.455+0000 7fa6a568b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa6a015dfc0 0x7fa6a015e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.455+0000 7fa6a568b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa6a015dfc0 0x7fa6a015e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48308/0 (socket says 172.21.15.94:48308) 2024-04-25T16:52:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.455+0000 7fa6a568b640 1 -- 172.21.15.94:0/3225243309 learned_addr learned my addr 172.21.15.94:0/3225243309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.455+0000 7fa6a5e8c640 1 --2- 172.21.15.94:0/3225243309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a00ccf30 0x7fa6a015da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.455+0000 7fa6a568b640 1 -- 172.21.15.94:0/3225243309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a00ccf30 msgr2=0x7fa6a015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.455+0000 7fa6a568b640 1 --2- 172.21.15.94:0/3225243309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a00ccf30 0x7fa6a015da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.455+0000 7fa6a568b640 1 -- 172.21.15.94:0/3225243309 >> v1:172.21.15.94:6789/0 conn(0x7fa6a0161480 legacy=0x7fa6a0161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.455+0000 7fa6a568b640 1 -- 172.21.15.94:0/3225243309 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a00cd9e0 con 0x7fa6a015dfc0 2024-04-25T16:52:14.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a568b640 1 --2- 172.21.15.94:0/3225243309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6a015dfc0 0x7fa6a015e3d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa690009870 tx=0x7fa69001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34069e5c6d31a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:14.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a4e8a640 1 -- 172.21.15.94:0/3225243309 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6900ab650 con 0x7fa6a015dfc0 2024-04-25T16:52:14.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a4e8a640 1 -- 172.21.15.94:0/3225243309 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6900b0070 con 0x7fa6a015dfc0 2024-04-25T16:52:14.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a4e8a640 1 -- 172.21.15.94:0/3225243309 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6900aba10 con 0x7fa6a015dfc0 2024-04-25T16:52:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a80f0640 1 -- 172.21.15.94:0/3225243309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6a015dfc0 msgr2=0x7fa6a015e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a80f0640 1 --2- 172.21.15.94:0/3225243309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6a015dfc0 0x7fa6a015e3d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa690009870 tx=0x7fa69001c560 comp rx=0 tx=0).stop 2024-04-25T16:52:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a80f0640 1 -- 172.21.15.94:0/3225243309 shutdown_connections 2024-04-25T16:52:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a80f0640 1 --2- 172.21.15.94:0/3225243309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6a015dfc0 0x7fa6a015e3d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a80f0640 1 --2- 172.21.15.94:0/3225243309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a00ccf30 0x7fa6a015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a80f0640 1 -- 172.21.15.94:0/3225243309 >> 172.21.15.94:0/3225243309 conn(0x7fa6a00ce490 msgr2=0x7fa6a00ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a80f0640 1 -- 172.21.15.94:0/3225243309 shutdown_connections 2024-04-25T16:52:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a80f0640 1 -- 172.21.15.94:0/3225243309 wait complete. 2024-04-25T16:52:14.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a80f0640 1 Processor -- start 2024-04-25T16:52:14.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.459+0000 7fa6a80f0640 1 -- start start 2024-04-25T16:52:14.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a80f0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6a00ccf30 0x7fa6a015b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a80f0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a015dfc0 0x7fa6a00cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a80f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6a0161480 0x7fa6a00b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a80f0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a016b1b0 con 0x7fa6a00ccf30 2024-04-25T16:52:14.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a80f0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a016b030 con 0x7fa6a0161480 2024-04-25T16:52:14.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a80f0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a016b330 con 0x7fa6a015dfc0 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a5e8c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6a00ccf30 0x7fa6a015b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a5e8c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6a00ccf30 0x7fa6a015b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48320/0 (socket says 172.21.15.94:48320) 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a5e8c640 1 -- 172.21.15.94:0/2358510191 learned_addr learned my addr 172.21.15.94:0/2358510191 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a5e8c640 1 -- 172.21.15.94:0/2358510191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a015dfc0 msgr2=0x7fa6a00cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a5e8c640 1 --2- 172.21.15.94:0/2358510191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a015dfc0 0x7fa6a00cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a5e8c640 1 -- 172.21.15.94:0/2358510191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6a0161480 msgr2=0x7fa6a00b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a5e8c640 1 --2- 172.21.15.94:0/2358510191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6a0161480 0x7fa6a00b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a5e8c640 1 -- 172.21.15.94:0/2358510191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a00c9910 con 0x7fa6a00ccf30 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a5e8c640 1 --2- 172.21.15.94:0/2358510191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6a00ccf30 0x7fa6a015b2d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00d5d0 tx=0x7fa69c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6967fc640 1 -- 172.21.15.94:0/2358510191 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c004240 con 0x7fa6a00ccf30 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6967fc640 1 -- 172.21.15.94:0/2358510191 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69c0043e0 con 0x7fa6a00ccf30 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a80f0640 1 -- 172.21.15.94:0/2358510191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a00c9bf0 con 0x7fa6a00ccf30 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6967fc640 1 -- 172.21.15.94:0/2358510191 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c003840 con 0x7fa6a00ccf30 2024-04-25T16:52:14.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a80f0640 1 -- 172.21.15.94:0/2358510191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a00ca1b0 con 0x7fa6a00ccf30 2024-04-25T16:52:14.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6a80f0640 1 -- 172.21.15.94:0/2358510191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a006a730 con 0x7fa6a00ccf30 2024-04-25T16:52:14.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6967fc640 1 -- 172.21.15.94:0/2358510191 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa69c0039e0 con 0x7fa6a00ccf30 2024-04-25T16:52:14.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.463+0000 7fa6967fc640 1 --2- 172.21.15.94:0/2358510191 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa68406f400 0x7fa684071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.467+0000 7fa6a568b640 1 --2- 172.21.15.94:0/2358510191 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa68406f400 0x7fa684071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.467+0000 7fa6967fc640 1 -- 172.21.15.94:0/2358510191 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa69c08f9a0 con 0x7fa6a00ccf30 2024-04-25T16:52:14.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.467+0000 7fa6a568b640 1 --2- 172.21.15.94:0/2358510191 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa68406f400 0x7fa684071910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa6900af000 tx=0x7fa690004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:14.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.471+0000 7fa6967fc640 1 -- 172.21.15.94:0/2358510191 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa69c05c490 con 0x7fa6a00ccf30 2024-04-25T16:52:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.667+0000 7fa6a80f0640 1 -- 172.21.15.94:0/2358510191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa6a00c5320 con 0x7fa6a00ccf30 2024-04-25T16:52:14.666 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-04-25T16:52:14.667+0000 7fb03286f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T16:52:14.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.671+0000 7fa6967fc640 1 -- 172.21.15.94:0/2358510191 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fa69c05bd90 con 0x7fa6a00ccf30 2024-04-25T16:52:14.669 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T16:52:14.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.671+0000 7fa6a80f0640 1 -- 172.21.15.94:0/2358510191 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa68406f400 msgr2=0x7fa684071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.675+0000 7fa6a80f0640 1 --2- 172.21.15.94:0/2358510191 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa68406f400 0x7fa684071910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa6900af000 tx=0x7fa690004740 comp rx=0 tx=0).stop 2024-04-25T16:52:14.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.675+0000 7fa6a80f0640 1 -- 172.21.15.94:0/2358510191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6a00ccf30 msgr2=0x7fa6a015b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.675+0000 7fa6a80f0640 1 --2- 172.21.15.94:0/2358510191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6a00ccf30 0x7fa6a015b2d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00d5d0 tx=0x7fa69c00da90 comp rx=0 tx=0).stop 2024-04-25T16:52:14.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.675+0000 7fa6a80f0640 1 -- 172.21.15.94:0/2358510191 shutdown_connections 2024-04-25T16:52:14.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.675+0000 7fa6a80f0640 1 --2- 172.21.15.94:0/2358510191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6a0161480 0x7fa6a00b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.675+0000 7fa6a80f0640 1 --2- 172.21.15.94:0/2358510191 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa68406f400 0x7fa684071910 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.675+0000 7fa6a80f0640 1 --2- 172.21.15.94:0/2358510191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a015dfc0 0x7fa6a00cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.675+0000 7fa6a80f0640 1 --2- 172.21.15.94:0/2358510191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6a00ccf30 0x7fa6a015b2d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.675+0000 7fa6a80f0640 1 -- 172.21.15.94:0/2358510191 >> 172.21.15.94:0/2358510191 conn(0x7fa6a00ce490 msgr2=0x7fa6a015ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:14.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.675+0000 7fa6a80f0640 1 -- 172.21.15.94:0/2358510191 shutdown_connections 2024-04-25T16:52:14.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.675+0000 7fa6a80f0640 1 -- 172.21.15.94:0/2358510191 wait complete. 2024-04-25T16:52:14.689 INFO:tasks.ceph.mds.b.smithi138.stderr: -5221> 2024-04-25T16:50:58.438+0000 7f7b7a727640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T16:52:14.693 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T16:52:14.800 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-04-25T16:52:14.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9cd16fe640 1 Processor -- start 2024-04-25T16:52:14.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9cd16fe640 1 -- start start 2024-04-25T16:52:14.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9cd16fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9ccc0cfe80 0x7f9ccc153900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9cd16fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9ccc0ce8c0 0x7f9ccc0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9cd16fe640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ccc0ccb10 con 0x7f9ccc06e010 2024-04-25T16:52:14.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9cd16fe640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f9ccc15d3f0 con 0x7f9ccc0ce8c0 2024-04-25T16:52:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9cd16fe640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9ccc15d6d0 con 0x7f9ccc0cfe80 2024-04-25T16:52:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9ccb7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9ccc0ce8c0 0x7f9ccc0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9ccb7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9ccc0ce8c0 0x7f9ccc0cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48330/0 (socket says 172.21.15.94:48330) 2024-04-25T16:52:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9ccb7fe640 1 -- 172.21.15.94:0/1451009704 learned_addr learned my addr 172.21.15.94:0/1451009704 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9ccaffd640 1 --2- 172.21.15.94:0/1451009704 >> v2:172.21.15.138:3300/0 conn(0x7f9ccc0cfe80 0x7f9ccc153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9ccb7fe640 1 -- 172.21.15.94:0/1451009704 >> v2:172.21.15.138:3300/0 conn(0x7f9ccc0cfe80 msgr2=0x7f9ccc153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9ccb7fe640 1 --2- 172.21.15.94:0/1451009704 >> v2:172.21.15.138:3300/0 conn(0x7f9ccc0cfe80 0x7f9ccc153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9ccb7fe640 1 -- 172.21.15.94:0/1451009704 >> v1:172.21.15.138:6789/0 conn(0x7f9ccc06e010 legacy=0x7f9ccc0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.843+0000 7f9ccb7fe640 1 -- 172.21.15.94:0/1451009704 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc15d9b0 con 0x7f9ccc0ce8c0 2024-04-25T16:52:14.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9ccb7fe640 1 --2- 172.21.15.94:0/1451009704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ccc0ce8c0 0x7f9ccc0cd010 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc06cb70 tx=0x7f9cc401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d75e4f9d84a7e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:14.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9cc9ffb640 1 -- 172.21.15.94:0/1451009704 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40a56c0 con 0x7f9ccc0ce8c0 2024-04-25T16:52:14.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9cc9ffb640 1 -- 172.21.15.94:0/1451009704 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc40a5860 con 0x7f9ccc0ce8c0 2024-04-25T16:52:14.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9cc9ffb640 1 -- 172.21.15.94:0/1451009704 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40a5c30 con 0x7f9ccc0ce8c0 2024-04-25T16:52:14.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9cd16fe640 1 -- 172.21.15.94:0/1451009704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ccc0ce8c0 msgr2=0x7f9ccc0cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9cd16fe640 1 --2- 172.21.15.94:0/1451009704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ccc0ce8c0 0x7f9ccc0cd010 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc06cb70 tx=0x7f9cc401c9f0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9cd16fe640 1 -- 172.21.15.94:0/1451009704 shutdown_connections 2024-04-25T16:52:14.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9cd16fe640 1 --2- 172.21.15.94:0/1451009704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ccc0ce8c0 0x7f9ccc0cd010 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9cd16fe640 1 --2- 172.21.15.94:0/1451009704 >> v2:172.21.15.138:3300/0 conn(0x7f9ccc0cfe80 0x7f9ccc153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9cd16fe640 1 -- 172.21.15.94:0/1451009704 >> 172.21.15.94:0/1451009704 conn(0x7f9ccc0ced30 msgr2=0x7f9ccc0cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:14.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9cd16fe640 1 -- 172.21.15.94:0/1451009704 shutdown_connections 2024-04-25T16:52:14.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9cd16fe640 1 -- 172.21.15.94:0/1451009704 wait complete. 2024-04-25T16:52:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9cd16fe640 1 Processor -- start 2024-04-25T16:52:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.847+0000 7f9cd16fe640 1 -- start start 2024-04-25T16:52:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cd16fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ccc06e010 0x7f9ccc148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cd16fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ccc0cfe80 0x7f9ccc14d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cd16fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ccc14d610 0x7f9ccc14e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cd16fe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc06e5a0 con 0x7f9ccc06e010 2024-04-25T16:52:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cd16fe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc06e420 con 0x7f9ccc14d610 2024-04-25T16:52:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cd16fe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9ccc06fbc0 con 0x7f9ccc0cfe80 2024-04-25T16:52:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9ccaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ccc06e010 0x7f9ccc148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9ccaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ccc06e010 0x7f9ccc148a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48340/0 (socket says 172.21.15.94:48340) 2024-04-25T16:52:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9ccaffd640 1 -- 172.21.15.94:0/736558063 learned_addr learned my addr 172.21.15.94:0/736558063 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9ccb7fe640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ccc14d610 0x7f9ccc14e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cca7fc640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ccc0cfe80 0x7f9ccc14d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9ccaffd640 1 -- 172.21.15.94:0/736558063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ccc0cfe80 msgr2=0x7f9ccc14d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9ccaffd640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ccc0cfe80 0x7f9ccc14d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9ccaffd640 1 -- 172.21.15.94:0/736558063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ccc14d610 msgr2=0x7f9ccc14e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9ccaffd640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ccc14d610 0x7f9ccc14e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9ccaffd640 1 -- 172.21.15.94:0/736558063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc14e0a0 con 0x7f9ccc06e010 2024-04-25T16:52:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9ccaffd640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ccc06e010 0x7f9ccc148a70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000d5f0 tx=0x7f9cc000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cb37fe640 1 -- 172.21.15.94:0/736558063 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00125f0 con 0x7f9ccc06e010 2024-04-25T16:52:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cb37fe640 1 -- 172.21.15.94:0/736558063 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc0019070 con 0x7f9ccc06e010 2024-04-25T16:52:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cd16fe640 1 -- 172.21.15.94:0/736558063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc14b190 con 0x7f9ccc06e010 2024-04-25T16:52:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cb37fe640 1 -- 172.21.15.94:0/736558063 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00129b0 con 0x7f9ccc06e010 2024-04-25T16:52:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cd16fe640 1 -- 172.21.15.94:0/736558063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ccc14b430 con 0x7f9ccc06e010 2024-04-25T16:52:14.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cd16fe640 1 -- 172.21.15.94:0/736558063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ccc06a730 con 0x7f9ccc06e010 2024-04-25T16:52:14.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cb37fe640 1 -- 172.21.15.94:0/736558063 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9cc0012b50 con 0x7f9ccc06e010 2024-04-25T16:52:14.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cb37fe640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9cb806f430 0x7f9cb8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:14.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cb37fe640 1 -- 172.21.15.94:0/736558063 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f9cc008f9a0 con 0x7f9ccc06e010 2024-04-25T16:52:14.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.851+0000 7f9cca7fc640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9cb806f430 0x7f9cb8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:14.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.855+0000 7f9cca7fc640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9cb806f430 0x7f9cb8071940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc1497d0 tx=0x7f9cbc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:14.859+0000 7f9cb37fe640 1 -- 172.21.15.94:0/736558063 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9cc005be70 con 0x7f9ccc06e010 2024-04-25T16:52:15.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.043+0000 7f9cd16fe640 1 -- 172.21.15.94:0/736558063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9ccc0d1cd0 con 0x7f9ccc06e010 2024-04-25T16:52:15.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.675+0000 7f9cb37fe640 1 -- 172.21.15.94:0/736558063 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f9cc0060170 con 0x7f9ccc06e010 2024-04-25T16:52:15.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.675+0000 7f9cd16fe640 1 -- 172.21.15.94:0/736558063 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9cb806f430 msgr2=0x7f9cb8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:15.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.675+0000 7f9cd16fe640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9cb806f430 0x7f9cb8071940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc1497d0 tx=0x7f9cbc009040 comp rx=0 tx=0).stop 2024-04-25T16:52:15.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.675+0000 7f9cd16fe640 1 -- 172.21.15.94:0/736558063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ccc06e010 msgr2=0x7f9ccc148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:15.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.675+0000 7f9cd16fe640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ccc06e010 0x7f9ccc148a70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000d5f0 tx=0x7f9cc000dab0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.679+0000 7f9cd16fe640 1 -- 172.21.15.94:0/736558063 shutdown_connections 2024-04-25T16:52:15.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.679+0000 7f9cd16fe640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ccc14d610 0x7f9ccc14e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.679+0000 7f9cd16fe640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9cb806f430 0x7f9cb8071940 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.679+0000 7f9cd16fe640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9ccc0cfe80 0x7f9ccc14d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.679+0000 7f9cd16fe640 1 --2- 172.21.15.94:0/736558063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ccc06e010 0x7f9ccc148a70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.679+0000 7f9cd16fe640 1 -- 172.21.15.94:0/736558063 >> 172.21.15.94:0/736558063 conn(0x7f9ccc0ced30 msgr2=0x7f9ccc0d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:15.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.679+0000 7f9cd16fe640 1 -- 172.21.15.94:0/736558063 shutdown_connections 2024-04-25T16:52:15.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.679+0000 7f9cd16fe640 1 -- 172.21.15.94:0/736558063 wait complete. 2024-04-25T16:52:15.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T16:52:15.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.867+0000 7fd2effc3640 1 Processor -- start 2024-04-25T16:52:15.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.867+0000 7fd2effc3640 1 -- start start 2024-04-25T16:52:15.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2effc3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd2e815de60 0x7fd2e80b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:15.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2effc3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2e80b3120 0x7fd2e80b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:15.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2effc3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2e80b94b0 con 0x7fd2e80ce080 2024-04-25T16:52:15.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2effc3640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd2e815e270 con 0x7fd2e815de60 2024-04-25T16:52:15.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2effc3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd2e80d2750 con 0x7fd2e80b3120 2024-04-25T16:52:15.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2edd5f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd2e815de60 0x7fd2e80b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:15.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2edd5f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd2e815de60 0x7fd2e80b8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48350/0 (socket says 172.21.15.94:48350) 2024-04-25T16:52:15.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2edd5f640 1 -- 172.21.15.94:0/604484977 learned_addr learned my addr 172.21.15.94:0/604484977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:15.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2ed55e640 1 --1- 172.21.15.94:0/604484977 >> v1:172.21.15.94:6789/0 conn(0x7fd2e80ce080 0x7fd2e80ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:43152/0 (socket says 172.21.15.94:43152) 2024-04-25T16:52:15.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2ee560640 1 --2- 172.21.15.94:0/604484977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2e80b3120 0x7fd2e80b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:15.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2edd5f640 1 -- 172.21.15.94:0/604484977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2e80b3120 msgr2=0x7fd2e80b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:15.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2edd5f640 1 --2- 172.21.15.94:0/604484977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2e80b3120 0x7fd2e80b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2edd5f640 1 -- 172.21.15.94:0/604484977 >> v1:172.21.15.94:6789/0 conn(0x7fd2e80ce080 legacy=0x7fd2e80ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:15.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2edd5f640 1 -- 172.21.15.94:0/604484977 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e80d3290 con 0x7fd2e815de60 2024-04-25T16:52:15.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.871+0000 7fd2edd5f640 1 --2- 172.21.15.94:0/604484977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2e815de60 0x7fd2e80b8f70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009870 tx=0x7fd2e401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c44c7709df24161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:15.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.875+0000 7fd2ecd5d640 1 -- 172.21.15.94:0/604484977 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e40b1070 con 0x7fd2e815de60 2024-04-25T16:52:15.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.875+0000 7fd2ecd5d640 1 -- 172.21.15.94:0/604484977 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e40ac620 con 0x7fd2e815de60 2024-04-25T16:52:15.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.875+0000 7fd2ecd5d640 1 -- 172.21.15.94:0/604484977 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e40ac9f0 con 0x7fd2e815de60 2024-04-25T16:52:15.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.875+0000 7fd2effc3640 1 -- 172.21.15.94:0/604484977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2e815de60 msgr2=0x7fd2e80b8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:15.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.875+0000 7fd2effc3640 1 --2- 172.21.15.94:0/604484977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2e815de60 0x7fd2e80b8f70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009870 tx=0x7fd2e401c4d0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.875+0000 7fd2effc3640 1 -- 172.21.15.94:0/604484977 shutdown_connections 2024-04-25T16:52:15.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.875+0000 7fd2effc3640 1 --2- 172.21.15.94:0/604484977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2e80b3120 0x7fd2e80b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.875+0000 7fd2effc3640 1 --2- 172.21.15.94:0/604484977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2e815de60 0x7fd2e80b8f70 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.875+0000 7fd2effc3640 1 -- 172.21.15.94:0/604484977 >> 172.21.15.94:0/604484977 conn(0x7fd2e80d34e0 msgr2=0x7fd2e80d38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:15.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.875+0000 7fd2effc3640 1 -- 172.21.15.94:0/604484977 shutdown_connections 2024-04-25T16:52:15.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.875+0000 7fd2effc3640 1 -- 172.21.15.94:0/604484977 wait complete. 2024-04-25T16:52:15.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.875+0000 7fd2effc3640 1 Processor -- start 2024-04-25T16:52:15.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.875+0000 7fd2effc3640 1 -- start start 2024-04-25T16:52:15.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2effc3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2e80b3120 0x7fd2e8157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:15.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2effc3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2e80ce080 0x7fd2e814d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:15.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2effc3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2e815de60 0x7fd2e8159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:15.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2effc3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e816acc0 con 0x7fd2e80b3120 2024-04-25T16:52:15.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2effc3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e80d2fb0 con 0x7fd2e80ce080 2024-04-25T16:52:15.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2effc3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd2e816ae40 con 0x7fd2e815de60 2024-04-25T16:52:15.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2edd5f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2e80b3120 0x7fd2e8157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2edd5f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2e80b3120 0x7fd2e8157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48352/0 (socket says 172.21.15.94:48352) 2024-04-25T16:52:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2edd5f640 1 -- 172.21.15.94:0/589506430 learned_addr learned my addr 172.21.15.94:0/589506430 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2ee560640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2e815de60 0x7fd2e8159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2ed55e640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2e80ce080 0x7fd2e814d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2edd5f640 1 -- 172.21.15.94:0/589506430 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2e815de60 msgr2=0x7fd2e8159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2edd5f640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2e815de60 0x7fd2e8159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2edd5f640 1 -- 172.21.15.94:0/589506430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2e80ce080 msgr2=0x7fd2e814d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2edd5f640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2e80ce080 0x7fd2e814d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2edd5f640 1 -- 172.21.15.94:0/589506430 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e815fc70 con 0x7fd2e80b3120 2024-04-25T16:52:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2ee560640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2e815de60 0x7fd2e8159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2ed55e640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2e80ce080 0x7fd2e814d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2edd5f640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2e80b3120 0x7fd2e8157be0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd2e401a100 tx=0x7fd2e4019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2de7fc640 1 -- 172.21.15.94:0/589506430 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e40b9020 con 0x7fd2e80b3120 2024-04-25T16:52:15.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2de7fc640 1 -- 172.21.15.94:0/589506430 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e40b5610 con 0x7fd2e80b3120 2024-04-25T16:52:15.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2effc3640 1 -- 172.21.15.94:0/589506430 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e815ff50 con 0x7fd2e80b3120 2024-04-25T16:52:15.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2de7fc640 1 -- 172.21.15.94:0/589506430 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e40b1040 con 0x7fd2e80b3120 2024-04-25T16:52:15.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2effc3640 1 -- 172.21.15.94:0/589506430 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e80b6b00 con 0x7fd2e80b3120 2024-04-25T16:52:15.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2effc3640 1 -- 172.21.15.94:0/589506430 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e80d1620 con 0x7fd2e80b3120 2024-04-25T16:52:15.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2de7fc640 1 -- 172.21.15.94:0/589506430 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd2e40bc020 con 0x7fd2e80b3120 2024-04-25T16:52:15.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2de7fc640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd2cc06f430 0x7fd2cc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:15.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2de7fc640 1 -- 172.21.15.94:0/589506430 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7fd2e41294e0 con 0x7fd2e80b3120 2024-04-25T16:52:15.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.879+0000 7fd2ed55e640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd2cc06f430 0x7fd2cc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:15.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.883+0000 7fd2ed55e640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd2cc06f430 0x7fd2cc071940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8000f70 tx=0x7fd2d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:15.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:15.887+0000 7fd2de7fc640 1 -- 172.21.15.94:0/589506430 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd2e40c3030 con 0x7fd2e80b3120 2024-04-25T16:52:16.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.047+0000 7fd2effc3640 1 -- 172.21.15.94:0/589506430 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd2e8154390 con 0x7fd2e80b3120 2024-04-25T16:52:16.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.687+0000 7fd2de7fc640 1 -- 172.21.15.94:0/589506430 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7fd2e4016150 con 0x7fd2e80b3120 2024-04-25T16:52:16.686 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-04-25T16:52:16.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.691+0000 7fd2effc3640 1 -- 172.21.15.94:0/589506430 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd2cc06f430 msgr2=0x7fd2cc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:16.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.691+0000 7fd2effc3640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd2cc06f430 0x7fd2cc071940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8000f70 tx=0x7fd2d8009040 comp rx=0 tx=0).stop 2024-04-25T16:52:16.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.691+0000 7fd2effc3640 1 -- 172.21.15.94:0/589506430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2e80b3120 msgr2=0x7fd2e8157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:16.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.691+0000 7fd2effc3640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2e80b3120 0x7fd2e8157be0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd2e401a100 tx=0x7fd2e4019270 comp rx=0 tx=0).stop 2024-04-25T16:52:16.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.691+0000 7fd2effc3640 1 -- 172.21.15.94:0/589506430 shutdown_connections 2024-04-25T16:52:16.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.691+0000 7fd2effc3640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd2cc06f430 0x7fd2cc071940 secure :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8000f70 tx=0x7fd2d8009040 comp rx=0 tx=0).stop 2024-04-25T16:52:16.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.691+0000 7fd2effc3640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2e815de60 0x7fd2e8159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:16.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.691+0000 7fd2effc3640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2e80ce080 0x7fd2e814d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.691+0000 7fd2effc3640 1 --2- 172.21.15.94:0/589506430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2e80b3120 0x7fd2e8157be0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.691+0000 7fd2effc3640 1 -- 172.21.15.94:0/589506430 >> 172.21.15.94:0/589506430 conn(0x7fd2e80d34e0 msgr2=0x7fd2e8068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.691+0000 7fd2effc3640 1 -- 172.21.15.94:0/589506430 shutdown_connections 2024-04-25T16:52:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.691+0000 7fd2effc3640 1 -- 172.21.15.94:0/589506430 wait complete. 2024-04-25T16:52:16.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T16:52:16.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.875+0000 7faa9a917640 1 Processor -- start 2024-04-25T16:52:16.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.875+0000 7faa9a917640 1 -- start start 2024-04-25T16:52:16.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.875+0000 7faa9a917640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faa9406fcf0 0x7faa94070100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:16.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.875+0000 7faa9a917640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa940cd500 0x7faa940cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:16.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.875+0000 7faa9a917640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9406d230 con 0x7faa9406c600 2024-04-25T16:52:16.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.875+0000 7faa9a917640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7faa9406ec30 con 0x7faa9406fcf0 2024-04-25T16:52:16.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.875+0000 7faa9a917640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa9406d510 con 0x7faa940cd500 2024-04-25T16:52:16.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.875+0000 7faa937fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faa9406fcf0 0x7faa94070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:16.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.875+0000 7faa937fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faa9406fcf0 0x7faa94070100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48360/0 (socket says 172.21.15.94:48360) 2024-04-25T16:52:16.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.875+0000 7faa937fe640 1 -- 172.21.15.94:0/2301524801 learned_addr learned my addr 172.21.15.94:0/2301524801 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:16.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa937fe640 1 -- 172.21.15.94:0/2301524801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa940cd500 msgr2=0x7faa940cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:16.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa937fe640 1 --2- 172.21.15.94:0/2301524801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa940cd500 0x7faa940cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:16.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa937fe640 1 -- 172.21.15.94:0/2301524801 >> v1:172.21.15.94:6789/0 conn(0x7faa9406c600 legacy=0x7faa9406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:16.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa937fe640 1 -- 172.21.15.94:0/2301524801 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9406d7f0 con 0x7faa9406fcf0 2024-04-25T16:52:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa937fe640 1 --2- 172.21.15.94:0/2301524801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa9406fcf0 0x7faa94070100 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7faa88009870 tx=0x7faa8801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb6b0889d28fa03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa92ffd640 1 -- 172.21.15.94:0/2301524801 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa880aa430 con 0x7faa9406fcf0 2024-04-25T16:52:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa92ffd640 1 -- 172.21.15.94:0/2301524801 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa880af070 con 0x7faa9406fcf0 2024-04-25T16:52:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa92ffd640 1 -- 172.21.15.94:0/2301524801 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa880aa7f0 con 0x7faa9406fcf0 2024-04-25T16:52:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa9a917640 1 -- 172.21.15.94:0/2301524801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa9406fcf0 msgr2=0x7faa94070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa9a917640 1 --2- 172.21.15.94:0/2301524801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa9406fcf0 0x7faa94070100 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7faa88009870 tx=0x7faa8801c530 comp rx=0 tx=0).stop 2024-04-25T16:52:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa9a917640 1 -- 172.21.15.94:0/2301524801 shutdown_connections 2024-04-25T16:52:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa9a917640 1 --2- 172.21.15.94:0/2301524801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa940cd500 0x7faa940cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa9a917640 1 --2- 172.21.15.94:0/2301524801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa9406fcf0 0x7faa94070100 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa9a917640 1 -- 172.21.15.94:0/2301524801 >> 172.21.15.94:0/2301524801 conn(0x7faa940d13c0 msgr2=0x7faa940d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa9a917640 1 -- 172.21.15.94:0/2301524801 shutdown_connections 2024-04-25T16:52:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.879+0000 7faa9a917640 1 -- 172.21.15.94:0/2301524801 wait complete. 2024-04-25T16:52:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa9a917640 1 Processor -- start 2024-04-25T16:52:16.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa9a917640 1 -- start start 2024-04-25T16:52:16.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa9a917640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa9406c600 0x7faa9409ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:16.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa9a917640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa9406fcf0 0x7faa94155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:16.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa9a917640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa940cd500 0x7faa94156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:16.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa9a917640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa940ccd80 con 0x7faa940cd500 2024-04-25T16:52:16.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa9a917640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa940ccc00 con 0x7faa9406c600 2024-04-25T16:52:16.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa9a917640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa9416a920 con 0x7faa9406fcf0 2024-04-25T16:52:16.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa98eb4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa940cd500 0x7faa94156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:16.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa98eb4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa940cd500 0x7faa94156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48372/0 (socket says 172.21.15.94:48372) 2024-04-25T16:52:16.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa98eb4640 1 -- 172.21.15.94:0/2670205890 learned_addr learned my addr 172.21.15.94:0/2670205890 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:16.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa937fe640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa9406fcf0 0x7faa94155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:16.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa93fff640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa9406c600 0x7faa9409ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:16.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa98eb4640 1 -- 172.21.15.94:0/2670205890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa9406fcf0 msgr2=0x7faa94155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:16.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa98eb4640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa9406fcf0 0x7faa94155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:16.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa98eb4640 1 -- 172.21.15.94:0/2670205890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa9406c600 msgr2=0x7faa9409ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:16.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa98eb4640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa9406c600 0x7faa9409ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:16.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa98eb4640 1 -- 172.21.15.94:0/2670205890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa940727e0 con 0x7faa940cd500 2024-04-25T16:52:16.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa93fff640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa9406c600 0x7faa9409ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:16.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa937fe640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa9406fcf0 0x7faa94155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:16.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa98eb4640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa940cd500 0x7faa94156df0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00b9e0 tx=0x7faa8c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:16.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa90ff9640 1 -- 172.21.15.94:0/2670205890 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c019070 con 0x7faa940cd500 2024-04-25T16:52:16.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa90ff9640 1 -- 172.21.15.94:0/2670205890 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c00cb20 con 0x7faa940cd500 2024-04-25T16:52:16.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa9a917640 1 -- 172.21.15.94:0/2670205890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa94072ac0 con 0x7faa940cd500 2024-04-25T16:52:16.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa90ff9640 1 -- 172.21.15.94:0/2670205890 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c014440 con 0x7faa940cd500 2024-04-25T16:52:16.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.883+0000 7faa9a917640 1 -- 172.21.15.94:0/2670205890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa94073080 con 0x7faa940cd500 2024-04-25T16:52:16.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.887+0000 7faa9a917640 1 -- 172.21.15.94:0/2670205890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa940d2260 con 0x7faa940cd500 2024-04-25T16:52:16.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.887+0000 7faa90ff9640 1 -- 172.21.15.94:0/2670205890 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faa8c01c070 con 0x7faa940cd500 2024-04-25T16:52:16.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.887+0000 7faa90ff9640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa8006f400 0x7faa80071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:16.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.887+0000 7faa90ff9640 1 -- 172.21.15.94:0/2670205890 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6383+0+0 (secure 0 0 0) 0x7faa8c08f230 con 0x7faa940cd500 2024-04-25T16:52:16.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.887+0000 7faa93fff640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa8006f400 0x7faa80071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:16.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.887+0000 7faa93fff640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa8006f400 0x7faa80071910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faa84001650 tx=0x7faa84009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:16.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:16.891+0000 7faa90ff9640 1 -- 172.21.15.94:0/2670205890 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faa8c05bf10 con 0x7faa940cd500 2024-04-25T16:52:17.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.059+0000 7faa9a917640 1 -- 172.21.15.94:0/2670205890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7faa940b2a70 con 0x7faa940cd500 2024-04-25T16:52:17.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.691+0000 7faa90ff9640 1 -- 172.21.15.94:0/2670205890 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7faa8c05b810 con 0x7faa940cd500 2024-04-25T16:52:17.688 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-04-25T16:52:17.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.691+0000 7faa9a917640 1 -- 172.21.15.94:0/2670205890 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa8006f400 msgr2=0x7faa80071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:17.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.691+0000 7faa9a917640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa8006f400 0x7faa80071910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faa84001650 tx=0x7faa84009040 comp rx=0 tx=0).stop 2024-04-25T16:52:17.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.691+0000 7faa9a917640 1 -- 172.21.15.94:0/2670205890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa940cd500 msgr2=0x7faa94156df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:17.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.691+0000 7faa9a917640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa940cd500 0x7faa94156df0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00b9e0 tx=0x7faa8c00bea0 comp rx=0 tx=0).stop 2024-04-25T16:52:17.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.691+0000 7faa9a917640 1 -- 172.21.15.94:0/2670205890 shutdown_connections 2024-04-25T16:52:17.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.691+0000 7faa9a917640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa940cd500 0x7faa94156df0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:17.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.691+0000 7faa9a917640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa8006f400 0x7faa80071910 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:17.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.691+0000 7faa9a917640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa9406fcf0 0x7faa94155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:17.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.691+0000 7faa9a917640 1 --2- 172.21.15.94:0/2670205890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa9406c600 0x7faa9409ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:17.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.691+0000 7faa9a917640 1 -- 172.21.15.94:0/2670205890 >> 172.21.15.94:0/2670205890 conn(0x7faa940d13c0 msgr2=0x7faa94068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:17.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.691+0000 7faa9a917640 1 -- 172.21.15.94:0/2670205890 shutdown_connections 2024-04-25T16:52:17.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.695+0000 7faa9a917640 1 -- 172.21.15.94:0/2670205890 wait complete. 2024-04-25T16:52:17.719 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T16:52:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.867+0000 7f0a9d47a640 1 Processor -- start 2024-04-25T16:52:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.867+0000 7f0a9d47a640 1 -- start start 2024-04-25T16:52:17.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.871+0000 7f0a9d47a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0a980cf700 0x7f0a980b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.871+0000 7f0a9d47a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a980702f0 0x7f0a98070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.871+0000 7f0a9d47a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a980b6510 con 0x7f0a980d2050 2024-04-25T16:52:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.871+0000 7f0a9d47a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f0a980b62c0 con 0x7f0a980cf700 2024-04-25T16:52:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.871+0000 7f0a9d47a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0a980b67f0 con 0x7f0a980702f0 2024-04-25T16:52:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.871+0000 7f0a96ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0a980cf700 0x7f0a980b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.871+0000 7f0a96ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0a980cf700 0x7f0a980b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48378/0 (socket says 172.21.15.94:48378) 2024-04-25T16:52:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.871+0000 7f0a96ffd640 1 -- 172.21.15.94:0/1932122447 learned_addr learned my addr 172.21.15.94:0/1932122447 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.871+0000 7f0a977fe640 1 --2- 172.21.15.94:0/1932122447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a980702f0 0x7f0a98070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.871+0000 7f0a96ffd640 1 -- 172.21.15.94:0/1932122447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a980702f0 msgr2=0x7f0a98070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.871+0000 7f0a96ffd640 1 --2- 172.21.15.94:0/1932122447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a980702f0 0x7f0a98070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.871+0000 7f0a96ffd640 1 -- 172.21.15.94:0/1932122447 >> v1:172.21.15.138:6789/0 conn(0x7f0a980d2050 legacy=0x7f0a980d2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:17.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.871+0000 7f0a96ffd640 1 -- 172.21.15.94:0/1932122447 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a980b6ad0 con 0x7f0a980cf700 2024-04-25T16:52:17.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a96ffd640 1 --2- 172.21.15.94:0/1932122447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a980cf700 0x7f0a980b72d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0a98074150 tx=0x7f0a8801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc8b496fc704675 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:17.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a95ffb640 1 -- 172.21.15.94:0/1932122447 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a880b0070 con 0x7f0a980cf700 2024-04-25T16:52:17.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a95ffb640 1 -- 172.21.15.94:0/1932122447 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a880ab600 con 0x7f0a980cf700 2024-04-25T16:52:17.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a95ffb640 1 -- 172.21.15.94:0/1932122447 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a880ab9d0 con 0x7f0a980cf700 2024-04-25T16:52:17.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1932122447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a980cf700 msgr2=0x7f0a980b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:17.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a9d47a640 1 --2- 172.21.15.94:0/1932122447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a980cf700 0x7f0a980b72d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0a98074150 tx=0x7f0a8801c500 comp rx=0 tx=0).stop 2024-04-25T16:52:17.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1932122447 shutdown_connections 2024-04-25T16:52:17.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a9d47a640 1 --2- 172.21.15.94:0/1932122447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a980702f0 0x7f0a98070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:17.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a9d47a640 1 --2- 172.21.15.94:0/1932122447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a980cf700 0x7f0a980b72d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:17.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1932122447 >> 172.21.15.94:0/1932122447 conn(0x7f0a980cfef0 msgr2=0x7f0a980d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:17.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1932122447 shutdown_connections 2024-04-25T16:52:17.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1932122447 wait complete. 2024-04-25T16:52:17.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a9d47a640 1 Processor -- start 2024-04-25T16:52:17.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.875+0000 7f0a9d47a640 1 -- start start 2024-04-25T16:52:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a9d47a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a980702f0 0x7f0a980c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a9d47a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a980cf700 0x7f0a9814d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a9d47a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a980d2050 0x7f0a981589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a9d47a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9815deb0 con 0x7f0a980702f0 2024-04-25T16:52:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a9d47a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9815dd30 con 0x7f0a980cf700 2024-04-25T16:52:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a9d47a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9816ae70 con 0x7f0a980d2050 2024-04-25T16:52:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a96ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a980702f0 0x7f0a980c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a96ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a980702f0 0x7f0a980c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48384/0 (socket says 172.21.15.94:48384) 2024-04-25T16:52:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a96ffd640 1 -- 172.21.15.94:0/1223616285 learned_addr learned my addr 172.21.15.94:0/1223616285 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a967fc640 1 --2- 172.21.15.94:0/1223616285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a980cf700 0x7f0a9814d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a96ffd640 1 -- 172.21.15.94:0/1223616285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a980d2050 msgr2=0x7f0a981589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a96ffd640 1 --2- 172.21.15.94:0/1223616285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a980d2050 0x7f0a981589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a96ffd640 1 -- 172.21.15.94:0/1223616285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a980cf700 msgr2=0x7f0a9814d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a96ffd640 1 --2- 172.21.15.94:0/1223616285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a980cf700 0x7f0a9814d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a96ffd640 1 -- 172.21.15.94:0/1223616285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a980c8dc0 con 0x7f0a980702f0 2024-04-25T16:52:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a96ffd640 1 --2- 172.21.15.94:0/1223616285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a980702f0 0x7f0a980c9830 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0a8801cda0 tx=0x7f0a880192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a7f7fe640 1 -- 172.21.15.94:0/1223616285 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a880ba070 con 0x7f0a980702f0 2024-04-25T16:52:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a7f7fe640 1 -- 172.21.15.94:0/1223616285 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a880b4620 con 0x7f0a980702f0 2024-04-25T16:52:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1223616285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a980c7b30 con 0x7f0a980702f0 2024-04-25T16:52:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a7f7fe640 1 -- 172.21.15.94:0/1223616285 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a880b0040 con 0x7f0a980702f0 2024-04-25T16:52:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1223616285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a980c9d70 con 0x7f0a980702f0 2024-04-25T16:52:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1223616285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a98158400 con 0x7f0a980702f0 2024-04-25T16:52:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a7f7fe640 1 -- 172.21.15.94:0/1223616285 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0a880bd070 con 0x7f0a980702f0 2024-04-25T16:52:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a7f7fe640 1 --2- 172.21.15.94:0/1223616285 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a8406f430 0x7f0a84071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a967fc640 1 --2- 172.21.15.94:0/1223616285 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a8406f430 0x7f0a84071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a7f7fe640 1 -- 172.21.15.94:0/1223616285 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7f0a88128500 con 0x7f0a980702f0 2024-04-25T16:52:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.879+0000 7f0a967fc640 1 --2- 172.21.15.94:0/1223616285 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a8406f430 0x7f0a84071940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a981586b0 tx=0x7f0a8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:17.883+0000 7f0a7f7fe640 1 -- 172.21.15.94:0/1223616285 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0a880160f0 con 0x7f0a980702f0 2024-04-25T16:52:18.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.051+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1223616285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0a9814e600 con 0x7f0a980702f0 2024-04-25T16:52:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.703+0000 7f0a7f7fe640 1 -- 172.21.15.94:0/1223616285 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7f0a880f5360 con 0x7f0a980702f0 2024-04-25T16:52:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data_ec' removed 2024-04-25T16:52:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.707+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1223616285 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a8406f430 msgr2=0x7f0a84071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.707+0000 7f0a9d47a640 1 --2- 172.21.15.94:0/1223616285 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a8406f430 0x7f0a84071940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a981586b0 tx=0x7f0a8c008040 comp rx=0 tx=0).stop 2024-04-25T16:52:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.707+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1223616285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a980702f0 msgr2=0x7f0a980c9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.707+0000 7f0a9d47a640 1 --2- 172.21.15.94:0/1223616285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a980702f0 0x7f0a980c9830 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0a8801cda0 tx=0x7f0a880192a0 comp rx=0 tx=0).stop 2024-04-25T16:52:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.707+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1223616285 shutdown_connections 2024-04-25T16:52:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.707+0000 7f0a9d47a640 1 --2- 172.21.15.94:0/1223616285 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a8406f430 0x7f0a84071940 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a981586b0 tx=0x7f0a8c008040 comp rx=0 tx=0).stop 2024-04-25T16:52:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.707+0000 7f0a9d47a640 1 --2- 172.21.15.94:0/1223616285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a980d2050 0x7f0a981589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.707+0000 7f0a9d47a640 1 --2- 172.21.15.94:0/1223616285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a980cf700 0x7f0a9814d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.707+0000 7f0a9d47a640 1 --2- 172.21.15.94:0/1223616285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a980702f0 0x7f0a980c9830 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.707+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1223616285 >> 172.21.15.94:0/1223616285 conn(0x7f0a980cfef0 msgr2=0x7f0a98153790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.707+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1223616285 shutdown_connections 2024-04-25T16:52:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.707+0000 7f0a9d47a640 1 -- 172.21.15.94:0/1223616285 wait complete. 2024-04-25T16:52:18.732 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T16:52:18.732 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T16:52:18.733 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T16:52:18.733 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:52:18.733 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T16:52:18.733 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T16:52:18.733 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T16:52:18.733 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:52:18.733 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T16:52:18.734 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T16:52:18.734 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T16:52:18.734 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:52:18.734 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T16:52:18.734 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T16:52:18.734 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T16:52:18.735 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:52:18.747 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T16:52:18.748 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T16:52:18.749 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T16:52:18.749 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T16:52:18.755 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T16:52:18.755 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T16:52:18.757 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T16:52:18.757 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T16:52:18.796 INFO:tasks.ceph.mds.a:Started 2024-04-25T16:52:18.797 DEBUG:teuthology.parallel:result is None 2024-04-25T16:52:18.797 INFO:tasks.ceph.mds.c:Started 2024-04-25T16:52:18.798 DEBUG:teuthology.parallel:result is None 2024-04-25T16:52:18.798 INFO:tasks.ceph.mds.d:Started 2024-04-25T16:52:18.799 DEBUG:teuthology.parallel:result is None 2024-04-25T16:52:18.800 INFO:tasks.ceph.mds.b:Started 2024-04-25T16:52:18.800 DEBUG:teuthology.parallel:result is None 2024-04-25T16:52:18.800 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T16:52:18.889 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-04-25T16:52:18.892 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-04-25T16:52:18.899 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-04-25T16:52:18.901 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-04-25T16:52:18.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.943+0000 7f88d8724640 1 Processor -- start 2024-04-25T16:52:18.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.943+0000 7f88d8724640 1 -- start start 2024-04-25T16:52:18.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d8724640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f88d00d3450 0x7f88d00d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:18.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d8724640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88d00d2440 0x7f88d00b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:18.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d8724640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88d0164510 con 0x7f88d00d3da0 2024-04-25T16:52:18.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d8724640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f88d0164a90 con 0x7f88d00d3450 2024-04-25T16:52:18.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d8724640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f88d00d51b0 con 0x7f88d00d2440 2024-04-25T16:52:18.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d6cc1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88d00d2440 0x7f88d00b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:18.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d64c0640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f88d00d3450 0x7f88d00d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:18.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d5cbf640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f88d00d3da0 0x7f88d00b6fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:44602/0 (socket says 172.21.15.94:44602) 2024-04-25T16:52:18.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d5cbf640 1 -- 172.21.15.94:0/894457657 learned_addr learned my addr 172.21.15.94:0/894457657 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:18.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d6cc1640 1 -- 172.21.15.94:0/894457657 >> v2:172.21.15.138:3300/0 conn(0x7f88d00d3450 msgr2=0x7f88d00d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:18.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d6cc1640 1 --2- 172.21.15.94:0/894457657 >> v2:172.21.15.138:3300/0 conn(0x7f88d00d3450 0x7f88d00d3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:18.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d6cc1640 1 -- 172.21.15.94:0/894457657 >> v1:172.21.15.138:6789/0 conn(0x7f88d00d3da0 legacy=0x7f88d00b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:18.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d6cc1640 1 -- 172.21.15.94:0/894457657 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d01647b0 con 0x7f88d00d2440 2024-04-25T16:52:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d6cc1640 1 --2- 172.21.15.94:0/894457657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88d00d2440 0x7f88d00b76e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009d90 tx=0x7f88c801c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3fdfe05806c0a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d54be640 1 -- 172.21.15.94:0/894457657 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c80b2070 con 0x7f88d00d2440 2024-04-25T16:52:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d54be640 1 -- 172.21.15.94:0/894457657 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c80a5970 con 0x7f88d00d2440 2024-04-25T16:52:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.947+0000 7f88d54be640 1 -- 172.21.15.94:0/894457657 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c80a5d40 con 0x7f88d00d2440 2024-04-25T16:52:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 -- 172.21.15.94:0/894457657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88d00d2440 msgr2=0x7f88d00b76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 --2- 172.21.15.94:0/894457657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88d00d2440 0x7f88d00b76e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009d90 tx=0x7f88c801c8b0 comp rx=0 tx=0).stop 2024-04-25T16:52:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 -- 172.21.15.94:0/894457657 shutdown_connections 2024-04-25T16:52:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 --2- 172.21.15.94:0/894457657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88d00d2440 0x7f88d00b76e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 --2- 172.21.15.94:0/894457657 >> v2:172.21.15.138:3300/0 conn(0x7f88d00d3450 0x7f88d00d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 -- 172.21.15.94:0/894457657 >> 172.21.15.94:0/894457657 conn(0x7f88d009e8d0 msgr2=0x7f88d00b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 -- 172.21.15.94:0/894457657 shutdown_connections 2024-04-25T16:52:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 -- 172.21.15.94:0/894457657 wait complete. 2024-04-25T16:52:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 Processor -- start 2024-04-25T16:52:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 -- start start 2024-04-25T16:52:18.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d00d2440 0x7f88d0155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:18.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88d00d3450 0x7f88d015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:18.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d00d3da0 0x7f88d00c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88d00b8cd0 con 0x7f88d00d2440 2024-04-25T16:52:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88d00d41b0 con 0x7f88d00d3da0 2024-04-25T16:52:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d8724640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f88d00b8e50 con 0x7f88d00d3450 2024-04-25T16:52:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d64c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d00d2440 0x7f88d0155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d64c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d00d2440 0x7f88d0155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37570/0 (socket says 172.21.15.94:37570) 2024-04-25T16:52:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d64c0640 1 -- 172.21.15.94:0/1708625994 learned_addr learned my addr 172.21.15.94:0/1708625994 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d5cbf640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88d00d3450 0x7f88d015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d6cc1640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d00d3da0 0x7f88d00c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d64c0640 1 -- 172.21.15.94:0/1708625994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88d00d3450 msgr2=0x7f88d015dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d64c0640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88d00d3450 0x7f88d015dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d64c0640 1 -- 172.21.15.94:0/1708625994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d00d3da0 msgr2=0x7f88d00c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d64c0640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d00d3da0 0x7f88d00c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d64c0640 1 -- 172.21.15.94:0/1708625994 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d015e8a0 con 0x7f88d00d2440 2024-04-25T16:52:18.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.951+0000 7f88d5cbf640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88d00d3450 0x7f88d015dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:18.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.955+0000 7f88d64c0640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d00d2440 0x7f88d0155e10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f88c000e840 tx=0x7f88c000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:18.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.955+0000 7f88c6ffd640 1 -- 172.21.15.94:0/1708625994 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0017070 con 0x7f88d00d2440 2024-04-25T16:52:18.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.955+0000 7f88c6ffd640 1 -- 172.21.15.94:0/1708625994 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c000ee90 con 0x7f88d00d2440 2024-04-25T16:52:18.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.955+0000 7f88c6ffd640 1 -- 172.21.15.94:0/1708625994 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c000cb60 con 0x7f88d00d2440 2024-04-25T16:52:18.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.955+0000 7f88d8724640 1 -- 172.21.15.94:0/1708625994 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d015eb80 con 0x7f88d00d2440 2024-04-25T16:52:18.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.955+0000 7f88d8724640 1 -- 172.21.15.94:0/1708625994 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d015f140 con 0x7f88d00d2440 2024-04-25T16:52:18.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.955+0000 7f88d8724640 1 -- 172.21.15.94:0/1708625994 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d0076830 con 0x7f88d00d2440 2024-04-25T16:52:18.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.955+0000 7f88c6ffd640 1 -- 172.21.15.94:0/1708625994 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f88c001a070 con 0x7f88d00d2440 2024-04-25T16:52:18.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.955+0000 7f88c6ffd640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f88b406f400 0x7f88b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:18.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.955+0000 7f88c6ffd640 1 -- 172.21.15.94:0/1708625994 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7f88c008f1e0 con 0x7f88d00d2440 2024-04-25T16:52:18.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.955+0000 7f88d5cbf640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f88b406f400 0x7f88b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:18.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.955+0000 7f88d5cbf640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f88b406f400 0x7f88b4071910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f88d0162d00 tx=0x7f88cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:18.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:18.959+0000 7f88c6ffd640 1 -- 172.21.15.94:0/1708625994 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f88c005bd10 con 0x7f88d00d2440 2024-04-25T16:52:19.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.131+0000 7f88d8724640 1 -- 172.21.15.94:0/1708625994 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f88d00d2c40 con 0x7f88d00d2440 2024-04-25T16:52:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.723+0000 7f88c6ffd640 1 -- 172.21.15.94:0/1708625994 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7f88c0060010 con 0x7f88d00d2440 2024-04-25T16:52:19.722 INFO:teuthology.orchestra.run.smithi094.stderr: removed all blocklist entries 2024-04-25T16:52:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.727+0000 7f88d8724640 1 -- 172.21.15.94:0/1708625994 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f88b406f400 msgr2=0x7f88b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.727+0000 7f88d8724640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f88b406f400 0x7f88b4071910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f88d0162d00 tx=0x7f88cc009040 comp rx=0 tx=0).stop 2024-04-25T16:52:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.727+0000 7f88d8724640 1 -- 172.21.15.94:0/1708625994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d00d2440 msgr2=0x7f88d0155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.727+0000 7f88d8724640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d00d2440 0x7f88d0155e10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f88c000e840 tx=0x7f88c000ec40 comp rx=0 tx=0).stop 2024-04-25T16:52:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.727+0000 7f88d8724640 1 -- 172.21.15.94:0/1708625994 shutdown_connections 2024-04-25T16:52:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.727+0000 7f88d8724640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d00d3da0 0x7f88d00c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.727+0000 7f88d8724640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f88b406f400 0x7f88b4071910 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.727+0000 7f88d8724640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88d00d3450 0x7f88d015dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.727+0000 7f88d8724640 1 --2- 172.21.15.94:0/1708625994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d00d2440 0x7f88d0155e10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.727+0000 7f88d8724640 1 -- 172.21.15.94:0/1708625994 >> 172.21.15.94:0/1708625994 conn(0x7f88d009e8d0 msgr2=0x7f88d00b5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.727+0000 7f88d8724640 1 -- 172.21.15.94:0/1708625994 shutdown_connections 2024-04-25T16:52:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.727+0000 7f88d8724640 1 -- 172.21.15.94:0/1708625994 wait complete. 2024-04-25T16:52:19.757 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T16:52:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.903+0000 7fcd9ae79640 1 Processor -- start 2024-04-25T16:52:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.903+0000 7fcd9ae79640 1 -- start start 2024-04-25T16:52:19.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.907+0000 7fcd9ae79640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcd94070180 0x7fcd9415cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:19.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.907+0000 7fcd9ae79640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcd9406de20 0x7fcd940af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:19.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.907+0000 7fcd9ae79640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd9415d620 con 0x7fcd94160bf0 2024-04-25T16:52:19.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.907+0000 7fcd9ae79640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9415dec0 con 0x7fcd94070180 2024-04-25T16:52:19.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.907+0000 7fcd9ae79640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcd94160760 con 0x7fcd9406de20 2024-04-25T16:52:19.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.907+0000 7fcd98c15640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcd94070180 0x7fcd9415cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:19.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.907+0000 7fcd98c15640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcd94070180 0x7fcd9415cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37584/0 (socket says 172.21.15.94:37584) 2024-04-25T16:52:19.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.907+0000 7fcd98c15640 1 -- 172.21.15.94:0/4189914976 learned_addr learned my addr 172.21.15.94:0/4189914976 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:19.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.907+0000 7fcd99416640 1 --2- 172.21.15.94:0/4189914976 >> v2:172.21.15.138:3300/0 conn(0x7fcd9406de20 0x7fcd940af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:19.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.907+0000 7fcd98c15640 1 -- 172.21.15.94:0/4189914976 >> v2:172.21.15.138:3300/0 conn(0x7fcd9406de20 msgr2=0x7fcd940af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:19.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.907+0000 7fcd98c15640 1 --2- 172.21.15.94:0/4189914976 >> v2:172.21.15.138:3300/0 conn(0x7fcd9406de20 0x7fcd940af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:19.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.907+0000 7fcd98c15640 1 -- 172.21.15.94:0/4189914976 >> v1:172.21.15.138:6789/0 conn(0x7fcd94160bf0 legacy=0x7fcd9406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:19.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.907+0000 7fcd98c15640 1 -- 172.21.15.94:0/4189914976 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9415d900 con 0x7fcd94070180 2024-04-25T16:52:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd98c15640 1 --2- 172.21.15.94:0/4189914976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd94070180 0x7fcd9415cf40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcd940b0d10 tx=0x7fcd8401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c4531f014bab99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd937fe640 1 -- 172.21.15.94:0/4189914976 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd840b1070 con 0x7fcd94070180 2024-04-25T16:52:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd937fe640 1 -- 172.21.15.94:0/4189914976 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd840ac620 con 0x7fcd94070180 2024-04-25T16:52:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd937fe640 1 -- 172.21.15.94:0/4189914976 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd840ac9f0 con 0x7fcd94070180 2024-04-25T16:52:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd9ae79640 1 -- 172.21.15.94:0/4189914976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd94070180 msgr2=0x7fcd9415cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd9ae79640 1 --2- 172.21.15.94:0/4189914976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd94070180 0x7fcd9415cf40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcd940b0d10 tx=0x7fcd8401c4b0 comp rx=0 tx=0).stop 2024-04-25T16:52:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd9ae79640 1 -- 172.21.15.94:0/4189914976 shutdown_connections 2024-04-25T16:52:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd9ae79640 1 --2- 172.21.15.94:0/4189914976 >> v2:172.21.15.138:3300/0 conn(0x7fcd9406de20 0x7fcd940af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd9ae79640 1 --2- 172.21.15.94:0/4189914976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd94070180 0x7fcd9415cf40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd9ae79640 1 -- 172.21.15.94:0/4189914976 >> 172.21.15.94:0/4189914976 conn(0x7fcd940b47f0 msgr2=0x7fcd940b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd9ae79640 1 -- 172.21.15.94:0/4189914976 shutdown_connections 2024-04-25T16:52:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd9ae79640 1 -- 172.21.15.94:0/4189914976 wait complete. 2024-04-25T16:52:19.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd9ae79640 1 Processor -- start 2024-04-25T16:52:19.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.911+0000 7fcd9ae79640 1 -- start start 2024-04-25T16:52:19.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd9ae79640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd94070180 0x7fcd9414c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:19.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd9ae79640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94160bf0 0x7fcd9406c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:19.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd9ae79640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd941f9990 0x7fcd9406cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:19.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd9ae79640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9416a260 con 0x7fcd941f9990 2024-04-25T16:52:19.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd9ae79640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9416a0e0 con 0x7fcd94160bf0 2024-04-25T16:52:19.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd9ae79640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9416a3e0 con 0x7fcd94070180 2024-04-25T16:52:19.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd99416640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd941f9990 0x7fcd9406cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:19.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd99416640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd941f9990 0x7fcd9406cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37594/0 (socket says 172.21.15.94:37594) 2024-04-25T16:52:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd99416640 1 -- 172.21.15.94:0/2884094264 learned_addr learned my addr 172.21.15.94:0/2884094264 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd99416640 1 -- 172.21.15.94:0/2884094264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd94070180 msgr2=0x7fcd9414c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd99416640 1 --2- 172.21.15.94:0/2884094264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd94070180 0x7fcd9414c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd99416640 1 -- 172.21.15.94:0/2884094264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94160bf0 msgr2=0x7fcd9406c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd99416640 1 --2- 172.21.15.94:0/2884094264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94160bf0 0x7fcd9406c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd99416640 1 -- 172.21.15.94:0/2884094264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9414d2b0 con 0x7fcd941f9990 2024-04-25T16:52:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd99416640 1 --2- 172.21.15.94:0/2884094264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd941f9990 0x7fcd9406cb40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c007e60 tx=0x7fcd8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd917fa640 1 -- 172.21.15.94:0/2884094264 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c014420 con 0x7fcd941f9990 2024-04-25T16:52:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd917fa640 1 -- 172.21.15.94:0/2884094264 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8c019070 con 0x7fcd941f9990 2024-04-25T16:52:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd9ae79640 1 -- 172.21.15.94:0/2884094264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd94165b10 con 0x7fcd941f9990 2024-04-25T16:52:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd917fa640 1 -- 172.21.15.94:0/2884094264 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c014820 con 0x7fcd941f9990 2024-04-25T16:52:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd9ae79640 1 -- 172.21.15.94:0/2884094264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd941660d0 con 0x7fcd941f9990 2024-04-25T16:52:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.915+0000 7fcd9ae79640 1 -- 172.21.15.94:0/2884094264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9414d020 con 0x7fcd941f9990 2024-04-25T16:52:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.919+0000 7fcd917fa640 1 -- 172.21.15.94:0/2884094264 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcd8c0149c0 con 0x7fcd941f9990 2024-04-25T16:52:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.919+0000 7fcd917fa640 1 --2- 172.21.15.94:0/2884094264 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcd8006f430 0x7fcd80071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.919+0000 7fcd917fa640 1 -- 172.21.15.94:0/2884094264 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fcd8c08f910 con 0x7fcd941f9990 2024-04-25T16:52:19.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.927+0000 7fcd98c15640 1 --2- 172.21.15.94:0/2884094264 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcd8006f430 0x7fcd80071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:19.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.927+0000 7fcd98c15640 1 --2- 172.21.15.94:0/2884094264 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcd8006f430 0x7fcd80071940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd840ae000 tx=0x7fcd84004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:19.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:19.927+0000 7fcd917fa640 1 -- 172.21.15.94:0/2884094264 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcd8c05c9c0 con 0x7fcd941f9990 2024-04-25T16:52:20.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.147+0000 7fcd9ae79640 1 -- 172.21.15.94:0/2884094264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcd94076830 con 0x7fcd941f9990 2024-04-25T16:52:20.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.147+0000 7fcd917fa640 1 -- 172.21.15.94:0/2884094264 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7fcd8c05c2c0 con 0x7fcd941f9990 2024-04-25T16:52:20.146 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:20.146 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_dump": [ 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.a", 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCqiSpmgKayFBAAU7YU3CXQPeY2aOZRmZpT/w==", 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.b", 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCqiSpmRpnTIRAAiPFPCHSxVWe9q0F0deXgaA==", 2024-04-25T16:52:20.147 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.148 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-04-25T16:52:20.148 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-04-25T16:52:20.148 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-04-25T16:52:20.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.148 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.148 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.c", 2024-04-25T16:52:20.148 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCqiSpmC24sHBAAdDAXtY+87Lt/TawZKwTdcA==", 2024-04-25T16:52:20.148 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.148 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-04-25T16:52:20.148 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.d", 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCqiSpmyy+aKRAARdfTN3rKu+E6bDvZZbD47g==", 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.0", 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCsiSpmgSfGBhAAD6ugAvKS2vtU9ufyA8T2UQ==", 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.1", 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCviSpmEXKZDRAA2/cTL5Irr+vAXiKKpYzxDg==", 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.150 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.2", 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCyiSpm3IxSExAAzs5LaP8CwRLgPVAps2ccvQ==", 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.151 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.3", 2024-04-25T16:52:20.152 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC1iSpmukcIGhAAFmLhwf4KHPJJvNYzSop6jA==", 2024-04-25T16:52:20.152 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.152 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-04-25T16:52:20.152 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-04-25T16:52:20.152 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.152 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.152 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.4", 2024-04-25T16:52:20.152 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC7iSpm0RWhGxAA4KKPB45Iod7zoVw6IW4bJQ==", 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.5", 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC+iSpmetnNIxAAF+1t+rPK23w/VvsRnJgyyw==", 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.6", 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDBiSpmyEXoKhAAx5EDhq1pz+2EeD/dWUParg==", 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.7", 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDEiSpmDeSDMhAArpZmLFT/VyYTzp/gq6v87w==", 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-04-25T16:52:20.154 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-04-25T16:52:20.155 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.155 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.155 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.0", 2024-04-25T16:52:20.155 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCqiSpmzcOULhAAGKJvQYLzCNAQecQi22z2GQ==", 2024-04-25T16:52:20.155 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.155 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-04-25T16:52:20.155 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-04-25T16:52:20.155 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-04-25T16:52:20.155 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-04-25T16:52:20.155 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.1", 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCqiSpmjfIfOBAAs0gA62roRxxeirvZV36x+g==", 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.2", 2024-04-25T16:52:20.156 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCqiSpmm3c+MxAAJUovvS+EqAog+i2eUBeh9Q==", 2024-04-25T16:52:20.157 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.157 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-04-25T16:52:20.157 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-04-25T16:52:20.157 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-04-25T16:52:20.157 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-04-25T16:52:20.157 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.157 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.3", 2024-04-25T16:52:20.157 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCriSpmefDuABAA7Uo70NTIVW+wZYj15VbmEg==", 2024-04-25T16:52:20.157 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.admin", 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCpiSpmAupiMxAAurdin/1JEoAui1v0rpMSPA==", 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-04-25T16:52:20.158 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-mds", 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDUiSpmWWWlOBAApHp+pbGdfVBtC6Ucpr+r+A==", 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDUiSpmMIGlOBAA5yUmeMGnZSaxGa+ULm0SWg==", 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.159 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-osd", 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDUiSpmqp6lOBAAYO7FWiciTY+uK5GBSM1bEA==", 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDUiSpm2LalOBAAQnBwAxbbZ4h60vPmLUtXEQ==", 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.160 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDUiSpmnM+lOBAAXUwQAzRRb/3BXtYqpcUqPg==", 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDUiSpmP+ilOBAA/gDL1PrU1pkTa0AwgpOY1Q==", 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T16:52:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.x", 2024-04-25T16:52:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCqiSpmo1esDxAAxzoc9ioXsQxGPWDgEh4Jgg==", 2024-04-25T16:52:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-04-25T16:52:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-04-25T16:52:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.y", 2024-04-25T16:52:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQCqiSpmkuy3ChAAK079xLd0VAzJ2fHWGSqAKg==", 2024-04-25T16:52:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-04-25T16:52:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-04-25T16:52:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-04-25T16:52:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-04-25T16:52:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T16:52:20.163 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:52:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.151+0000 7fcd9ae79640 1 -- 172.21.15.94:0/2884094264 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcd8006f430 msgr2=0x7fcd80071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.151+0000 7fcd9ae79640 1 --2- 172.21.15.94:0/2884094264 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcd8006f430 0x7fcd80071940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd840ae000 tx=0x7fcd84004740 comp rx=0 tx=0).stop 2024-04-25T16:52:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.151+0000 7fcd9ae79640 1 -- 172.21.15.94:0/2884094264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd941f9990 msgr2=0x7fcd9406cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.151+0000 7fcd9ae79640 1 --2- 172.21.15.94:0/2884094264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd941f9990 0x7fcd9406cb40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c007e60 tx=0x7fcd8c00e4b0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.151+0000 7fcd9ae79640 1 -- 172.21.15.94:0/2884094264 shutdown_connections 2024-04-25T16:52:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.151+0000 7fcd9ae79640 1 --2- 172.21.15.94:0/2884094264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd941f9990 0x7fcd9406cb40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.151+0000 7fcd9ae79640 1 --2- 172.21.15.94:0/2884094264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94160bf0 0x7fcd9406c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.151+0000 7fcd9ae79640 1 --2- 172.21.15.94:0/2884094264 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcd8006f430 0x7fcd80071940 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.151+0000 7fcd9ae79640 1 --2- 172.21.15.94:0/2884094264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd94070180 0x7fcd9414c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.151+0000 7fcd9ae79640 1 -- 172.21.15.94:0/2884094264 >> 172.21.15.94:0/2884094264 conn(0x7fcd940b47f0 msgr2=0x7fcd940b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.151+0000 7fcd9ae79640 1 -- 172.21.15.94:0/2884094264 shutdown_connections 2024-04-25T16:52:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.151+0000 7fcd9ae79640 1 -- 172.21.15.94:0/2884094264 wait complete. 2024-04-25T16:52:20.167 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T16:52:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.307+0000 7fc2447d3640 1 Processor -- start 2024-04-25T16:52:20.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.307+0000 7fc2447d3640 1 -- start start 2024-04-25T16:52:20.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc2447d3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc23c0b0420 0x7fc23c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc2447d3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc23c15ec30 0x7fc23c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc2447d3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc23c070630 con 0x7fc23c06f2a0 2024-04-25T16:52:20.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc2447d3640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fc23c0aeaa0 con 0x7fc23c15ec30 2024-04-25T16:52:20.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc2447d3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc23c0703e0 con 0x7fc23c0b0420 2024-04-25T16:52:20.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc242d70640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc23c15ec30 0x7fc23c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:20.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc241d6e640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc23c06f2a0 0x7fc23c06c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45820/0 (socket says 172.21.15.94:45820) 2024-04-25T16:52:20.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc241d6e640 1 -- 172.21.15.94:0/58420556 learned_addr learned my addr 172.21.15.94:0/58420556 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:20.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc24256f640 1 --2- 172.21.15.94:0/58420556 >> v2:172.21.15.138:3300/0 conn(0x7fc23c0b0420 0x7fc23c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:20.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc242d70640 1 -- 172.21.15.94:0/58420556 >> v2:172.21.15.138:3300/0 conn(0x7fc23c0b0420 msgr2=0x7fc23c06ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:20.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc242d70640 1 --2- 172.21.15.94:0/58420556 >> v2:172.21.15.138:3300/0 conn(0x7fc23c0b0420 0x7fc23c06ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc242d70640 1 -- 172.21.15.94:0/58420556 >> v1:172.21.15.94:6789/0 conn(0x7fc23c06f2a0 legacy=0x7fc23c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc242d70640 1 -- 172.21.15.94:0/58420556 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c06f6b0 con 0x7fc23c15ec30 2024-04-25T16:52:20.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.311+0000 7fc24156d640 1 -- 172.21.15.94:0/58420556 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4121502544 0 0) 0x7fc23c070630 con 0x7fc23c06f2a0 2024-04-25T16:52:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc242d70640 1 --2- 172.21.15.94:0/58420556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc23c15ec30 0x7fc23c06cd10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc23c071670 tx=0x7fc23401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd376bdc8e5ebff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:20.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc24156d640 1 -- 172.21.15.94:0/58420556 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2340ab650 con 0x7fc23c15ec30 2024-04-25T16:52:20.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc24156d640 1 -- 172.21.15.94:0/58420556 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2340b0070 con 0x7fc23c15ec30 2024-04-25T16:52:20.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc24156d640 1 -- 172.21.15.94:0/58420556 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2340aba10 con 0x7fc23c15ec30 2024-04-25T16:52:20.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc2447d3640 1 -- 172.21.15.94:0/58420556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc23c15ec30 msgr2=0x7fc23c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc2447d3640 1 --2- 172.21.15.94:0/58420556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc23c15ec30 0x7fc23c06cd10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc23c071670 tx=0x7fc23401c450 comp rx=0 tx=0).stop 2024-04-25T16:52:20.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc2447d3640 1 -- 172.21.15.94:0/58420556 shutdown_connections 2024-04-25T16:52:20.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc2447d3640 1 --2- 172.21.15.94:0/58420556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc23c15ec30 0x7fc23c06cd10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc2447d3640 1 --2- 172.21.15.94:0/58420556 >> v2:172.21.15.138:3300/0 conn(0x7fc23c0b0420 0x7fc23c06ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc2447d3640 1 -- 172.21.15.94:0/58420556 >> 172.21.15.94:0/58420556 conn(0x7fc23c0b47f0 msgr2=0x7fc23c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:20.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc2447d3640 1 -- 172.21.15.94:0/58420556 shutdown_connections 2024-04-25T16:52:20.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc2447d3640 1 -- 172.21.15.94:0/58420556 wait complete. 2024-04-25T16:52:20.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc2447d3640 1 Processor -- start 2024-04-25T16:52:20.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.315+0000 7fc2447d3640 1 -- start start 2024-04-25T16:52:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc2447d3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc23c06f2a0 0x7fc23c156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc2447d3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc23c0b0420 0x7fc23c099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc2447d3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc23c15ec30 0x7fc23c1464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc2447d3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c0b0830 con 0x7fc23c15ec30 2024-04-25T16:52:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc2447d3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c0aef30 con 0x7fc23c06f2a0 2024-04-25T16:52:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc2447d3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc23c0b09b0 con 0x7fc23c0b0420 2024-04-25T16:52:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc242d70640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc23c15ec30 0x7fc23c1464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc242d70640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc23c15ec30 0x7fc23c1464a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37606/0 (socket says 172.21.15.94:37606) 2024-04-25T16:52:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc242d70640 1 -- 172.21.15.94:0/3136869043 learned_addr learned my addr 172.21.15.94:0/3136869043 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc24256f640 1 --2- 172.21.15.94:0/3136869043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc23c06f2a0 0x7fc23c156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc242d70640 1 -- 172.21.15.94:0/3136869043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc23c0b0420 msgr2=0x7fc23c099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc242d70640 1 --2- 172.21.15.94:0/3136869043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc23c0b0420 0x7fc23c099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc242d70640 1 -- 172.21.15.94:0/3136869043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc23c06f2a0 msgr2=0x7fc23c156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc242d70640 1 --2- 172.21.15.94:0/3136869043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc23c06f2a0 0x7fc23c156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc242d70640 1 -- 172.21.15.94:0/3136869043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c09a240 con 0x7fc23c15ec30 2024-04-25T16:52:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc242d70640 1 --2- 172.21.15.94:0/3136869043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc23c15ec30 0x7fc23c1464a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc23401a080 tx=0x7fc23401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc232ffd640 1 -- 172.21.15.94:0/3136869043 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234006e20 con 0x7fc23c15ec30 2024-04-25T16:52:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc232ffd640 1 -- 172.21.15.94:0/3136869043 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc234004230 con 0x7fc23c15ec30 2024-04-25T16:52:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc2447d3640 1 -- 172.21.15.94:0/3136869043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c156180 con 0x7fc23c15ec30 2024-04-25T16:52:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc232ffd640 1 -- 172.21.15.94:0/3136869043 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234003620 con 0x7fc23c15ec30 2024-04-25T16:52:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc2447d3640 1 -- 172.21.15.94:0/3136869043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c156740 con 0x7fc23c15ec30 2024-04-25T16:52:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc2447d3640 1 -- 172.21.15.94:0/3136869043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc23c09a410 con 0x7fc23c15ec30 2024-04-25T16:52:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc232ffd640 1 -- 172.21.15.94:0/3136869043 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc2340037c0 con 0x7fc23c15ec30 2024-04-25T16:52:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc232ffd640 1 --2- 172.21.15.94:0/3136869043 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc22006f430 0x7fc220071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc232ffd640 1 -- 172.21.15.94:0/3136869043 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fc234129ed0 con 0x7fc23c15ec30 2024-04-25T16:52:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc24256f640 1 --2- 172.21.15.94:0/3136869043 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc22006f430 0x7fc220071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.319+0000 7fc24256f640 1 --2- 172.21.15.94:0/3136869043 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc22006f430 0x7fc220071940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0cd2f0 tx=0x7fc23800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.327+0000 7fc232ffd640 1 -- 172.21.15.94:0/3136869043 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc23c09a410 con 0x7fc23c15ec30 2024-04-25T16:52:20.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.531+0000 7fc2447d3640 1 -- 172.21.15.94:0/3136869043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fc23c076830 con 0x7fc23c15ec30 2024-04-25T16:52:20.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.535+0000 7fc232ffd640 1 -- 172.21.15.94:0/3136869043 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fc23c06f6b0 con 0x7fc23c15ec30 2024-04-25T16:52:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.539+0000 7fc2447d3640 1 -- 172.21.15.94:0/3136869043 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc22006f430 msgr2=0x7fc220071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.539+0000 7fc2447d3640 1 --2- 172.21.15.94:0/3136869043 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc22006f430 0x7fc220071940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0cd2f0 tx=0x7fc23800b040 comp rx=0 tx=0).stop 2024-04-25T16:52:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.539+0000 7fc2447d3640 1 -- 172.21.15.94:0/3136869043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc23c15ec30 msgr2=0x7fc23c1464a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.539+0000 7fc2447d3640 1 --2- 172.21.15.94:0/3136869043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc23c15ec30 0x7fc23c1464a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc23401a080 tx=0x7fc23401ac90 comp rx=0 tx=0).stop 2024-04-25T16:52:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.539+0000 7fc2447d3640 1 -- 172.21.15.94:0/3136869043 shutdown_connections 2024-04-25T16:52:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.539+0000 7fc2447d3640 1 --2- 172.21.15.94:0/3136869043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc23c15ec30 0x7fc23c1464a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.539+0000 7fc2447d3640 1 --2- 172.21.15.94:0/3136869043 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc22006f430 0x7fc220071940 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.539+0000 7fc2447d3640 1 --2- 172.21.15.94:0/3136869043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc23c0b0420 0x7fc23c099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.539+0000 7fc2447d3640 1 --2- 172.21.15.94:0/3136869043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc23c06f2a0 0x7fc23c156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.539+0000 7fc2447d3640 1 -- 172.21.15.94:0/3136869043 >> 172.21.15.94:0/3136869043 conn(0x7fc23c0b47f0 msgr2=0x7fc23c157a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:20.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.539+0000 7fc2447d3640 1 -- 172.21.15.94:0/3136869043 shutdown_connections 2024-04-25T16:52:20.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.539+0000 7fc2447d3640 1 -- 172.21.15.94:0/3136869043 wait complete. 2024-04-25T16:52:20.553 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T16:52:20.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a6bf07640 1 Processor -- start 2024-04-25T16:52:20.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a6bf07640 1 -- start start 2024-04-25T16:52:20.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a6bf07640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0a640d0960 0x7f0a64160b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a6bf07640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a64161050 con 0x7f0a640b73d0 2024-04-25T16:52:20.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a6bf07640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a640d0d70 con 0x7f0a640cea70 2024-04-25T16:52:20.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a6bf07640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0a641463c0 con 0x7f0a640d0960 2024-04-25T16:52:20.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a694a2640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f0a640b73d0 0x7f0a640b77e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45830/0 (socket says 172.21.15.94:45830) 2024-04-25T16:52:20.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a694a2640 1 -- 172.21.15.94:0/2299769819 learned_addr learned my addr 172.21.15.94:0/2299769819 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:20.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3177117802 0 0) 0x7f0a54001120 con 0x7f0a640b73d0 2024-04-25T16:52:20.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a69ca3640 1 --2- 172.21.15.94:0/2299769819 >> v2:172.21.15.138:3300/0 conn(0x7f0a640d0960 0x7f0a64160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:20.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a50003530 con 0x7f0a640b73d0 2024-04-25T16:52:20.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 298963002 0 0) 0x7f0a50003530 con 0x7f0a640b73d0 2024-04-25T16:52:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 >> v2:172.21.15.138:3300/0 conn(0x7f0a640d0960 msgr2=0x7f0a64160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a68ca1640 1 --2- 172.21.15.94:0/2299769819 >> v2:172.21.15.138:3300/0 conn(0x7f0a640d0960 0x7f0a64160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 >> v1:172.21.15.138:6789/0 conn(0x7f0a640cea70 legacy=0x7f0a640cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a640cd1d0 con 0x7f0a640b73d0 2024-04-25T16:52:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f0a54001850 con 0x7f0a640b73d0 2024-04-25T16:52:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 >> v1:172.21.15.94:6789/0 conn(0x7f0a640b73d0 legacy=0x7f0a640b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a68ca1640 1 --2- 172.21.15.94:0/2299769819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a500039c0 0x7f0a50003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a68ca1640 1 --2- 172.21.15.94:0/2299769819 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a50004810 0x7f0a50004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.691+0000 7f0a68ca1640 1 --2- 172.21.15.94:0/2299769819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a50005260 0x7f0a5000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641463c0 con 0x7f0a500039c0 2024-04-25T16:52:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5000c1f0 con 0x7f0a50005260 2024-04-25T16:52:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0a5000c3f0 con 0x7f0a50004810 2024-04-25T16:52:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a69ca3640 1 --2- 172.21.15.94:0/2299769819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a500039c0 0x7f0a50003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a69ca3640 1 -- 172.21.15.94:0/2299769819 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a50004810 msgr2=0x7f0a50004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a69ca3640 1 --2- 172.21.15.94:0/2299769819 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a50004810 0x7f0a50004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a69ca3640 1 -- 172.21.15.94:0/2299769819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a50005260 msgr2=0x7f0a5000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a69ca3640 1 --2- 172.21.15.94:0/2299769819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a50005260 0x7f0a5000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a69ca3640 1 -- 172.21.15.94:0/2299769819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a50003530 con 0x7f0a500039c0 2024-04-25T16:52:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a6a4a4640 1 -- 172.21.15.94:0/2299769819 reap_dead start 2024-04-25T16:52:20.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a69ca3640 1 --2- 172.21.15.94:0/2299769819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a500039c0 0x7f0a50003e30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009ae0 tx=0x7f0a6001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173974e81d6edc89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:20.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600096b0 con 0x7f0a500039c0 2024-04-25T16:52:20.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a600a6490 con 0x7f0a500039c0 2024-04-25T16:52:20.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a68ca1640 1 -- 172.21.15.94:0/2299769819 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600a6880 con 0x7f0a500039c0 2024-04-25T16:52:20.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a6bf07640 1 -- 172.21.15.94:0/2299769819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a500039c0 msgr2=0x7f0a50003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a6bf07640 1 --2- 172.21.15.94:0/2299769819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a500039c0 0x7f0a50003e30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009ae0 tx=0x7f0a6001cab0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a6bf07640 1 -- 172.21.15.94:0/2299769819 shutdown_connections 2024-04-25T16:52:20.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a6bf07640 1 --2- 172.21.15.94:0/2299769819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a500039c0 0x7f0a50003e30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a6bf07640 1 -- 172.21.15.94:0/2299769819 >> 172.21.15.94:0/2299769819 conn(0x7f0a640d33d0 msgr2=0x7f0a640d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:20.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a6bf07640 1 -- 172.21.15.94:0/2299769819 shutdown_connections 2024-04-25T16:52:20.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.695+0000 7f0a6bf07640 1 -- 172.21.15.94:0/2299769819 wait complete. 2024-04-25T16:52:20.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a6bf07640 1 Processor -- start 2024-04-25T16:52:20.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a6bf07640 1 -- start start 2024-04-25T16:52:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a6bf07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a500039c0 0x7f0a640abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a6bf07640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a64072bf0 0x7f0a641fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a6bf07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a64073000 0x7f0a641fd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a6bf07640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a640cc870 con 0x7f0a500039c0 2024-04-25T16:52:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a6bf07640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a640d04d0 con 0x7f0a64073000 2024-04-25T16:52:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a6bf07640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0a640cce50 con 0x7f0a64072bf0 2024-04-25T16:52:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a69ca3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a500039c0 0x7f0a640abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a69ca3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a500039c0 0x7f0a640abdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37624/0 (socket says 172.21.15.94:37624) 2024-04-25T16:52:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a69ca3640 1 -- 172.21.15.94:0/1558704658 learned_addr learned my addr 172.21.15.94:0/1558704658 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a6a4a4640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a64073000 0x7f0a641fd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a694a2640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a64072bf0 0x7f0a641fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:20.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a69ca3640 1 -- 172.21.15.94:0/1558704658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a64072bf0 msgr2=0x7f0a641fcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a69ca3640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a64072bf0 0x7f0a641fcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a69ca3640 1 -- 172.21.15.94:0/1558704658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a64073000 msgr2=0x7f0a641fd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a69ca3640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a64073000 0x7f0a641fd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a69ca3640 1 -- 172.21.15.94:0/1558704658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a640ac330 con 0x7f0a500039c0 2024-04-25T16:52:20.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a6a4a4640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a64073000 0x7f0a641fd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:20.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a69ca3640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a500039c0 0x7f0a640abdf0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0a6001c8e0 tx=0x7f0a60019850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:20.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a5a7fc640 1 -- 172.21.15.94:0/1558704658 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600a6cc0 con 0x7f0a500039c0 2024-04-25T16:52:20.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a5a7fc640 1 -- 172.21.15.94:0/1558704658 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a600a7470 con 0x7f0a500039c0 2024-04-25T16:52:20.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a6bf07640 1 -- 172.21.15.94:0/1558704658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a640aa570 con 0x7f0a500039c0 2024-04-25T16:52:20.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a5a7fc640 1 -- 172.21.15.94:0/1558704658 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600a6e60 con 0x7f0a500039c0 2024-04-25T16:52:20.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.699+0000 7f0a6bf07640 1 -- 172.21.15.94:0/1558704658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a640aab30 con 0x7f0a500039c0 2024-04-25T16:52:20.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.703+0000 7f0a6bf07640 1 -- 172.21.15.94:0/1558704658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a6406a730 con 0x7f0a500039c0 2024-04-25T16:52:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.703+0000 7f0a5a7fc640 1 -- 172.21.15.94:0/1558704658 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0a600a7820 con 0x7f0a500039c0 2024-04-25T16:52:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.703+0000 7f0a5a7fc640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a4806f430 0x7f0a48071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.703+0000 7f0a5a7fc640 1 -- 172.21.15.94:0/1558704658 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f0a60129020 con 0x7f0a500039c0 2024-04-25T16:52:20.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.703+0000 7f0a694a2640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a4806f430 0x7f0a48071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:20.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.703+0000 7f0a694a2640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a4806f430 0x7f0a48071940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a54001060 tx=0x7f0a54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:20.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.707+0000 7f0a5a7fc640 1 -- 172.21.15.94:0/1558704658 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0a600bf060 con 0x7f0a500039c0 2024-04-25T16:52:20.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.915+0000 7f0a6bf07640 1 -- 172.21.15.94:0/1558704658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f0a640b23c0 con 0x7f0a500039c0 2024-04-25T16:52:20.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.919+0000 7f0a5a7fc640 1 -- 172.21.15.94:0/1558704658 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f0a640ac330 con 0x7f0a500039c0 2024-04-25T16:52:20.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.919+0000 7f0a6bf07640 1 -- 172.21.15.94:0/1558704658 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a4806f430 msgr2=0x7f0a48071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.919+0000 7f0a6bf07640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a4806f430 0x7f0a48071940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a54001060 tx=0x7f0a54009040 comp rx=0 tx=0).stop 2024-04-25T16:52:20.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.919+0000 7f0a6bf07640 1 -- 172.21.15.94:0/1558704658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a500039c0 msgr2=0x7f0a640abdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:20.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.919+0000 7f0a6bf07640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a500039c0 0x7f0a640abdf0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0a6001c8e0 tx=0x7f0a60019850 comp rx=0 tx=0).stop 2024-04-25T16:52:20.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.919+0000 7f0a6bf07640 1 -- 172.21.15.94:0/1558704658 shutdown_connections 2024-04-25T16:52:20.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.919+0000 7f0a6bf07640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a64073000 0x7f0a641fd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.919+0000 7f0a6bf07640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a4806f430 0x7f0a48071940 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.919+0000 7f0a6bf07640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a64072bf0 0x7f0a641fcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.919+0000 7f0a6bf07640 1 --2- 172.21.15.94:0/1558704658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a500039c0 0x7f0a640abdf0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:20.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.919+0000 7f0a6bf07640 1 -- 172.21.15.94:0/1558704658 >> 172.21.15.94:0/1558704658 conn(0x7f0a640d33d0 msgr2=0x7f0a640b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:20.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.923+0000 7f0a6bf07640 1 -- 172.21.15.94:0/1558704658 shutdown_connections 2024-04-25T16:52:20.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:20.923+0000 7f0a6bf07640 1 -- 172.21.15.94:0/1558704658 wait complete. 2024-04-25T16:52:20.937 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T16:52:21.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.083+0000 7fbeb54a1640 1 Processor -- start 2024-04-25T16:52:21.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeb54a1640 1 -- start start 2024-04-25T16:52:21.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeb54a1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeb0074120 0x7fbeb0161890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeb54a1640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb015ef20 con 0x7fbeb00d0190 2024-04-25T16:52:21.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeb54a1640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb015e3a0 con 0x7fbeb00b68a0 2024-04-25T16:52:21.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeb54a1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb015e680 con 0x7fbeb0074120 2024-04-25T16:52:21.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeae7fc640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fbeb00d0190 0x7fbeb0073a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45842/0 (socket says 172.21.15.94:45842) 2024-04-25T16:52:21.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeae7fc640 1 -- 172.21.15.94:0/1664106619 learned_addr learned my addr 172.21.15.94:0/1664106619 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:21.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeaf7fe640 1 --2- 172.21.15.94:0/1664106619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeb0074120 0x7fbeb0161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4142271944 0 0) 0x7fbeb015ef20 con 0x7fbeb00d0190 2024-04-25T16:52:21.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe98003530 con 0x7fbeb00d0190 2024-04-25T16:52:21.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3664192942 0 0) 0x7fbe98003530 con 0x7fbeb00d0190 2024-04-25T16:52:21.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeb0074120 msgr2=0x7fbeb0161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:21.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 --2- 172.21.15.94:0/1664106619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeb0074120 0x7fbeb0161890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 >> v1:172.21.15.138:6789/0 conn(0x7fbeb00b68a0 legacy=0x7fbeb015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb015e960 con 0x7fbeb00d0190 2024-04-25T16:52:21.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fbea4001680 con 0x7fbeb00d0190 2024-04-25T16:52:21.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 >> v1:172.21.15.94:6789/0 conn(0x7fbeb00d0190 legacy=0x7fbeb0073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 --2- 172.21.15.94:0/1664106619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe980039f0 0x7fbe98003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 --2- 172.21.15.94:0/1664106619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe98004810 0x7fbe98004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 --2- 172.21.15.94:0/1664106619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe98005160 0x7fbe9800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb015e680 con 0x7fbe980039f0 2024-04-25T16:52:21.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9800bf10 con 0x7fbe98005160 2024-04-25T16:52:21.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbe9800c110 con 0x7fbe98004810 2024-04-25T16:52:21.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeaeffd640 1 --2- 172.21.15.94:0/1664106619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe980039f0 0x7fbe98003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeae7fc640 1 --2- 172.21.15.94:0/1664106619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe98004810 0x7fbe98004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeaeffd640 1 -- 172.21.15.94:0/1664106619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe98004810 msgr2=0x7fbe98004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:21.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeaeffd640 1 --2- 172.21.15.94:0/1664106619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe98004810 0x7fbe98004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeaeffd640 1 -- 172.21.15.94:0/1664106619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe98005160 msgr2=0x7fbe9800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:21.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeaeffd640 1 --2- 172.21.15.94:0/1664106619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe98005160 0x7fbe9800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.087+0000 7fbeaeffd640 1 -- 172.21.15.94:0/1664106619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe98003530 con 0x7fbe980039f0 2024-04-25T16:52:21.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.091+0000 7fbeaeffd640 1 --2- 172.21.15.94:0/1664106619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe980039f0 0x7fbe98003e60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbea0001750 tx=0x7fbea001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4648e6179ed1d7fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:21.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.091+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea00013a0 con 0x7fbe980039f0 2024-04-25T16:52:21.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.091+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea00052f0 con 0x7fbe980039f0 2024-04-25T16:52:21.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.091+0000 7fbeadffb640 1 -- 172.21.15.94:0/1664106619 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0003840 con 0x7fbe980039f0 2024-04-25T16:52:21.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.091+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1664106619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe980039f0 msgr2=0x7fbe98003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.091+0000 7fbeb54a1640 1 --2- 172.21.15.94:0/1664106619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe980039f0 0x7fbe98003e60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbea0001750 tx=0x7fbea001c3e0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.091+0000 7fbeaf7fe640 1 -- 172.21.15.94:0/1664106619 reap_dead start 2024-04-25T16:52:21.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.091+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1664106619 shutdown_connections 2024-04-25T16:52:21.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.091+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1664106619 >> 172.21.15.94:0/1664106619 conn(0x7fbeb00d2690 msgr2=0x7fbeb00d2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:21.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.091+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1664106619 shutdown_connections 2024-04-25T16:52:21.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.091+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1664106619 wait complete. 2024-04-25T16:52:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeb54a1640 1 Processor -- start 2024-04-25T16:52:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeb54a1640 1 -- start start 2024-04-25T16:52:21.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeb54a1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe980039f0 0x7fbeb0155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeb54a1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb015f6f0 0x7fbeb015fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeb54a1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeb00c71d0 0x7fbeb00c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeb54a1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0162220 con 0x7fbeb015f6f0 2024-04-25T16:52:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeb54a1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01620a0 con 0x7fbeb00c71d0 2024-04-25T16:52:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeb54a1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb01623a0 con 0x7fbe980039f0 2024-04-25T16:52:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeae7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb015f6f0 0x7fbeb015fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeae7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb015f6f0 0x7fbeb015fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37642/0 (socket says 172.21.15.94:37642) 2024-04-25T16:52:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeae7fc640 1 -- 172.21.15.94:0/1609681673 learned_addr learned my addr 172.21.15.94:0/1609681673 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeaeffd640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe980039f0 0x7fbeb0155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeaf7fe640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeb00c71d0 0x7fbeb00c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeae7fc640 1 -- 172.21.15.94:0/1609681673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe980039f0 msgr2=0x7fbeb0155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeae7fc640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe980039f0 0x7fbeb0155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeae7fc640 1 -- 172.21.15.94:0/1609681673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeb00c71d0 msgr2=0x7fbeb00c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeae7fc640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeb00c71d0 0x7fbeb00c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeae7fc640 1 -- 172.21.15.94:0/1609681673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb00c7b20 con 0x7fbeb015f6f0 2024-04-25T16:52:21.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeae7fc640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb015f6f0 0x7fbeb015fb00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbea400d730 tx=0x7fbea400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:21.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbe977fe640 1 -- 172.21.15.94:0/1609681673 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4019070 con 0x7fbeb015f6f0 2024-04-25T16:52:21.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbe977fe640 1 -- 172.21.15.94:0/1609681673 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea40074e0 con 0x7fbeb015f6f0 2024-04-25T16:52:21.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1609681673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb0156420 con 0x7fbeb015f6f0 2024-04-25T16:52:21.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbe977fe640 1 -- 172.21.15.94:0/1609681673 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4004a30 con 0x7fbeb015f6f0 2024-04-25T16:52:21.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1609681673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb00ca3e0 con 0x7fbeb015f6f0 2024-04-25T16:52:21.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.095+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1609681673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb00c9e60 con 0x7fbeb015f6f0 2024-04-25T16:52:21.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.099+0000 7fbe977fe640 1 -- 172.21.15.94:0/1609681673 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbea401c070 con 0x7fbeb015f6f0 2024-04-25T16:52:21.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.099+0000 7fbe977fe640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbe9c06f400 0x7fbe9c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.099+0000 7fbe977fe640 1 -- 172.21.15.94:0/1609681673 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fbea408e970 con 0x7fbeb015f6f0 2024-04-25T16:52:21.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.099+0000 7fbeaeffd640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbe9c06f400 0x7fbe9c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.099+0000 7fbeaeffd640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbe9c06f400 0x7fbe9c071910 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbea00ac000 tx=0x7fbea00011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.103+0000 7fbe977fe640 1 -- 172.21.15.94:0/1609681673 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbeb00c9e60 con 0x7fbeb015f6f0 2024-04-25T16:52:21.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.315+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1609681673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fbeb006a730 con 0x7fbeb015f6f0 2024-04-25T16:52:21.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.319+0000 7fbe977fe640 1 -- 172.21.15.94:0/1609681673 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fbeb00c7b20 con 0x7fbeb015f6f0 2024-04-25T16:52:21.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.319+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1609681673 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbe9c06f400 msgr2=0x7fbe9c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.319+0000 7fbeb54a1640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbe9c06f400 0x7fbe9c071910 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbea00ac000 tx=0x7fbea00011a0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.319+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1609681673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb015f6f0 msgr2=0x7fbeb015fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.319+0000 7fbeb54a1640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb015f6f0 0x7fbeb015fb00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbea400d730 tx=0x7fbea400dbf0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.319+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1609681673 shutdown_connections 2024-04-25T16:52:21.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.319+0000 7fbeb54a1640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeb00c71d0 0x7fbeb00c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.319+0000 7fbeb54a1640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb015f6f0 0x7fbeb015fb00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.319+0000 7fbeb54a1640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbe9c06f400 0x7fbe9c071910 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.319+0000 7fbeb54a1640 1 --2- 172.21.15.94:0/1609681673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe980039f0 0x7fbeb0155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.323+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1609681673 >> 172.21.15.94:0/1609681673 conn(0x7fbeb00d2690 msgr2=0x7fbeb015dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:21.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.323+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1609681673 shutdown_connections 2024-04-25T16:52:21.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.323+0000 7fbeb54a1640 1 -- 172.21.15.94:0/1609681673 wait complete. 2024-04-25T16:52:21.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T16:52:21.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe211005640 1 Processor -- start 2024-04-25T16:52:21.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe211005640 1 -- start start 2024-04-25T16:52:21.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe211005640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe20c15c7e0 0x7fe20c0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe211005640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe20c06de20 0x7fe20c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe211005640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe20c074380 con 0x7fe20c0b0330 2024-04-25T16:52:21.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe211005640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe20c0727e0 con 0x7fe20c06de20 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe211005640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe20c072ac0 con 0x7fe20c15c7e0 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe20a59c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fe20c0b0330 0x7fe20c06d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45846/0 (socket says 172.21.15.94:45846) 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe20a59c640 1 -- 172.21.15.94:0/2594391827 learned_addr learned my addr 172.21.15.94:0/2594391827 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe20ad9d640 1 --2- 172.21.15.94:0/2594391827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe20c15c7e0 0x7fe20c0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe20b59e640 1 --2- 172.21.15.94:0/2594391827 >> v2:172.21.15.138:3300/0 conn(0x7fe20c06de20 0x7fe20c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2806496624 0 0) 0x7fe200001120 con 0x7fe20c0b0330 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1f4003530 con 0x7fe20c0b0330 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 224065289 0 0) 0x7fe1f4003530 con 0x7fe20c0b0330 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe20c15c7e0 msgr2=0x7fe20c0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 --2- 172.21.15.94:0/2594391827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe20c15c7e0 0x7fe20c0afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 >> v2:172.21.15.138:3300/0 conn(0x7fe20c06de20 msgr2=0x7fe20c06e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 --2- 172.21.15.94:0/2594391827 >> v2:172.21.15.138:3300/0 conn(0x7fe20c06de20 0x7fe20c06e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20c072da0 con 0x7fe20c0b0330 2024-04-25T16:52:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fe200001850 con 0x7fe20c0b0330 2024-04-25T16:52:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 >> v1:172.21.15.94:6789/0 conn(0x7fe20c0b0330 legacy=0x7fe20c06d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 --2- 172.21.15.94:0/2594391827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe20c15c7e0 0x7fe1f40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 --2- 172.21.15.94:0/2594391827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1f4003f40 0x7fe1f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 --2- 172.21.15.94:0/2594391827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f4004d50 0x7fe1f4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c0727e0 con 0x7fe1f4003f40 2024-04-25T16:52:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c072ac0 con 0x7fe1f4004d50 2024-04-25T16:52:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.487+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f400ba20 con 0x7fe20c15c7e0 2024-04-25T16:52:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe20a59c640 1 --2- 172.21.15.94:0/2594391827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1f4003f40 0x7fe1f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe20ad9d640 1 --2- 172.21.15.94:0/2594391827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe20c15c7e0 0x7fe1f40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe20b59e640 1 --2- 172.21.15.94:0/2594391827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f4004d50 0x7fe1f4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe20a59c640 1 -- 172.21.15.94:0/2594391827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe20c15c7e0 msgr2=0x7fe1f40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe20a59c640 1 --2- 172.21.15.94:0/2594391827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe20c15c7e0 0x7fe1f40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe20a59c640 1 -- 172.21.15.94:0/2594391827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f4004d50 msgr2=0x7fe1f4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe20a59c640 1 --2- 172.21.15.94:0/2594391827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f4004d50 0x7fe1f4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe20a59c640 1 -- 172.21.15.94:0/2594391827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f4003530 con 0x7fe1f4003f40 2024-04-25T16:52:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe20b59e640 1 --2- 172.21.15.94:0/2594391827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f4004d50 0x7fe1f4005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe20a59c640 1 --2- 172.21.15.94:0/2594391827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1f4003f40 0x7fe1f4004810 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0b6970 tx=0x7fe20001f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43a456f4a639b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20000c0e0 con 0x7fe1f4003f40 2024-04-25T16:52:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe200005b00 con 0x7fe1f4003f40 2024-04-25T16:52:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe209d9b640 1 -- 172.21.15.94:0/2594391827 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200007070 con 0x7fe1f4003f40 2024-04-25T16:52:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe211005640 1 -- 172.21.15.94:0/2594391827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1f4003f40 msgr2=0x7fe1f4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe211005640 1 --2- 172.21.15.94:0/2594391827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1f4003f40 0x7fe1f4004810 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0b6970 tx=0x7fe20001f3a0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe20b59e640 1 -- 172.21.15.94:0/2594391827 reap_dead start 2024-04-25T16:52:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe211005640 1 -- 172.21.15.94:0/2594391827 shutdown_connections 2024-04-25T16:52:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe211005640 1 -- 172.21.15.94:0/2594391827 >> 172.21.15.94:0/2594391827 conn(0x7fe20c0b47f0 msgr2=0x7fe20c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe211005640 1 -- 172.21.15.94:0/2594391827 shutdown_connections 2024-04-25T16:52:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.491+0000 7fe211005640 1 -- 172.21.15.94:0/2594391827 wait complete. 2024-04-25T16:52:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe211005640 1 Processor -- start 2024-04-25T16:52:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe211005640 1 -- start start 2024-04-25T16:52:21.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe211005640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1f4003f40 0x7fe20c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe211005640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe20c09bb80 0x7fe20c14d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe211005640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe20c1f78f0 0x7fe20c1f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe211005640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c073ca0 con 0x7fe20c1f78f0 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe211005640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c073360 con 0x7fe20c09bb80 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe211005640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe20c15cbf0 con 0x7fe1f4003f40 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe20b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe20c1f78f0 0x7fe20c1f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe20b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe20c1f78f0 0x7fe20c1f7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37656/0 (socket says 172.21.15.94:37656) 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe20b59e640 1 -- 172.21.15.94:0/1732727199 learned_addr learned my addr 172.21.15.94:0/1732727199 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe20a59c640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe20c09bb80 0x7fe20c14d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe20ad9d640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1f4003f40 0x7fe20c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe20b59e640 1 -- 172.21.15.94:0/1732727199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1f4003f40 msgr2=0x7fe20c15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe20b59e640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1f4003f40 0x7fe20c15a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe20b59e640 1 -- 172.21.15.94:0/1732727199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe20c09bb80 msgr2=0x7fe20c14d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe20b59e640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe20c09bb80 0x7fe20c14d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe20b59e640 1 -- 172.21.15.94:0/1732727199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20c1f8240 con 0x7fe20c1f78f0 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe20ad9d640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1f4003f40 0x7fe20c15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe20b59e640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe20c1f78f0 0x7fe20c1f7d00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0ae550 tx=0x7fe20400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:21.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe1f37fe640 1 -- 172.21.15.94:0/1732727199 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20401c070 con 0x7fe20c1f78f0 2024-04-25T16:52:21.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe1f37fe640 1 -- 172.21.15.94:0/1732727199 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe204017400 con 0x7fe20c1f78f0 2024-04-25T16:52:21.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe211005640 1 -- 172.21.15.94:0/1732727199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20c15ae10 con 0x7fe20c1f78f0 2024-04-25T16:52:21.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe1f37fe640 1 -- 172.21.15.94:0/1732727199 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2040177d0 con 0x7fe20c1f78f0 2024-04-25T16:52:21.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.495+0000 7fe211005640 1 -- 172.21.15.94:0/1732727199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20c15b3d0 con 0x7fe20c1f78f0 2024-04-25T16:52:21.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.499+0000 7fe211005640 1 -- 172.21.15.94:0/1732727199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe20c15b610 con 0x7fe20c1f78f0 2024-04-25T16:52:21.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.499+0000 7fe1f37fe640 1 -- 172.21.15.94:0/1732727199 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe20401f070 con 0x7fe20c1f78f0 2024-04-25T16:52:21.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.499+0000 7fe1f37fe640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe1f806f400 0x7fe1f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.499+0000 7fe1f37fe640 1 -- 172.21.15.94:0/1732727199 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fe20408f720 con 0x7fe20c1f78f0 2024-04-25T16:52:21.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.499+0000 7fe20ad9d640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe1f806f400 0x7fe1f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.499+0000 7fe20ad9d640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe1f806f400 0x7fe1f8071910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009160 tx=0x7fe1fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:21.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.503+0000 7fe1f37fe640 1 -- 172.21.15.94:0/1732727199 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe204026080 con 0x7fe20c1f78f0 2024-04-25T16:52:21.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.707+0000 7fe211005640 1 -- 172.21.15.94:0/1732727199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fe20c15aba0 con 0x7fe20c1f78f0 2024-04-25T16:52:21.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.711+0000 7fe1f37fe640 1 -- 172.21.15.94:0/1732727199 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fe20c1f8240 con 0x7fe20c1f78f0 2024-04-25T16:52:21.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.711+0000 7fe211005640 1 -- 172.21.15.94:0/1732727199 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe1f806f400 msgr2=0x7fe1f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.711+0000 7fe211005640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe1f806f400 0x7fe1f8071910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009160 tx=0x7fe1fc008040 comp rx=0 tx=0).stop 2024-04-25T16:52:21.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.711+0000 7fe211005640 1 -- 172.21.15.94:0/1732727199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe20c1f78f0 msgr2=0x7fe20c1f7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.711+0000 7fe211005640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe20c1f78f0 0x7fe20c1f7d00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0ae550 tx=0x7fe20400f4a0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.711+0000 7fe211005640 1 -- 172.21.15.94:0/1732727199 shutdown_connections 2024-04-25T16:52:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.711+0000 7fe211005640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe20c1f78f0 0x7fe20c1f7d00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.711+0000 7fe211005640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe20c09bb80 0x7fe20c14d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.711+0000 7fe211005640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe1f806f400 0x7fe1f8071910 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.711+0000 7fe211005640 1 --2- 172.21.15.94:0/1732727199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe1f4003f40 0x7fe20c15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.711+0000 7fe211005640 1 -- 172.21.15.94:0/1732727199 >> 172.21.15.94:0/1732727199 conn(0x7fe20c0b47f0 msgr2=0x7fe20c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.711+0000 7fe211005640 1 -- 172.21.15.94:0/1732727199 shutdown_connections 2024-04-25T16:52:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.711+0000 7fe211005640 1 -- 172.21.15.94:0/1732727199 wait complete. 2024-04-25T16:52:21.729 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T16:52:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.871+0000 7f3c7081c640 1 Processor -- start 2024-04-25T16:52:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.871+0000 7f3c7081c640 1 -- start start 2024-04-25T16:52:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c7081c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c6c06f470 0x7f3c6c15dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c7081c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3c6c15e160 0x7f3c6c15e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c7081c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c6c070630 con 0x7f3c6c0b0420 2024-04-25T16:52:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c7081c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3c6c0aeaa0 con 0x7f3c6c15e160 2024-04-25T16:52:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c7081c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3c6c0703e0 con 0x7f3c6c06f470 2024-04-25T16:52:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6a59c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f3c6c0b0420 0x7f3c6c06ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45848/0 (socket says 172.21.15.94:45848) 2024-04-25T16:52:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6a59c640 1 -- 172.21.15.94:0/3133942498 learned_addr learned my addr 172.21.15.94:0/3133942498 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6ad9d640 1 --2- 172.21.15.94:0/3133942498 >> v2:172.21.15.138:3300/0 conn(0x7f3c6c15e160 0x7f3c6c15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1294986329 0 0) 0x7f3c6c070630 con 0x7f3c6c0b0420 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c54003530 con 0x7f3c6c0b0420 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c69d9b640 1 --2- 172.21.15.94:0/3133942498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c6c06f470 0x7f3c6c15dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1478711173 0 0) 0x7f3c54003530 con 0x7f3c6c0b0420 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c6c06f470 msgr2=0x7f3c6c15dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 --2- 172.21.15.94:0/3133942498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c6c06f470 0x7f3c6c15dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 >> v2:172.21.15.138:3300/0 conn(0x7f3c6c15e160 msgr2=0x7f3c6c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 --2- 172.21.15.94:0/3133942498 >> v2:172.21.15.138:3300/0 conn(0x7f3c6c15e160 0x7f3c6c15e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6c0ae170 con 0x7f3c6c0b0420 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f3c5c001680 con 0x7f3c6c0b0420 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 >> v1:172.21.15.94:6789/0 conn(0x7f3c6c0b0420 legacy=0x7f3c6c06ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 --2- 172.21.15.94:0/3133942498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c540039c0 0x7f3c54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 --2- 172.21.15.94:0/3133942498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54004350 0x7f3c54004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 --2- 172.21.15.94:0/3133942498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c54004ee0 0x7f3c5400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c0aeaa0 con 0x7f3c54004ee0 2024-04-25T16:52:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c0703e0 con 0x7f3c54004350 2024-04-25T16:52:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3c5400bf10 con 0x7f3c540039c0 2024-04-25T16:52:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6ad9d640 1 --2- 172.21.15.94:0/3133942498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c54004ee0 0x7f3c5400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6a59c640 1 --2- 172.21.15.94:0/3133942498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c540039c0 0x7f3c54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c69d9b640 1 --2- 172.21.15.94:0/3133942498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54004350 0x7f3c54004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6ad9d640 1 -- 172.21.15.94:0/3133942498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c540039c0 msgr2=0x7f3c54003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6ad9d640 1 --2- 172.21.15.94:0/3133942498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c540039c0 0x7f3c54003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6ad9d640 1 -- 172.21.15.94:0/3133942498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54004350 msgr2=0x7f3c54004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6ad9d640 1 --2- 172.21.15.94:0/3133942498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54004350 0x7f3c54004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c6ad9d640 1 -- 172.21.15.94:0/3133942498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c54003530 con 0x7f3c54004ee0 2024-04-25T16:52:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.875+0000 7f3c69d9b640 1 --2- 172.21.15.94:0/3133942498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54004350 0x7f3c54004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.879+0000 7f3c6ad9d640 1 --2- 172.21.15.94:0/3133942498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c54004ee0 0x7f3c5400b9d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009f90 tx=0x7f3c6401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2771fdf43be7f04a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.879+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c640b2070 con 0x7f3c54004ee0 2024-04-25T16:52:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.879+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c640a5c90 con 0x7f3c54004ee0 2024-04-25T16:52:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.879+0000 7f3c6959a640 1 -- 172.21.15.94:0/3133942498 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c64009c10 con 0x7f3c54004ee0 2024-04-25T16:52:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.879+0000 7f3c7081c640 1 -- 172.21.15.94:0/3133942498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c54004ee0 msgr2=0x7f3c5400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.879+0000 7f3c7081c640 1 --2- 172.21.15.94:0/3133942498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c54004ee0 0x7f3c5400b9d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009f90 tx=0x7f3c6401cc80 comp rx=0 tx=0).stop 2024-04-25T16:52:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.879+0000 7f3c6ad9d640 1 -- 172.21.15.94:0/3133942498 reap_dead start 2024-04-25T16:52:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.879+0000 7f3c7081c640 1 -- 172.21.15.94:0/3133942498 shutdown_connections 2024-04-25T16:52:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.879+0000 7f3c7081c640 1 -- 172.21.15.94:0/3133942498 >> 172.21.15.94:0/3133942498 conn(0x7f3c6c0b47f0 msgr2=0x7f3c6c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.879+0000 7f3c7081c640 1 -- 172.21.15.94:0/3133942498 shutdown_connections 2024-04-25T16:52:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.879+0000 7f3c7081c640 1 -- 172.21.15.94:0/3133942498 wait complete. 2024-04-25T16:52:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.879+0000 7f3c7081c640 1 Processor -- start 2024-04-25T16:52:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c7081c640 1 -- start start 2024-04-25T16:52:21.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c7081c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c54004ee0 0x7f3c6c157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c7081c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c6c1f3800 0x7f3c6c1f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c7081c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c6c1f4150 0x7f3c6c1f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c7081c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c0aef30 con 0x7f3c54004ee0 2024-04-25T16:52:21.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c7081c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c0ae730 con 0x7f3c6c1f4150 2024-04-25T16:52:21.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c7081c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3c6c0b0830 con 0x7f3c6c1f3800 2024-04-25T16:52:21.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c54004ee0 0x7f3c6c157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c54004ee0 0x7f3c6c157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37680/0 (socket says 172.21.15.94:37680) 2024-04-25T16:52:21.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c6a59c640 1 -- 172.21.15.94:0/3501125776 learned_addr learned my addr 172.21.15.94:0/3501125776 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:21.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c6a59c640 1 -- 172.21.15.94:0/3501125776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c6c1f3800 msgr2=0x7f3c6c1f3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c6a59c640 1 --2- 172.21.15.94:0/3501125776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c6c1f3800 0x7f3c6c1f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c6a59c640 1 -- 172.21.15.94:0/3501125776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c6c1f4150 msgr2=0x7f3c6c1f5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:21.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c6a59c640 1 --2- 172.21.15.94:0/3501125776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c6c1f4150 0x7f3c6c1f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:21.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c6a59c640 1 -- 172.21.15.94:0/3501125776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6c1f4ab0 con 0x7f3c54004ee0 2024-04-25T16:52:21.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c6a59c640 1 --2- 172.21.15.94:0/3501125776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c54004ee0 0x7f3c6c157380 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c001ec0 tx=0x7f3c5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:21.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c5affd640 1 -- 172.21.15.94:0/3501125776 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c00c820 con 0x7f3c54004ee0 2024-04-25T16:52:21.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c5affd640 1 -- 172.21.15.94:0/3501125776 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c5c018070 con 0x7f3c54004ee0 2024-04-25T16:52:21.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c7081c640 1 -- 172.21.15.94:0/3501125776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6c1f4d90 con 0x7f3c54004ee0 2024-04-25T16:52:21.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c5affd640 1 -- 172.21.15.94:0/3501125776 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c00cc20 con 0x7f3c54004ee0 2024-04-25T16:52:21.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c7081c640 1 -- 172.21.15.94:0/3501125776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c6c1f5350 con 0x7f3c54004ee0 2024-04-25T16:52:21.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c7081c640 1 -- 172.21.15.94:0/3501125776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c6c076830 con 0x7f3c54004ee0 2024-04-25T16:52:21.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.883+0000 7f3c5affd640 1 -- 172.21.15.94:0/3501125776 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3c5c01d460 con 0x7f3c54004ee0 2024-04-25T16:52:21.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.887+0000 7f3c5affd640 1 --2- 172.21.15.94:0/3501125776 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c4c06f400 0x7f3c4c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:21.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.887+0000 7f3c69d9b640 1 --2- 172.21.15.94:0/3501125776 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c4c06f400 0x7f3c4c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:21.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.887+0000 7f3c5affd640 1 -- 172.21.15.94:0/3501125776 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f3c5c08f000 con 0x7f3c54004ee0 2024-04-25T16:52:21.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.887+0000 7f3c69d9b640 1 --2- 172.21.15.94:0/3501125776 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c4c06f400 0x7f3c4c071910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c15bbf0 tx=0x7f3c60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:21.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:21.891+0000 7f3c5affd640 1 -- 172.21.15.94:0/3501125776 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3c5c05c140 con 0x7f3c54004ee0 2024-04-25T16:52:22.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.107+0000 7f3c7081c640 1 -- 172.21.15.94:0/3501125776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f3c6c06fb20 con 0x7f3c54004ee0 2024-04-25T16:52:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.111+0000 7f3c5affd640 1 -- 172.21.15.94:0/3501125776 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f3c5c064130 con 0x7f3c54004ee0 2024-04-25T16:52:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.119+0000 7f3c7081c640 1 -- 172.21.15.94:0/3501125776 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c4c06f400 msgr2=0x7f3c4c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.119+0000 7f3c7081c640 1 --2- 172.21.15.94:0/3501125776 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c4c06f400 0x7f3c4c071910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c15bbf0 tx=0x7f3c60008040 comp rx=0 tx=0).stop 2024-04-25T16:52:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.119+0000 7f3c7081c640 1 -- 172.21.15.94:0/3501125776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c54004ee0 msgr2=0x7f3c6c157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.119+0000 7f3c7081c640 1 --2- 172.21.15.94:0/3501125776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c54004ee0 0x7f3c6c157380 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c001ec0 tx=0x7f3c5c00ecb0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.119+0000 7f3c7081c640 1 -- 172.21.15.94:0/3501125776 shutdown_connections 2024-04-25T16:52:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.119+0000 7f3c7081c640 1 --2- 172.21.15.94:0/3501125776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c6c1f4150 0x7f3c6c1f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.119+0000 7f3c7081c640 1 --2- 172.21.15.94:0/3501125776 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c4c06f400 0x7f3c4c071910 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.119+0000 7f3c7081c640 1 --2- 172.21.15.94:0/3501125776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c6c1f3800 0x7f3c6c1f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.119+0000 7f3c7081c640 1 --2- 172.21.15.94:0/3501125776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c54004ee0 0x7f3c6c157380 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.119+0000 7f3c7081c640 1 -- 172.21.15.94:0/3501125776 >> 172.21.15.94:0/3501125776 conn(0x7f3c6c0b47f0 msgr2=0x7f3c6c15c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.119+0000 7f3c7081c640 1 -- 172.21.15.94:0/3501125776 shutdown_connections 2024-04-25T16:52:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.119+0000 7f3c7081c640 1 -- 172.21.15.94:0/3501125776 wait complete. 2024-04-25T16:52:22.135 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T16:52:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f07615640 1 Processor -- start 2024-04-25T16:52:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f07615640 1 -- start start 2024-04-25T16:52:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f07615640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f0015c7e0 0x7f9f000afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f07615640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f000b0330 0x7f9f0006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f07615640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9f0006dc50 0x7f9f0006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f07615640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f9f00074380 con 0x7f9f0006dc50 2024-04-25T16:52:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f07615640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9f0006e060 con 0x7f9f000b0330 2024-04-25T16:52:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f07615640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f000727e0 con 0x7f9f0015c7e0 2024-04-25T16:52:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f05bb2640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9f0006dc50 0x7f9f0006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f05bb2640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9f0006dc50 0x7f9f0006e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37684/0 (socket says 172.21.15.94:37684) 2024-04-25T16:52:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f05bb2640 1 -- 172.21.15.94:0/2090095090 learned_addr learned my addr 172.21.15.94:0/2090095090 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f04bb0640 1 --2- 172.21.15.94:0/2090095090 >> v2:172.21.15.138:3300/0 conn(0x7f9f000b0330 0x7f9f0006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f053b1640 1 --2- 172.21.15.94:0/2090095090 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f0015c7e0 0x7f9f000afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f05bb2640 1 -- 172.21.15.94:0/2090095090 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f0015c7e0 msgr2=0x7f9f000afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f05bb2640 1 --2- 172.21.15.94:0/2090095090 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f0015c7e0 0x7f9f000afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f05bb2640 1 -- 172.21.15.94:0/2090095090 >> v2:172.21.15.138:3300/0 conn(0x7f9f000b0330 msgr2=0x7f9f0006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f05bb2640 1 --2- 172.21.15.94:0/2090095090 >> v2:172.21.15.138:3300/0 conn(0x7f9f000b0330 0x7f9f0006d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f05bb2640 1 -- 172.21.15.94:0/2090095090 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f00072ac0 con 0x7f9f0006dc50 2024-04-25T16:52:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.275+0000 7f9f04bb0640 1 --2- 172.21.15.94:0/2090095090 >> v2:172.21.15.138:3300/0 conn(0x7f9f000b0330 0x7f9f0006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.279+0000 7f9f05bb2640 1 --2- 172.21.15.94:0/2090095090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f0006dc50 0x7f9f0006e7a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9f000b5050 tx=0x7f9ef801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7025159ce4c951a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.279+0000 7f9ef7fff640 1 -- 172.21.15.94:0/2090095090 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80a5840 con 0x7f9f0006dc50 2024-04-25T16:52:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.279+0000 7f9ef7fff640 1 -- 172.21.15.94:0/2090095090 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef80a59e0 con 0x7f9f0006dc50 2024-04-25T16:52:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.279+0000 7f9ef7fff640 1 -- 172.21.15.94:0/2090095090 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80a5db0 con 0x7f9f0006dc50 2024-04-25T16:52:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.279+0000 7f9f07615640 1 -- 172.21.15.94:0/2090095090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f0006dc50 msgr2=0x7f9f0006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.279+0000 7f9f07615640 1 --2- 172.21.15.94:0/2090095090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f0006dc50 0x7f9f0006e7a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9f000b5050 tx=0x7f9ef801c9c0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.279+0000 7f9f07615640 1 -- 172.21.15.94:0/2090095090 shutdown_connections 2024-04-25T16:52:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.279+0000 7f9f07615640 1 --2- 172.21.15.94:0/2090095090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f0006dc50 0x7f9f0006e7a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.279+0000 7f9f07615640 1 --2- 172.21.15.94:0/2090095090 >> v2:172.21.15.138:3300/0 conn(0x7f9f000b0330 0x7f9f0006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.279+0000 7f9f07615640 1 --2- 172.21.15.94:0/2090095090 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f0015c7e0 0x7f9f000afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.279+0000 7f9f07615640 1 -- 172.21.15.94:0/2090095090 >> 172.21.15.94:0/2090095090 conn(0x7f9f000b47f0 msgr2=0x7f9f000b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.279+0000 7f9f07615640 1 -- 172.21.15.94:0/2090095090 shutdown_connections 2024-04-25T16:52:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.279+0000 7f9f07615640 1 -- 172.21.15.94:0/2090095090 wait complete. 2024-04-25T16:52:22.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f07615640 1 Processor -- start 2024-04-25T16:52:22.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f07615640 1 -- start start 2024-04-25T16:52:22.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f07615640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f0006dc50 0x7f9f0009da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f07615640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f000b0330 0x7f9f0009df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f07615640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f0015c7e0 0x7f9f001f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f07615640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0015cbf0 con 0x7f9f000b0330 2024-04-25T16:52:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f07615640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f00073ca0 con 0x7f9f0006dc50 2024-04-25T16:52:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f07615640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f00160460 con 0x7f9f0015c7e0 2024-04-25T16:52:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f04bb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f000b0330 0x7f9f0009df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f04bb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f000b0330 0x7f9f0009df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37696/0 (socket says 172.21.15.94:37696) 2024-04-25T16:52:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f04bb0640 1 -- 172.21.15.94:0/1354828120 learned_addr learned my addr 172.21.15.94:0/1354828120 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f053b1640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f0006dc50 0x7f9f0009da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f05bb2640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f0015c7e0 0x7f9f001f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f04bb0640 1 -- 172.21.15.94:0/1354828120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f0015c7e0 msgr2=0x7f9f001f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f04bb0640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f0015c7e0 0x7f9f001f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f04bb0640 1 -- 172.21.15.94:0/1354828120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f0006dc50 msgr2=0x7f9f0009da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f04bb0640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f0006dc50 0x7f9f0009da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f04bb0640 1 -- 172.21.15.94:0/1354828120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0009be20 con 0x7f9f000b0330 2024-04-25T16:52:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f05bb2640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f0015c7e0 0x7f9f001f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f04bb0640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f000b0330 0x7f9f0009df90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000c5e0 tx=0x7f9ef000caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9ef5ffb640 1 -- 172.21.15.94:0/1354828120 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0015070 con 0x7f9f000b0330 2024-04-25T16:52:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9ef5ffb640 1 -- 172.21.15.94:0/1354828120 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef0011800 con 0x7f9f000b0330 2024-04-25T16:52:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f07615640 1 -- 172.21.15.94:0/1354828120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0009c100 con 0x7f9f000b0330 2024-04-25T16:52:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9ef5ffb640 1 -- 172.21.15.94:0/1354828120 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0011bf0 con 0x7f9f000b0330 2024-04-25T16:52:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f07615640 1 -- 172.21.15.94:0/1354828120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0009c6c0 con 0x7f9f000b0330 2024-04-25T16:52:22.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.283+0000 7f9f07615640 1 -- 172.21.15.94:0/1354828120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f00076830 con 0x7f9f000b0330 2024-04-25T16:52:22.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.287+0000 7f9ef5ffb640 1 -- 172.21.15.94:0/1354828120 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9ef0018070 con 0x7f9f000b0330 2024-04-25T16:52:22.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.287+0000 7f9ef5ffb640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9ee406f400 0x7f9ee4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.287+0000 7f9ef5ffb640 1 -- 172.21.15.94:0/1354828120 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f9ef008f3b0 con 0x7f9f000b0330 2024-04-25T16:52:22.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.287+0000 7f9f053b1640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9ee406f400 0x7f9ee4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.287+0000 7f9f053b1640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9ee406f400 0x7f9ee4071910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00a8d0 tx=0x7f9efc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.291+0000 7f9ef5ffb640 1 -- 172.21.15.94:0/1354828120 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9ef005c460 con 0x7f9f000b0330 2024-04-25T16:52:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.491+0000 7f9f07615640 1 -- 172.21.15.94:0/1354828120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f9f0009bb70 con 0x7f9f000b0330 2024-04-25T16:52:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.499+0000 7f9ef5ffb640 1 -- 172.21.15.94:0/1354828120 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f9ef005bd60 con 0x7f9f000b0330 2024-04-25T16:52:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.503+0000 7f9f07615640 1 -- 172.21.15.94:0/1354828120 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9ee406f400 msgr2=0x7f9ee4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.503+0000 7f9f07615640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9ee406f400 0x7f9ee4071910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00a8d0 tx=0x7f9efc008040 comp rx=0 tx=0).stop 2024-04-25T16:52:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.503+0000 7f9f07615640 1 -- 172.21.15.94:0/1354828120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f000b0330 msgr2=0x7f9f0009df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.503+0000 7f9f07615640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f000b0330 0x7f9f0009df90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000c5e0 tx=0x7f9ef000caa0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.503+0000 7f9f07615640 1 -- 172.21.15.94:0/1354828120 shutdown_connections 2024-04-25T16:52:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.503+0000 7f9f07615640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f9ee406f400 0x7f9ee4071910 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.503+0000 7f9f07615640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f0015c7e0 0x7f9f001f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.503+0000 7f9f07615640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f000b0330 0x7f9f0009df90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.503+0000 7f9f07615640 1 --2- 172.21.15.94:0/1354828120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f0006dc50 0x7f9f0009da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.503+0000 7f9f07615640 1 -- 172.21.15.94:0/1354828120 >> 172.21.15.94:0/1354828120 conn(0x7f9f000b47f0 msgr2=0x7f9f0015ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.503+0000 7f9f07615640 1 -- 172.21.15.94:0/1354828120 shutdown_connections 2024-04-25T16:52:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.503+0000 7f9f07615640 1 -- 172.21.15.94:0/1354828120 wait complete. 2024-04-25T16:52:22.519 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:52:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f303abe4640 1 Processor -- start 2024-04-25T16:52:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f303abe4640 1 -- start start 2024-04-25T16:52:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f303abe4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f303415e990 0x7f3034165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f303abe4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3034165b90 0x7f3034167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f303abe4640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f303406d610 con 0x7f3034070080 2024-04-25T16:52:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f303abe4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f3034158550 con 0x7f303415e990 2024-04-25T16:52:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f303abe4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f303406f7d0 con 0x7f3034165b90 2024-04-25T16:52:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f302bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f303415e990 0x7f3034165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f302bfff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f303415e990 0x7f3034165650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37712/0 (socket says 172.21.15.94:37712) 2024-04-25T16:52:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f302bfff640 1 -- 172.21.15.94:0/1237954063 learned_addr learned my addr 172.21.15.94:0/1237954063 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f3039181640 1 --2- 172.21.15.94:0/1237954063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3034165b90 0x7f3034167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f302bfff640 1 -- 172.21.15.94:0/1237954063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3034165b90 msgr2=0x7f3034167fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f302bfff640 1 --2- 172.21.15.94:0/1237954063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3034165b90 0x7f3034167fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f302bfff640 1 -- 172.21.15.94:0/1237954063 >> v1:172.21.15.138:6789/0 conn(0x7f3034070080 legacy=0x7f303415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.663+0000 7f302bfff640 1 -- 172.21.15.94:0/1237954063 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303406fc60 con 0x7f303415e990 2024-04-25T16:52:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f302bfff640 1 --2- 172.21.15.94:0/1237954063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f303415e990 0x7f3034165650 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3024009870 tx=0x7f302401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa05dc5945883dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f302b7fe640 1 -- 172.21.15.94:0/1237954063 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30240a9400 con 0x7f303415e990 2024-04-25T16:52:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f302b7fe640 1 -- 172.21.15.94:0/1237954063 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30240a95a0 con 0x7f303415e990 2024-04-25T16:52:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f302b7fe640 1 -- 172.21.15.94:0/1237954063 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30240a9970 con 0x7f303415e990 2024-04-25T16:52:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f303abe4640 1 -- 172.21.15.94:0/1237954063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f303415e990 msgr2=0x7f3034165650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f303abe4640 1 --2- 172.21.15.94:0/1237954063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f303415e990 0x7f3034165650 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3024009870 tx=0x7f302401c540 comp rx=0 tx=0).stop 2024-04-25T16:52:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f303abe4640 1 -- 172.21.15.94:0/1237954063 shutdown_connections 2024-04-25T16:52:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f303abe4640 1 --2- 172.21.15.94:0/1237954063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3034165b90 0x7f3034167fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f303abe4640 1 --2- 172.21.15.94:0/1237954063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f303415e990 0x7f3034165650 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f303abe4640 1 -- 172.21.15.94:0/1237954063 >> 172.21.15.94:0/1237954063 conn(0x7f30340b47f0 msgr2=0x7f30340b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f303abe4640 1 -- 172.21.15.94:0/1237954063 shutdown_connections 2024-04-25T16:52:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f303abe4640 1 -- 172.21.15.94:0/1237954063 wait complete. 2024-04-25T16:52:22.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f303abe4640 1 Processor -- start 2024-04-25T16:52:22.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.667+0000 7f303abe4640 1 -- start start 2024-04-25T16:52:22.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f303abe4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034070080 0x7f3034164de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f303abe4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f303415e990 0x7f303409a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f303abe4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3034165b90 0x7f30340cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f303abe4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f30340ae6c0 con 0x7f3034070080 2024-04-25T16:52:22.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f303abe4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f303406f460 con 0x7f303415e990 2024-04-25T16:52:22.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f303abe4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3034070490 con 0x7f3034165b90 2024-04-25T16:52:22.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f3038980640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034070080 0x7f3034164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f3038980640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034070080 0x7f3034164de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37720/0 (socket says 172.21.15.94:37720) 2024-04-25T16:52:22.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f3038980640 1 -- 172.21.15.94:0/267479444 learned_addr learned my addr 172.21.15.94:0/267479444 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:22.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f302bfff640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f303415e990 0x7f303409a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f3039181640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3034165b90 0x7f30340cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f3038980640 1 -- 172.21.15.94:0/267479444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3034165b90 msgr2=0x7f30340cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f3038980640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3034165b90 0x7f30340cfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f3038980640 1 -- 172.21.15.94:0/267479444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f303415e990 msgr2=0x7f303409a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f3038980640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f303415e990 0x7f303409a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f3038980640 1 -- 172.21.15.94:0/267479444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30341535e0 con 0x7f3034070080 2024-04-25T16:52:22.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f3039181640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3034165b90 0x7f30340cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:22.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f3038980640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034070080 0x7f3034164de0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f303000c3c0 tx=0x7f303000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:22.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f30297fa640 1 -- 172.21.15.94:0/267479444 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030017070 con 0x7f3034070080 2024-04-25T16:52:22.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f30297fa640 1 -- 172.21.15.94:0/267479444 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30300052f0 con 0x7f3034070080 2024-04-25T16:52:22.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f30297fa640 1 -- 172.21.15.94:0/267479444 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030003860 con 0x7f3034070080 2024-04-25T16:52:22.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f303abe4640 1 -- 172.21.15.94:0/267479444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3034153ae0 con 0x7f3034070080 2024-04-25T16:52:22.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f303abe4640 1 -- 172.21.15.94:0/267479444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30341540a0 con 0x7f3034070080 2024-04-25T16:52:22.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f303abe4640 1 -- 172.21.15.94:0/267479444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3034076830 con 0x7f3034070080 2024-04-25T16:52:22.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f30297fa640 1 -- 172.21.15.94:0/267479444 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f303001a070 con 0x7f3034070080 2024-04-25T16:52:22.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.671+0000 7f30297fa640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f302006f400 0x7f3020071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.675+0000 7f30297fa640 1 -- 172.21.15.94:0/267479444 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f303008e2d0 con 0x7f3034070080 2024-04-25T16:52:22.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.675+0000 7f302bfff640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f302006f400 0x7f3020071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.675+0000 7f302bfff640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f302006f400 0x7f3020071910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f30240ae000 tx=0x7f3024008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:22.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.679+0000 7f30297fa640 1 -- 172.21.15.94:0/267479444 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f303406d610 con 0x7f3034070080 2024-04-25T16:52:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.831+0000 7f303abe4640 1 -- 172.21.15.94:0/267479444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f303415e490 con 0x7f3034070080 2024-04-25T16:52:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.835+0000 7f30297fa640 1 -- 172.21.15.94:0/267479444 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+13176 (secure 0 0 0) 0x7f303005b410 con 0x7f3034070080 2024-04-25T16:52:22.832 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:22.833 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":34,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:52:19.738249+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:52:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.835+0000 7f303abe4640 1 -- 172.21.15.94:0/267479444 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f302006f400 msgr2=0x7f3020071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.835+0000 7f303abe4640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f302006f400 0x7f3020071910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f30240ae000 tx=0x7f3024008040 comp rx=0 tx=0).stop 2024-04-25T16:52:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.835+0000 7f303abe4640 1 -- 172.21.15.94:0/267479444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034070080 msgr2=0x7f3034164de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.835+0000 7f303abe4640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034070080 0x7f3034164de0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f303000c3c0 tx=0x7f303000c880 comp rx=0 tx=0).stop 2024-04-25T16:52:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.835+0000 7f303abe4640 1 -- 172.21.15.94:0/267479444 shutdown_connections 2024-04-25T16:52:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.835+0000 7f303abe4640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f302006f400 0x7f3020071910 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.835+0000 7f303abe4640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3034165b90 0x7f30340cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.835+0000 7f303abe4640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f303415e990 0x7f303409a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.835+0000 7f303abe4640 1 --2- 172.21.15.94:0/267479444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034070080 0x7f3034164de0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.835+0000 7f303abe4640 1 -- 172.21.15.94:0/267479444 >> 172.21.15.94:0/267479444 conn(0x7f30340b47f0 msgr2=0x7f303415f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.835+0000 7f303abe4640 1 -- 172.21.15.94:0/267479444 shutdown_connections 2024-04-25T16:52:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.835+0000 7f303abe4640 1 -- 172.21.15.94:0/267479444 wait complete. 2024-04-25T16:52:22.852 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:52:22.852 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:52:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02c6914640 1 Processor -- start 2024-04-25T16:52:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02c6914640 1 -- start start 2024-04-25T16:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02c6914640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02c00d3450 0x7f02c00d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02c6914640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f02c00d3da0 0x7f02c00b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02c6914640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02c0164510 con 0x7f02c00b7510 2024-04-25T16:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02c6914640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f02c0164a90 con 0x7f02c00d3da0 2024-04-25T16:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02c6914640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f02c00d51b0 con 0x7f02c00d3450 2024-04-25T16:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02bf7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f02c00d3da0 0x7f02c00b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02bf7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f02c00d3da0 0x7f02c00b6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37730/0 (socket says 172.21.15.94:37730) 2024-04-25T16:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02bf7fe640 1 -- 172.21.15.94:0/1619875832 learned_addr learned my addr 172.21.15.94:0/1619875832 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02c4eb1640 1 --1- 172.21.15.94:0/1619875832 >> v1:172.21.15.94:6789/0 conn(0x7f02c00b7510 0x7f02c00d1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45852/0 (socket says 172.21.15.94:45852) 2024-04-25T16:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02bffff640 1 --2- 172.21.15.94:0/1619875832 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02c00d3450 0x7f02c00d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02bf7fe640 1 -- 172.21.15.94:0/1619875832 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02c00d3450 msgr2=0x7f02c00d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02bf7fe640 1 --2- 172.21.15.94:0/1619875832 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02c00d3450 0x7f02c00d3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02bf7fe640 1 -- 172.21.15.94:0/1619875832 >> v1:172.21.15.94:6789/0 conn(0x7f02c00b7510 legacy=0x7f02c00d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02bf7fe640 1 -- 172.21.15.94:0/1619875832 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c01647b0 con 0x7f02c00d3da0 2024-04-25T16:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.995+0000 7f02beffd640 1 -- 172.21.15.94:0/1619875832 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3800051567 0 0) 0x7f02c0164510 con 0x7f02c00b7510 2024-04-25T16:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.999+0000 7f02bf7fe640 1 --2- 172.21.15.94:0/1619875832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c00d3da0 0x7f02c00b6fd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f02c0164c10 tx=0x7f02b001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84af03ed699bbd26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.999+0000 7f02beffd640 1 -- 172.21.15.94:0/1619875832 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b00ab680 con 0x7f02c00d3da0 2024-04-25T16:52:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.999+0000 7f02beffd640 1 -- 172.21.15.94:0/1619875832 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b00b0070 con 0x7f02c00d3da0 2024-04-25T16:52:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.999+0000 7f02beffd640 1 -- 172.21.15.94:0/1619875832 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b00aba40 con 0x7f02c00d3da0 2024-04-25T16:52:22.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.999+0000 7f02c6914640 1 -- 172.21.15.94:0/1619875832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c00d3da0 msgr2=0x7f02c00b6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:22.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.999+0000 7f02c6914640 1 --2- 172.21.15.94:0/1619875832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c00d3da0 0x7f02c00b6fd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f02c0164c10 tx=0x7f02b001c480 comp rx=0 tx=0).stop 2024-04-25T16:52:22.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.999+0000 7f02c6914640 1 -- 172.21.15.94:0/1619875832 shutdown_connections 2024-04-25T16:52:22.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.999+0000 7f02c6914640 1 --2- 172.21.15.94:0/1619875832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c00d3da0 0x7f02c00b6fd0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.999+0000 7f02c6914640 1 --2- 172.21.15.94:0/1619875832 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02c00d3450 0x7f02c00d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:22.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.999+0000 7f02c6914640 1 -- 172.21.15.94:0/1619875832 >> 172.21.15.94:0/1619875832 conn(0x7f02c009e8d0 msgr2=0x7f02c00b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:22.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.999+0000 7f02c6914640 1 -- 172.21.15.94:0/1619875832 shutdown_connections 2024-04-25T16:52:22.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.999+0000 7f02c6914640 1 -- 172.21.15.94:0/1619875832 wait complete. 2024-04-25T16:52:22.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:22.999+0000 7f02c6914640 1 Processor -- start 2024-04-25T16:52:22.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02c6914640 1 -- start start 2024-04-25T16:52:23.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02c6914640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02c00b7510 0x7f02c015f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02c6914640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c00d3450 0x7f02c006c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02c6914640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02c00d3da0 0x7f02c0157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02c6914640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f02c00b7d30 con 0x7f02c00d3450 2024-04-25T16:52:23.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02c6914640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02c00d41b0 con 0x7f02c00d3da0 2024-04-25T16:52:23.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02c6914640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f02c00b8cd0 con 0x7f02c00b7510 2024-04-25T16:52:23.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02bf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c00d3450 0x7f02c006c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:23.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02bf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c00d3450 0x7f02c006c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37732/0 (socket says 172.21.15.94:37732) 2024-04-25T16:52:23.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02bf7fe640 1 -- 172.21.15.94:0/2954578880 learned_addr learned my addr 172.21.15.94:0/2954578880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:23.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02bf7fe640 1 -- 172.21.15.94:0/2954578880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02c00b7510 msgr2=0x7f02c015f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02bf7fe640 1 --2- 172.21.15.94:0/2954578880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02c00b7510 0x7f02c015f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02bf7fe640 1 -- 172.21.15.94:0/2954578880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02c00d3da0 msgr2=0x7f02c0157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02bf7fe640 1 --2- 172.21.15.94:0/2954578880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02c00d3da0 0x7f02c0157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02bf7fe640 1 -- 172.21.15.94:0/2954578880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c006e390 con 0x7f02c00d3450 2024-04-25T16:52:23.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02bf7fe640 1 --2- 172.21.15.94:0/2954578880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c00d3450 0x7f02c006c600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f02b00ad040 tx=0x7f02b0019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:23.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02bcff9640 1 -- 172.21.15.94:0/2954578880 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b00b5810 con 0x7f02c00d3450 2024-04-25T16:52:23.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02bcff9640 1 -- 172.21.15.94:0/2954578880 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b00a7070 con 0x7f02c00d3450 2024-04-25T16:52:23.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02c6914640 1 -- 172.21.15.94:0/2954578880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c006d2f0 con 0x7f02c00d3450 2024-04-25T16:52:23.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02bcff9640 1 -- 172.21.15.94:0/2954578880 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b00b5c10 con 0x7f02c00d3450 2024-04-25T16:52:23.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02c6914640 1 -- 172.21.15.94:0/2954578880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c006d8b0 con 0x7f02c00d3450 2024-04-25T16:52:23.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02c6914640 1 -- 172.21.15.94:0/2954578880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c0076830 con 0x7f02c00d3450 2024-04-25T16:52:23.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.003+0000 7f02bcff9640 1 -- 172.21.15.94:0/2954578880 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f02b00bd020 con 0x7f02c00d3450 2024-04-25T16:52:23.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.007+0000 7f02bcff9640 1 --2- 172.21.15.94:0/2954578880 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f02ac06f400 0x7f02ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.007+0000 7f02bffff640 1 --2- 172.21.15.94:0/2954578880 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f02ac06f400 0x7f02ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:23.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.007+0000 7f02bcff9640 1 -- 172.21.15.94:0/2954578880 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f02b0129a30 con 0x7f02c00d3450 2024-04-25T16:52:23.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.007+0000 7f02bffff640 1 --2- 172.21.15.94:0/2954578880 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f02ac06f400 0x7f02ac071910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f02c00b8c60 tx=0x7f02b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:23.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.011+0000 7f02bcff9640 1 -- 172.21.15.94:0/2954578880 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f02b0016130 con 0x7f02c00d3450 2024-04-25T16:52:23.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.163+0000 7f02c6914640 1 -- 172.21.15.94:0/2954578880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f02c006d080 con 0x7f02c00d3450 2024-04-25T16:52:23.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.163+0000 7f02bcff9640 1 -- 172.21.15.94:0/2954578880 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f02b00f6b70 con 0x7f02c00d3450 2024-04-25T16:52:23.161 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T16:52:23.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.163+0000 7f02c6914640 1 -- 172.21.15.94:0/2954578880 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f02ac06f400 msgr2=0x7f02ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.163+0000 7f02c6914640 1 --2- 172.21.15.94:0/2954578880 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f02ac06f400 0x7f02ac071910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f02c00b8c60 tx=0x7f02b4008040 comp rx=0 tx=0).stop 2024-04-25T16:52:23.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.163+0000 7f02c6914640 1 -- 172.21.15.94:0/2954578880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c00d3450 msgr2=0x7f02c006c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.163+0000 7f02c6914640 1 --2- 172.21.15.94:0/2954578880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c00d3450 0x7f02c006c600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f02b00ad040 tx=0x7f02b0019220 comp rx=0 tx=0).stop 2024-04-25T16:52:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.163+0000 7f02c6914640 1 -- 172.21.15.94:0/2954578880 shutdown_connections 2024-04-25T16:52:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.163+0000 7f02c6914640 1 --2- 172.21.15.94:0/2954578880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02c00d3da0 0x7f02c0157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.163+0000 7f02c6914640 1 --2- 172.21.15.94:0/2954578880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c00d3450 0x7f02c006c600 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.163+0000 7f02c6914640 1 --2- 172.21.15.94:0/2954578880 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f02ac06f400 0x7f02ac071910 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.163+0000 7f02c6914640 1 --2- 172.21.15.94:0/2954578880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02c00b7510 0x7f02c015f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.163+0000 7f02c6914640 1 -- 172.21.15.94:0/2954578880 >> 172.21.15.94:0/2954578880 conn(0x7f02c009e8d0 msgr2=0x7f02c00b6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.167+0000 7f02c6914640 1 -- 172.21.15.94:0/2954578880 shutdown_connections 2024-04-25T16:52:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.167+0000 7f02c6914640 1 -- 172.21.15.94:0/2954578880 wait complete. 2024-04-25T16:52:23.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T16:52:23.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b829e640 1 Processor -- start 2024-04-25T16:52:23.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b829e640 1 -- start start 2024-04-25T16:52:23.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b829e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f68b0070180 0x7f68b015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b829e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68b0160bf0 0x7f68b006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b829e640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68b015d620 con 0x7f68b0161000 2024-04-25T16:52:23.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b829e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f68b015dec0 con 0x7f68b0070180 2024-04-25T16:52:23.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b829e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f68b0160760 con 0x7f68b0160bf0 2024-04-25T16:52:23.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b603a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f68b0070180 0x7f68b015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:23.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b603a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f68b0070180 0x7f68b015cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37742/0 (socket says 172.21.15.94:37742) 2024-04-25T16:52:23.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b683b640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f68b0161000 0x7f68b00af2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45864/0 (socket says 172.21.15.94:45864) 2024-04-25T16:52:23.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b603a640 1 -- 172.21.15.94:0/2392590695 learned_addr learned my addr 172.21.15.94:0/2392590695 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:23.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b5839640 1 --2- 172.21.15.94:0/2392590695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68b0160bf0 0x7f68b006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:23.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b603a640 1 -- 172.21.15.94:0/2392590695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68b0160bf0 msgr2=0x7f68b006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:23.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b603a640 1 --2- 172.21.15.94:0/2392590695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68b0160bf0 0x7f68b006d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b603a640 1 -- 172.21.15.94:0/2392590695 >> v1:172.21.15.94:6789/0 conn(0x7f68b0161000 legacy=0x7f68b00af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b5038640 1 -- 172.21.15.94:0/2392590695 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3773134532 0 0) 0x7f68b015d620 con 0x7f68b0161000 2024-04-25T16:52:23.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.323+0000 7f68b603a640 1 -- 172.21.15.94:0/2392590695 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b015d900 con 0x7f68b0070180 2024-04-25T16:52:23.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.327+0000 7f68b603a640 1 --2- 172.21.15.94:0/2392590695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0070180 0x7f68b015cf40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009870 tx=0x7f68ac01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3374f61bbc6eef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:23.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.327+0000 7f68b5038640 1 -- 172.21.15.94:0/2392590695 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0a7070 con 0x7f68b0070180 2024-04-25T16:52:23.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.327+0000 7f68b5038640 1 -- 172.21.15.94:0/2392590695 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68ac0b0070 con 0x7f68b0070180 2024-04-25T16:52:23.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.327+0000 7f68b5038640 1 -- 172.21.15.94:0/2392590695 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0ab890 con 0x7f68b0070180 2024-04-25T16:52:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.327+0000 7f68b829e640 1 -- 172.21.15.94:0/2392590695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0070180 msgr2=0x7f68b015cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.327+0000 7f68b829e640 1 --2- 172.21.15.94:0/2392590695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0070180 0x7f68b015cf40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009870 tx=0x7f68ac01c560 comp rx=0 tx=0).stop 2024-04-25T16:52:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.327+0000 7f68b829e640 1 -- 172.21.15.94:0/2392590695 shutdown_connections 2024-04-25T16:52:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.327+0000 7f68b829e640 1 --2- 172.21.15.94:0/2392590695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68b0160bf0 0x7f68b006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.327+0000 7f68b829e640 1 --2- 172.21.15.94:0/2392590695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0070180 0x7f68b015cf40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.327+0000 7f68b829e640 1 -- 172.21.15.94:0/2392590695 >> 172.21.15.94:0/2392590695 conn(0x7f68b00b47f0 msgr2=0x7f68b00b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:23.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.327+0000 7f68b829e640 1 -- 172.21.15.94:0/2392590695 shutdown_connections 2024-04-25T16:52:23.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.327+0000 7f68b829e640 1 -- 172.21.15.94:0/2392590695 wait complete. 2024-04-25T16:52:23.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b829e640 1 Processor -- start 2024-04-25T16:52:23.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b829e640 1 -- start start 2024-04-25T16:52:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b829e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68b0070180 0x7f68b01524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b829e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0160bf0 0x7f68b0152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b829e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b0161000 0x7f68b00cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b829e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68b006c990 con 0x7f68b0160bf0 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b829e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68b006c810 con 0x7f68b0161000 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b829e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f68b006cb10 con 0x7f68b0070180 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b5839640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0160bf0 0x7f68b0152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b5839640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0160bf0 0x7f68b0152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37750/0 (socket says 172.21.15.94:37750) 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b5839640 1 -- 172.21.15.94:0/2370481141 learned_addr learned my addr 172.21.15.94:0/2370481141 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b683b640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b0161000 0x7f68b00cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b603a640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68b0070180 0x7f68b01524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b5839640 1 -- 172.21.15.94:0/2370481141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68b0070180 msgr2=0x7f68b01524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b5839640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68b0070180 0x7f68b01524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b5839640 1 -- 172.21.15.94:0/2370481141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b0161000 msgr2=0x7f68b00cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b5839640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b0161000 0x7f68b00cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b5839640 1 -- 172.21.15.94:0/2370481141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b0150e70 con 0x7f68b0160bf0 2024-04-25T16:52:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b603a640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68b0070180 0x7f68b01524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:23.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b5839640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0160bf0 0x7f68b0152a10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f68a000d620 tx=0x7f68a000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:23.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68a6ffd640 1 -- 172.21.15.94:0/2370481141 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a000b620 con 0x7f68b0160bf0 2024-04-25T16:52:23.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68a6ffd640 1 -- 172.21.15.94:0/2370481141 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68a0019070 con 0x7f68b0160bf0 2024-04-25T16:52:23.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b829e640 1 -- 172.21.15.94:0/2370481141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b0153080 con 0x7f68b0160bf0 2024-04-25T16:52:23.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68a6ffd640 1 -- 172.21.15.94:0/2370481141 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a000b9e0 con 0x7f68b0160bf0 2024-04-25T16:52:23.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b829e640 1 -- 172.21.15.94:0/2370481141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b0151110 con 0x7f68b0160bf0 2024-04-25T16:52:23.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.331+0000 7f68b829e640 1 -- 172.21.15.94:0/2370481141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b0076830 con 0x7f68b0160bf0 2024-04-25T16:52:23.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.335+0000 7f68a6ffd640 1 -- 172.21.15.94:0/2370481141 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f68a000bb80 con 0x7f68b0160bf0 2024-04-25T16:52:23.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.335+0000 7f68a6ffd640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f689406f400 0x7f6894071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.335+0000 7f68a6ffd640 1 -- 172.21.15.94:0/2370481141 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f68a008f4d0 con 0x7f68b0160bf0 2024-04-25T16:52:23.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.335+0000 7f68b603a640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f689406f400 0x7f6894071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:23.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.335+0000 7f68b603a640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f689406f400 0x7f6894071910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0af000 tx=0x7f68ac0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:23.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.339+0000 7f68a6ffd640 1 -- 172.21.15.94:0/2370481141 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f68a005c610 con 0x7f68b0160bf0 2024-04-25T16:52:23.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.487+0000 7f68b829e640 1 -- 172.21.15.94:0/2370481141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f68b00b3a90 con 0x7f68b0160bf0 2024-04-25T16:52:23.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.491+0000 7f68a6ffd640 1 -- 172.21.15.94:0/2370481141 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7f68a005bf10 con 0x7f68b0160bf0 2024-04-25T16:52:23.488 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:23.488 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-04-25T16:52:23.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T16:52:23.488 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-04-25T16:52:23.488 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-04-25T16:52:23.489 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:23.489 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-04-25T16:52:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.491+0000 7f68b829e640 1 -- 172.21.15.94:0/2370481141 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f689406f400 msgr2=0x7f6894071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.491+0000 7f68b829e640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f689406f400 0x7f6894071910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0af000 tx=0x7f68ac0a7000 comp rx=0 tx=0).stop 2024-04-25T16:52:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.491+0000 7f68b829e640 1 -- 172.21.15.94:0/2370481141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0160bf0 msgr2=0x7f68b0152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.491+0000 7f68b829e640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0160bf0 0x7f68b0152a10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f68a000d620 tx=0x7f68a000dae0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.491+0000 7f68b829e640 1 -- 172.21.15.94:0/2370481141 shutdown_connections 2024-04-25T16:52:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.491+0000 7f68b829e640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b0161000 0x7f68b00cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.491+0000 7f68b829e640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0160bf0 0x7f68b0152a10 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.491+0000 7f68b829e640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f689406f400 0x7f6894071910 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.491+0000 7f68b829e640 1 --2- 172.21.15.94:0/2370481141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68b0070180 0x7f68b01524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.491+0000 7f68b829e640 1 -- 172.21.15.94:0/2370481141 >> 172.21.15.94:0/2370481141 conn(0x7f68b00b47f0 msgr2=0x7f68b0068690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.491+0000 7f68b829e640 1 -- 172.21.15.94:0/2370481141 shutdown_connections 2024-04-25T16:52:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.491+0000 7f68b829e640 1 -- 172.21.15.94:0/2370481141 wait complete. 2024-04-25T16:52:23.508 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T16:52:23.508 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T16:52:23.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.655+0000 7f3ba0474640 1 Processor -- start 2024-04-25T16:52:23.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.655+0000 7f3ba0474640 1 -- start start 2024-04-25T16:52:23.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3ba0474640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3b980b7150 0x7f3b980d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3ba0474640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b980d4ce0 con 0x7f3b980d48d0 2024-04-25T16:52:23.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3ba0474640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b980b7730 con 0x7f3b980d44c0 2024-04-25T16:52:23.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3ba0474640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f3b980b7fd0 con 0x7f3b980b7150 2024-04-25T16:52:23.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3b9e210640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3b980b7150 0x7f3b980d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:23.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3b9ea11640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f3b980d48d0 0x7f3b980d5640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45876/0 (socket says 172.21.15.94:45876) 2024-04-25T16:52:23.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3b9ea11640 1 -- 172.21.15.94:0/457214272 learned_addr learned my addr 172.21.15.94:0/457214272 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:23.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3b9e210640 1 -- 172.21.15.94:0/457214272 >> v1:172.21.15.138:6789/0 conn(0x7f3b980d44c0 legacy=0x7f3b98163ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:23.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3b9e210640 1 -- 172.21.15.94:0/457214272 >> v1:172.21.15.94:6789/0 conn(0x7f3b980d48d0 legacy=0x7f3b980d5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3b9e210640 1 -- 172.21.15.94:0/457214272 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b980b7a10 con 0x7f3b980b7150 2024-04-25T16:52:23.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3b9d20e640 1 -- 172.21.15.94:0/457214272 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3090151873 0 0) 0x7f3b980d4ce0 con 0x7f3b980d48d0 2024-04-25T16:52:23.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3b9e210640 1 --2- 172.21.15.94:0/457214272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b980b7150 0x7f3b980d1a90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3b940097c0 tx=0x7f3b9401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e441284954f89b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:23.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3b9d20e640 1 -- 172.21.15.94:0/457214272 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b940b0070 con 0x7f3b980b7150 2024-04-25T16:52:23.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3b9d20e640 1 -- 172.21.15.94:0/457214272 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b940ab600 con 0x7f3b980b7150 2024-04-25T16:52:23.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.659+0000 7f3b9d20e640 1 -- 172.21.15.94:0/457214272 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b940ab9d0 con 0x7f3b980b7150 2024-04-25T16:52:23.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 -- 172.21.15.94:0/457214272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b980b7150 msgr2=0x7f3b980d1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 --2- 172.21.15.94:0/457214272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b980b7150 0x7f3b980d1a90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3b940097c0 tx=0x7f3b9401c3d0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 -- 172.21.15.94:0/457214272 shutdown_connections 2024-04-25T16:52:23.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 --2- 172.21.15.94:0/457214272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b980b7150 0x7f3b980d1a90 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 -- 172.21.15.94:0/457214272 >> 172.21.15.94:0/457214272 conn(0x7f3b9809e940 msgr2=0x7f3b9808d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:23.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 -- 172.21.15.94:0/457214272 shutdown_connections 2024-04-25T16:52:23.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 -- 172.21.15.94:0/457214272 wait complete. 2024-04-25T16:52:23.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 Processor -- start 2024-04-25T16:52:23.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 -- start start 2024-04-25T16:52:23.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b980b7150 0x7f3b98150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b980d44c0 0x7f3b980b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b980d48d0 0x7f3b981fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9816e450 con 0x7f3b980b7150 2024-04-25T16:52:23.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b98164840 con 0x7f3b980d48d0 2024-04-25T16:52:23.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3ba0474640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9816e5d0 con 0x7f3b980d44c0 2024-04-25T16:52:23.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3b9e210640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b980b7150 0x7f3b98150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:23.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3b9e210640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b980b7150 0x7f3b98150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37772/0 (socket says 172.21.15.94:37772) 2024-04-25T16:52:23.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.663+0000 7f3b9e210640 1 -- 172.21.15.94:0/3286990279 learned_addr learned my addr 172.21.15.94:0/3286990279 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:23.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b9e210640 1 -- 172.21.15.94:0/3286990279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b980d44c0 msgr2=0x7f3b980b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b9ea11640 1 --2- 172.21.15.94:0/3286990279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b980d48d0 0x7f3b981fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:23.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b9e210640 1 --2- 172.21.15.94:0/3286990279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b980d44c0 0x7f3b980b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b9e210640 1 -- 172.21.15.94:0/3286990279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b980d48d0 msgr2=0x7f3b981fcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:23.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b9e210640 1 --2- 172.21.15.94:0/3286990279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b980d48d0 0x7f3b981fcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:23.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b9e210640 1 -- 172.21.15.94:0/3286990279 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b980b4260 con 0x7f3b980b7150 2024-04-25T16:52:23.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b9e210640 1 --2- 172.21.15.94:0/3286990279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b980b7150 0x7f3b98150630 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3b940ae040 tx=0x7f3b9401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:23.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b8effd640 1 -- 172.21.15.94:0/3286990279 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b940b0050 con 0x7f3b980b7150 2024-04-25T16:52:23.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b8effd640 1 -- 172.21.15.94:0/3286990279 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b940bc070 con 0x7f3b980b7150 2024-04-25T16:52:23.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3ba0474640 1 -- 172.21.15.94:0/3286990279 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b980b4500 con 0x7f3b980b7150 2024-04-25T16:52:23.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b8effd640 1 -- 172.21.15.94:0/3286990279 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b940b5870 con 0x7f3b980b7150 2024-04-25T16:52:23.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3ba0474640 1 -- 172.21.15.94:0/3286990279 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b98156ef0 con 0x7f3b980b7150 2024-04-25T16:52:23.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3ba0474640 1 -- 172.21.15.94:0/3286990279 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b9806ff60 con 0x7f3b980b7150 2024-04-25T16:52:23.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b8effd640 1 -- 172.21.15.94:0/3286990279 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3b940b5a10 con 0x7f3b980b7150 2024-04-25T16:52:23.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b8effd640 1 --2- 172.21.15.94:0/3286990279 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3b7c06f400 0x7f3b7c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:23.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b8effd640 1 -- 172.21.15.94:0/3286990279 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f3b9412a500 con 0x7f3b980b7150 2024-04-25T16:52:23.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b9da0f640 1 --2- 172.21.15.94:0/3286990279 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3b7c06f400 0x7f3b7c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:23.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.667+0000 7f3b9da0f640 1 --2- 172.21.15.94:0/3286990279 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3b7c06f400 0x7f3b7c071910 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3b9815f600 tx=0x7f3b88006bd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:23.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.671+0000 7f3b8effd640 1 -- 172.21.15.94:0/3286990279 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3b940c6020 con 0x7f3b980b7150 2024-04-25T16:52:23.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:23.827+0000 7f3ba0474640 1 -- 172.21.15.94:0/3286990279 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3b98150b70 con 0x7f3b980b7150 2024-04-25T16:52:24.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.747+0000 7f3b8effd640 1 -- 172.21.15.94:0/3286990279 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7f3b94016050 con 0x7f3b980b7150 2024-04-25T16:52:24.746 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-04-25T16:52:24.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.751+0000 7f3ba0474640 1 -- 172.21.15.94:0/3286990279 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3b7c06f400 msgr2=0x7f3b7c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:24.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.751+0000 7f3ba0474640 1 --2- 172.21.15.94:0/3286990279 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3b7c06f400 0x7f3b7c071910 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3b9815f600 tx=0x7f3b88006bd0 comp rx=0 tx=0).stop 2024-04-25T16:52:24.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.751+0000 7f3ba0474640 1 -- 172.21.15.94:0/3286990279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b980b7150 msgr2=0x7f3b98150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:24.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.751+0000 7f3ba0474640 1 --2- 172.21.15.94:0/3286990279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b980b7150 0x7f3b98150630 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3b940ae040 tx=0x7f3b9401ac10 comp rx=0 tx=0).stop 2024-04-25T16:52:24.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.751+0000 7f3ba0474640 1 -- 172.21.15.94:0/3286990279 shutdown_connections 2024-04-25T16:52:24.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.751+0000 7f3ba0474640 1 --2- 172.21.15.94:0/3286990279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b980d48d0 0x7f3b981fcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:24.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.751+0000 7f3ba0474640 1 --2- 172.21.15.94:0/3286990279 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3b7c06f400 0x7f3b7c071910 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:24.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.751+0000 7f3ba0474640 1 --2- 172.21.15.94:0/3286990279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b980d44c0 0x7f3b980b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:24.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.751+0000 7f3ba0474640 1 --2- 172.21.15.94:0/3286990279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b980b7150 0x7f3b98150630 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:24.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.751+0000 7f3ba0474640 1 -- 172.21.15.94:0/3286990279 >> 172.21.15.94:0/3286990279 conn(0x7f3b9809e940 msgr2=0x7f3b980b6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:24.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.751+0000 7f3ba0474640 1 -- 172.21.15.94:0/3286990279 shutdown_connections 2024-04-25T16:52:24.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.751+0000 7f3ba0474640 1 -- 172.21.15.94:0/3286990279 wait complete. 2024-04-25T16:52:24.777 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T16:52:24.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb7548aa640 1 Processor -- start 2024-04-25T16:52:24.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb7548aa640 1 -- start start 2024-04-25T16:52:24.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb7548aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b0420 0x7fb75006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:24.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb7548aa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb75006f2a0 0x7fb75006c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:24.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb7548aa640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb750070630 con 0x7fb75006cb40 2024-04-25T16:52:24.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb7548aa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb7500aeaa0 con 0x7fb75006f2a0 2024-04-25T16:52:24.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb7548aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb7500703e0 con 0x7fb7500b0420 2024-04-25T16:52:24.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74ed9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fb75006cb40 0x7fb75015dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45880/0 (socket says 172.21.15.94:45880) 2024-04-25T16:52:24.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74ed9d640 1 -- 172.21.15.94:0/313399607 learned_addr learned my addr 172.21.15.94:0/313399607 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:24.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74e59c640 1 --2- 172.21.15.94:0/313399607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b0420 0x7fb75006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:24.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74dd9b640 1 --2- 172.21.15.94:0/313399607 >> v2:172.21.15.138:3300/0 conn(0x7fb75006f2a0 0x7fb75006c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:24.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2968284312 0 0) 0x7fb750070630 con 0x7fb75006cb40 2024-04-25T16:52:24.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb738003530 con 0x7fb75006cb40 2024-04-25T16:52:24.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2900523615 0 0) 0x7fb738003530 con 0x7fb75006cb40 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b0420 msgr2=0x7fb75006ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 --2- 172.21.15.94:0/313399607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b0420 0x7fb75006ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 >> v2:172.21.15.138:3300/0 conn(0x7fb75006f2a0 msgr2=0x7fb75006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 --2- 172.21.15.94:0/313399607 >> v2:172.21.15.138:3300/0 conn(0x7fb75006f2a0 0x7fb75006c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75006f6b0 con 0x7fb75006cb40 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fb748001a00 con 0x7fb75006cb40 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 >> v1:172.21.15.94:6789/0 conn(0x7fb75006cb40 legacy=0x7fb75015dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 --2- 172.21.15.94:0/313399607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7380039c0 0x7fb738003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 --2- 172.21.15.94:0/313399607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b0420 0x7fb738004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 --2- 172.21.15.94:0/313399607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb738004d50 0x7fb738005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb7500aeaa0 con 0x7fb738004d50 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7500703e0 con 0x7fb7380039c0 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb73800ba00 con 0x7fb7500b0420 2024-04-25T16:52:24.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74ed9d640 1 --2- 172.21.15.94:0/313399607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb738004d50 0x7fb738005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:24.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74dd9b640 1 --2- 172.21.15.94:0/313399607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b0420 0x7fb738004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:24.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.939+0000 7fb74e59c640 1 --2- 172.21.15.94:0/313399607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7380039c0 0x7fb738003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:24.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb74ed9d640 1 -- 172.21.15.94:0/313399607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b0420 msgr2=0x7fb738004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:24.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb74ed9d640 1 --2- 172.21.15.94:0/313399607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b0420 0x7fb738004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:24.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb74ed9d640 1 -- 172.21.15.94:0/313399607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7380039c0 msgr2=0x7fb738003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:24.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb74ed9d640 1 --2- 172.21.15.94:0/313399607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7380039c0 0x7fb738003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:24.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb74ed9d640 1 -- 172.21.15.94:0/313399607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb738003530 con 0x7fb738004d50 2024-04-25T16:52:24.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb74dd9b640 1 --2- 172.21.15.94:0/313399607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7500b0420 0x7fb738004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:24.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb74ed9d640 1 --2- 172.21.15.94:0/313399607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb738004d50 0x7fb738005160 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb7500726f0 tx=0x7fb74801d180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f8b975cace4d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:24.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74800a4c0 con 0x7fb738004d50 2024-04-25T16:52:24.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7480ad630 con 0x7fb738004d50 2024-04-25T16:52:24.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb74d59a640 1 -- 172.21.15.94:0/313399607 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7480ada20 con 0x7fb738004d50 2024-04-25T16:52:24.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb7548aa640 1 -- 172.21.15.94:0/313399607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb738004d50 msgr2=0x7fb738005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:24.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb7548aa640 1 --2- 172.21.15.94:0/313399607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb738004d50 0x7fb738005160 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb7500726f0 tx=0x7fb74801d180 comp rx=0 tx=0).stop 2024-04-25T16:52:24.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb74ed9d640 1 -- 172.21.15.94:0/313399607 reap_dead start 2024-04-25T16:52:24.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb7548aa640 1 -- 172.21.15.94:0/313399607 shutdown_connections 2024-04-25T16:52:24.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb7548aa640 1 -- 172.21.15.94:0/313399607 >> 172.21.15.94:0/313399607 conn(0x7fb7500b47f0 msgr2=0x7fb7500b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:24.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb7548aa640 1 -- 172.21.15.94:0/313399607 shutdown_connections 2024-04-25T16:52:24.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.943+0000 7fb7548aa640 1 -- 172.21.15.94:0/313399607 wait complete. 2024-04-25T16:52:24.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb7548aa640 1 Processor -- start 2024-04-25T16:52:24.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb7548aa640 1 -- start start 2024-04-25T16:52:24.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb7548aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7501556e0 0x7fb7501f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:24.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb7548aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7500cee60 0x7fb7500cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:24.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb7548aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7501f78f0 0x7fb7501f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:24.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb7548aa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb7500ae450 con 0x7fb7500cee60 2024-04-25T16:52:24.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb7548aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb75015f340 con 0x7fb7501556e0 2024-04-25T16:52:24.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb7548aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb7500ae730 con 0x7fb7501f78f0 2024-04-25T16:52:24.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb74dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7500cee60 0x7fb7500cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb74dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7500cee60 0x7fb7500cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37792/0 (socket says 172.21.15.94:37792) 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb74dd9b640 1 -- 172.21.15.94:0/3102163151 learned_addr learned my addr 172.21.15.94:0/3102163151 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb74ed9d640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7501f78f0 0x7fb7501f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb74e59c640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7501556e0 0x7fb7501f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb74dd9b640 1 -- 172.21.15.94:0/3102163151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7501f78f0 msgr2=0x7fb7501f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb74dd9b640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7501f78f0 0x7fb7501f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb74dd9b640 1 -- 172.21.15.94:0/3102163151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7501556e0 msgr2=0x7fb7501f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb74dd9b640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7501556e0 0x7fb7501f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb74dd9b640 1 -- 172.21.15.94:0/3102163151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7500cf7b0 con 0x7fb7500cee60 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb74e59c640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7501556e0 0x7fb7501f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb74dd9b640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7500cee60 0x7fb7500cf270 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb744011540 tx=0x7fb744011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb73effd640 1 -- 172.21.15.94:0/3102163151 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744004220 con 0x7fb7500cee60 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb73effd640 1 -- 172.21.15.94:0/3102163151 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7440043c0 con 0x7fb7500cee60 2024-04-25T16:52:24.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb7548aa640 1 -- 172.21.15.94:0/3102163151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7501f8620 con 0x7fb7500cee60 2024-04-25T16:52:24.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb73effd640 1 -- 172.21.15.94:0/3102163151 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744015890 con 0x7fb7500cee60 2024-04-25T16:52:24.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.947+0000 7fb7548aa640 1 -- 172.21.15.94:0/3102163151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb750154ee0 con 0x7fb7500cee60 2024-04-25T16:52:24.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.951+0000 7fb7548aa640 1 -- 172.21.15.94:0/3102163151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7501549f0 con 0x7fb7500cee60 2024-04-25T16:52:24.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.951+0000 7fb73effd640 1 -- 172.21.15.94:0/3102163151 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb744015a30 con 0x7fb7500cee60 2024-04-25T16:52:24.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.951+0000 7fb73effd640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb73006f400 0x7fb730071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:24.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.951+0000 7fb73effd640 1 -- 172.21.15.94:0/3102163151 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7fb74408fbb0 con 0x7fb7500cee60 2024-04-25T16:52:24.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.951+0000 7fb74e59c640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb73006f400 0x7fb730071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:24.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.951+0000 7fb74e59c640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb73006f400 0x7fb730071910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb7500d0460 tx=0x7fb740008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:24.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:24.955+0000 7fb73effd640 1 -- 172.21.15.94:0/3102163151 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb7500cf7b0 con 0x7fb7500cee60 2024-04-25T16:52:25.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.115+0000 7fb7548aa640 1 -- 172.21.15.94:0/3102163151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb75006c690 con 0x7fb7500cee60 2024-04-25T16:52:25.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.759+0000 7fb73effd640 1 -- 172.21.15.94:0/3102163151 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7fb744019030 con 0x7fb7500cee60 2024-04-25T16:52:25.758 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-04-25T16:52:25.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.763+0000 7fb7548aa640 1 -- 172.21.15.94:0/3102163151 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb73006f400 msgr2=0x7fb730071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:25.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.763+0000 7fb7548aa640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb73006f400 0x7fb730071910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb7500d0460 tx=0x7fb740008040 comp rx=0 tx=0).stop 2024-04-25T16:52:25.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.763+0000 7fb7548aa640 1 -- 172.21.15.94:0/3102163151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7500cee60 msgr2=0x7fb7500cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:25.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.763+0000 7fb7548aa640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7500cee60 0x7fb7500cf270 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb744011540 tx=0x7fb744011a00 comp rx=0 tx=0).stop 2024-04-25T16:52:25.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.763+0000 7fb7548aa640 1 -- 172.21.15.94:0/3102163151 shutdown_connections 2024-04-25T16:52:25.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.763+0000 7fb7548aa640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fb73006f400 0x7fb730071910 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:25.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.763+0000 7fb7548aa640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7501f78f0 0x7fb7501f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:25.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.763+0000 7fb7548aa640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7500cee60 0x7fb7500cf270 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:25.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.763+0000 7fb7548aa640 1 --2- 172.21.15.94:0/3102163151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7501556e0 0x7fb7501f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:25.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.763+0000 7fb7548aa640 1 -- 172.21.15.94:0/3102163151 >> 172.21.15.94:0/3102163151 conn(0x7fb7500b47f0 msgr2=0x7fb75015dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:25.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.763+0000 7fb7548aa640 1 -- 172.21.15.94:0/3102163151 shutdown_connections 2024-04-25T16:52:25.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.763+0000 7fb7548aa640 1 -- 172.21.15.94:0/3102163151 wait complete. 2024-04-25T16:52:25.801 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T16:52:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd725852640 1 Processor -- start 2024-04-25T16:52:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd725852640 1 -- start start 2024-04-25T16:52:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd725852640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd72006f470 0x7fd72015dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd725852640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd72015e160 0x7fd72015e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd725852640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd720070630 con 0x7fd7200b0420 2024-04-25T16:52:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd725852640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd7200aeaa0 con 0x7fd72006f470 2024-04-25T16:52:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd725852640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd7200703e0 con 0x7fd72015e160 2024-04-25T16:52:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71effd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd7200b0420 0x7fd72006ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45882/0 (socket says 172.21.15.94:45882) 2024-04-25T16:52:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71effd640 1 -- 172.21.15.94:0/3913822812 learned_addr learned my addr 172.21.15.94:0/3913822812 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71e7fc640 1 --2- 172.21.15.94:0/3913822812 >> v2:172.21.15.138:3300/0 conn(0x7fd72006f470 0x7fd72015dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:25.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71f7fe640 1 --2- 172.21.15.94:0/3913822812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd72015e160 0x7fd72015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:25.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2071464726 0 0) 0x7fd720070630 con 0x7fd7200b0420 2024-04-25T16:52:25.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd708003530 con 0x7fd7200b0420 2024-04-25T16:52:25.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3080189437 0 0) 0x7fd708003530 con 0x7fd7200b0420 2024-04-25T16:52:25.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd72015e160 msgr2=0x7fd72015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 --2- 172.21.15.94:0/3913822812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd72015e160 0x7fd72015e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 >> v2:172.21.15.138:3300/0 conn(0x7fd72006f470 msgr2=0x7fd72015dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 --2- 172.21.15.94:0/3913822812 >> v2:172.21.15.138:3300/0 conn(0x7fd72006f470 0x7fd72015dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7200ae170 con 0x7fd7200b0420 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fd710001680 con 0x7fd7200b0420 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 >> v1:172.21.15.94:6789/0 conn(0x7fd7200b0420 legacy=0x7fd72006ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 --2- 172.21.15.94:0/3913822812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7080039c0 0x7fd708003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 --2- 172.21.15.94:0/3913822812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd72015e160 0x7fd708004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 --2- 172.21.15.94:0/3913822812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd708004d50 0x7fd708005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd7200aeaa0 con 0x7fd7080039c0 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd7200703e0 con 0x7fd708004d50 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd70800b9d0 con 0x7fd72015e160 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71effd640 1 --2- 172.21.15.94:0/3913822812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7080039c0 0x7fd708003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:25.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71e7fc640 1 --2- 172.21.15.94:0/3913822812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd72015e160 0x7fd708004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:25.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71f7fe640 1 --2- 172.21.15.94:0/3913822812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd708004d50 0x7fd708005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:25.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71effd640 1 -- 172.21.15.94:0/3913822812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd72015e160 msgr2=0x7fd708004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:25.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71effd640 1 --2- 172.21.15.94:0/3913822812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd72015e160 0x7fd708004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:25.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71effd640 1 -- 172.21.15.94:0/3913822812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd708004d50 msgr2=0x7fd708005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:25.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.959+0000 7fd71effd640 1 --2- 172.21.15.94:0/3913822812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd708004d50 0x7fd708005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:25.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.963+0000 7fd71effd640 1 -- 172.21.15.94:0/3913822812 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd708003530 con 0x7fd7080039c0 2024-04-25T16:52:25.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.963+0000 7fd71f7fe640 1 --2- 172.21.15.94:0/3913822812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd708004d50 0x7fd708005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:25.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.963+0000 7fd71effd640 1 --2- 172.21.15.94:0/3913822812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7080039c0 0x7fd708003e30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd71000c210 tx=0x7fd71001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b47c5b2dcea7ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:25.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.963+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710007d60 con 0x7fd7080039c0 2024-04-25T16:52:25.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.963+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7100a9020 con 0x7fd7080039c0 2024-04-25T16:52:25.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.963+0000 7fd71dffb640 1 -- 172.21.15.94:0/3913822812 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7100a5b30 con 0x7fd7080039c0 2024-04-25T16:52:25.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.963+0000 7fd725852640 1 -- 172.21.15.94:0/3913822812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7080039c0 msgr2=0x7fd708003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:25.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.963+0000 7fd725852640 1 --2- 172.21.15.94:0/3913822812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7080039c0 0x7fd708003e30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd71000c210 tx=0x7fd71001cc30 comp rx=0 tx=0).stop 2024-04-25T16:52:25.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.963+0000 7fd71f7fe640 1 -- 172.21.15.94:0/3913822812 reap_dead start 2024-04-25T16:52:25.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.963+0000 7fd725852640 1 -- 172.21.15.94:0/3913822812 shutdown_connections 2024-04-25T16:52:25.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.963+0000 7fd725852640 1 -- 172.21.15.94:0/3913822812 >> 172.21.15.94:0/3913822812 conn(0x7fd7200b47f0 msgr2=0x7fd7200b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:25.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.963+0000 7fd725852640 1 -- 172.21.15.94:0/3913822812 shutdown_connections 2024-04-25T16:52:25.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.963+0000 7fd725852640 1 -- 172.21.15.94:0/3913822812 wait complete. 2024-04-25T16:52:25.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd725852640 1 Processor -- start 2024-04-25T16:52:25.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd725852640 1 -- start start 2024-04-25T16:52:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd725852640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7080039c0 0x7fd7200d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd725852640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7201f6c70 0x7fd72014f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd725852640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd720152350 0x7fd720152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd725852640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd7200b09b0 con 0x7fd7080039c0 2024-04-25T16:52:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd725852640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd7200b0830 con 0x7fd7201f6c70 2024-04-25T16:52:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd725852640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd72015ed80 con 0x7fd720152350 2024-04-25T16:52:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd71effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7080039c0 0x7fd7200d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd71effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7080039c0 0x7fd7200d1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37812/0 (socket says 172.21.15.94:37812) 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd71effd640 1 -- 172.21.15.94:0/1783889726 learned_addr learned my addr 172.21.15.94:0/1783889726 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd71e7fc640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7201f6c70 0x7fd72014f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd71f7fe640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd720152350 0x7fd720152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd71effd640 1 -- 172.21.15.94:0/1783889726 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd720152350 msgr2=0x7fd720152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd71effd640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd720152350 0x7fd720152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd71effd640 1 -- 172.21.15.94:0/1783889726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7201f6c70 msgr2=0x7fd72014f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd71effd640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7201f6c70 0x7fd72014f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd71effd640 1 -- 172.21.15.94:0/1783889726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd720152ca0 con 0x7fd7080039c0 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd71e7fc640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7201f6c70 0x7fd72014f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd71effd640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7080039c0 0x7fd7200d1a30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd710007040 tx=0x7fd708005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd707fff640 1 -- 172.21.15.94:0/1783889726 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710001ca0 con 0x7fd7080039c0 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd707fff640 1 -- 172.21.15.94:0/1783889726 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7100a5b10 con 0x7fd7080039c0 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd725852640 1 -- 172.21.15.94:0/1783889726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72009b930 con 0x7fd7080039c0 2024-04-25T16:52:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd707fff640 1 -- 172.21.15.94:0/1783889726 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7100047f0 con 0x7fd7080039c0 2024-04-25T16:52:25.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.967+0000 7fd725852640 1 -- 172.21.15.94:0/1783889726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72009bef0 con 0x7fd7080039c0 2024-04-25T16:52:25.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.971+0000 7fd725852640 1 -- 172.21.15.94:0/1783889726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd720076830 con 0x7fd7080039c0 2024-04-25T16:52:25.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.971+0000 7fd707fff640 1 -- 172.21.15.94:0/1783889726 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd710004990 con 0x7fd7080039c0 2024-04-25T16:52:25.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.971+0000 7fd707fff640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd70c06f430 0x7fd70c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:25.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.971+0000 7fd707fff640 1 -- 172.21.15.94:0/1783889726 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7fd7101295e0 con 0x7fd7080039c0 2024-04-25T16:52:25.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.971+0000 7fd71e7fc640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd70c06f430 0x7fd70c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:25.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.971+0000 7fd71e7fc640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd70c06f430 0x7fd70c071940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd714009160 tx=0x7fd714008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:25.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:25.975+0000 7fd707fff640 1 -- 172.21.15.94:0/1783889726 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd7100c9080 con 0x7fd7080039c0 2024-04-25T16:52:26.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.171+0000 7fd725852640 1 -- 172.21.15.94:0/1783889726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd72006fb20 con 0x7fd7080039c0 2024-04-25T16:52:26.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.803+0000 7fd707fff640 1 -- 172.21.15.94:0/1783889726 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd720152ca0 con 0x7fd7080039c0 2024-04-25T16:52:26.805 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T16:52:26.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.815+0000 7fd725852640 1 -- 172.21.15.94:0/1783889726 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd70c06f430 msgr2=0x7fd70c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:26.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.815+0000 7fd725852640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd70c06f430 0x7fd70c071940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd714009160 tx=0x7fd714008040 comp rx=0 tx=0).stop 2024-04-25T16:52:26.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.815+0000 7fd725852640 1 -- 172.21.15.94:0/1783889726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7080039c0 msgr2=0x7fd7200d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:26.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.815+0000 7fd725852640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7080039c0 0x7fd7200d1a30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd710007040 tx=0x7fd708005840 comp rx=0 tx=0).stop 2024-04-25T16:52:26.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.815+0000 7fd725852640 1 -- 172.21.15.94:0/1783889726 shutdown_connections 2024-04-25T16:52:26.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.815+0000 7fd725852640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd70c06f430 0x7fd70c071940 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:26.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.815+0000 7fd725852640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd720152350 0x7fd720152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:26.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.815+0000 7fd725852640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7201f6c70 0x7fd72014f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:26.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.815+0000 7fd725852640 1 --2- 172.21.15.94:0/1783889726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7080039c0 0x7fd7200d1a30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:26.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.815+0000 7fd725852640 1 -- 172.21.15.94:0/1783889726 >> 172.21.15.94:0/1783889726 conn(0x7fd7200b47f0 msgr2=0x7fd7200b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:26.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.823+0000 7fd725852640 1 -- 172.21.15.94:0/1783889726 shutdown_connections 2024-04-25T16:52:26.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.823+0000 7fd725852640 1 -- 172.21.15.94:0/1783889726 wait complete. 2024-04-25T16:52:26.846 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:52:26.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d9c3f640 1 Processor -- start 2024-04-25T16:52:26.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d9c3f640 1 -- start start 2024-04-25T16:52:26.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d9c3f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd9d4070080 0x7fd9d415e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:26.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d9c3f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd9d415e7c0 0x7fd9d406c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:26.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d9c3f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9d406fc60 con 0x7fd9d415ebd0 2024-04-25T16:52:26.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d9c3f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd9d4158550 con 0x7fd9d415e7c0 2024-04-25T16:52:26.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d9c3f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd9d406f7d0 con 0x7fd9d4070080 2024-04-25T16:52:26.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d2ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd9d415e7c0 0x7fd9d406c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:26.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d3fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd9d415ebd0 0x7fd9d406cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45898/0 (socket says 172.21.15.94:45898) 2024-04-25T16:52:26.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d2ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd9d415e7c0 0x7fd9d406c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37826/0 (socket says 172.21.15.94:37826) 2024-04-25T16:52:26.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d3fff640 1 -- 172.21.15.94:0/2345283053 learned_addr learned my addr 172.21.15.94:0/2345283053 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:26.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d37fe640 1 --2- 172.21.15.94:0/2345283053 >> v2:172.21.15.138:3300/0 conn(0x7fd9d4070080 0x7fd9d415e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:26.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d2ffd640 1 -- 172.21.15.94:0/2345283053 >> v2:172.21.15.138:3300/0 conn(0x7fd9d4070080 msgr2=0x7fd9d415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:26.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d2ffd640 1 --2- 172.21.15.94:0/2345283053 >> v2:172.21.15.138:3300/0 conn(0x7fd9d4070080 0x7fd9d415e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:26.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d2ffd640 1 -- 172.21.15.94:0/2345283053 >> v1:172.21.15.94:6789/0 conn(0x7fd9d415ebd0 legacy=0x7fd9d406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:26.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d2ffd640 1 -- 172.21.15.94:0/2345283053 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d40aef70 con 0x7fd9d415e7c0 2024-04-25T16:52:26.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.991+0000 7fd9d27fc640 1 -- 172.21.15.94:0/2345283053 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1450597377 0 0) 0x7fd9d406fc60 con 0x7fd9d415ebd0 2024-04-25T16:52:26.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.995+0000 7fd9d2ffd640 1 --2- 172.21.15.94:0/2345283053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d415e7c0 0x7fd9d406c600 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4071f10 tx=0x7fd9c401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1b2175f5fa7203 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:26.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.995+0000 7fd9d27fc640 1 -- 172.21.15.94:0/2345283053 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40b2070 con 0x7fd9d415e7c0 2024-04-25T16:52:26.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.995+0000 7fd9d27fc640 1 -- 172.21.15.94:0/2345283053 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c40a9070 con 0x7fd9d415e7c0 2024-04-25T16:52:26.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.995+0000 7fd9d27fc640 1 -- 172.21.15.94:0/2345283053 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40ad880 con 0x7fd9d415e7c0 2024-04-25T16:52:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.995+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/2345283053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d415e7c0 msgr2=0x7fd9d406c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.995+0000 7fd9d9c3f640 1 --2- 172.21.15.94:0/2345283053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d415e7c0 0x7fd9d406c600 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4071f10 tx=0x7fd9c401c450 comp rx=0 tx=0).stop 2024-04-25T16:52:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.995+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/2345283053 shutdown_connections 2024-04-25T16:52:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.995+0000 7fd9d9c3f640 1 --2- 172.21.15.94:0/2345283053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d415e7c0 0x7fd9d406c600 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.995+0000 7fd9d9c3f640 1 --2- 172.21.15.94:0/2345283053 >> v2:172.21.15.138:3300/0 conn(0x7fd9d4070080 0x7fd9d415e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.995+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/2345283053 >> 172.21.15.94:0/2345283053 conn(0x7fd9d40b47f0 msgr2=0x7fd9d40b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.995+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/2345283053 shutdown_connections 2024-04-25T16:52:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.995+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/2345283053 wait complete. 2024-04-25T16:52:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d9c3f640 1 Processor -- start 2024-04-25T16:52:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d9c3f640 1 -- start start 2024-04-25T16:52:27.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d9c3f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d4070080 0x7fd9d409aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d9c3f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d415e7c0 0x7fd9d409c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d9c3f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d415ebd0 0x7fd9d4156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d9c3f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d4070490 con 0x7fd9d415ebd0 2024-04-25T16:52:27.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d9c3f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d40ae6c0 con 0x7fd9d4070080 2024-04-25T16:52:27.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d9c3f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d416ae90 con 0x7fd9d415e7c0 2024-04-25T16:52:27.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d415ebd0 0x7fd9d4156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d415ebd0 0x7fd9d4156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37840/0 (socket says 172.21.15.94:37840) 2024-04-25T16:52:27.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d3fff640 1 -- 172.21.15.94:0/3460409919 learned_addr learned my addr 172.21.15.94:0/3460409919 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d37fe640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d4070080 0x7fd9d409aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d2ffd640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d415e7c0 0x7fd9d409c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d3fff640 1 -- 172.21.15.94:0/3460409919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d415e7c0 msgr2=0x7fd9d409c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d3fff640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d415e7c0 0x7fd9d409c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d3fff640 1 -- 172.21.15.94:0/3460409919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d4070080 msgr2=0x7fd9d409aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d3fff640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d4070080 0x7fd9d409aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d3fff640 1 -- 172.21.15.94:0/3460409919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d409e550 con 0x7fd9d415ebd0 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:26.999+0000 7fd9d3fff640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d415ebd0 0x7fd9d4156940 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00c6d0 tx=0x7fd9cc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.003+0000 7fd9bbfff640 1 -- 172.21.15.94:0/3460409919 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc018070 con 0x7fd9d415ebd0 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.003+0000 7fd9bbfff640 1 -- 172.21.15.94:0/3460409919 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9cc004d70 con 0x7fd9d415ebd0 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.003+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/3460409919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d409e830 con 0x7fd9d415ebd0 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.003+0000 7fd9bbfff640 1 -- 172.21.15.94:0/3460409919 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00a820 con 0x7fd9d415ebd0 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.003+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/3460409919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d409edf0 con 0x7fd9d415ebd0 2024-04-25T16:52:27.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.003+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/3460409919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d4076830 con 0x7fd9d415ebd0 2024-04-25T16:52:27.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.003+0000 7fd9bbfff640 1 -- 172.21.15.94:0/3460409919 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd9cc01b070 con 0x7fd9d415ebd0 2024-04-25T16:52:27.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.003+0000 7fd9bbfff640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd9c006f430 0x7fd9c0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.003+0000 7fd9d37fe640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd9c006f430 0x7fd9c0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.003+0000 7fd9bbfff640 1 -- 172.21.15.94:0/3460409919 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7fd9cc08ed90 con 0x7fd9d415ebd0 2024-04-25T16:52:27.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.003+0000 7fd9d37fe640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd9c006f430 0x7fd9c0071940 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40b50c0 tx=0x7fd9c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:27.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.007+0000 7fd9bbfff640 1 -- 172.21.15.94:0/3460409919 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd9cc05b580 con 0x7fd9d415ebd0 2024-04-25T16:52:27.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.171+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/3460409919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9d4068590 con 0x7fd9d415ebd0 2024-04-25T16:52:27.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.171+0000 7fd9bbfff640 1 -- 172.21.15.94:0/3460409919 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16967 (secure 0 0 0) 0x7fd9cc05f880 con 0x7fd9d415ebd0 2024-04-25T16:52:27.170 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:27.170 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":37,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:52:26.175287+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:52:23.840281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T16:52:25.130347+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:52:27.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.175+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/3460409919 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd9c006f430 msgr2=0x7fd9c0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.175+0000 7fd9d9c3f640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd9c006f430 0x7fd9c0071940 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40b50c0 tx=0x7fd9c8008040 comp rx=0 tx=0).stop 2024-04-25T16:52:27.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.175+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/3460409919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d415ebd0 msgr2=0x7fd9d4156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.175+0000 7fd9d9c3f640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d415ebd0 0x7fd9d4156940 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00c6d0 tx=0x7fd9cc00cb90 comp rx=0 tx=0).stop 2024-04-25T16:52:27.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.175+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/3460409919 shutdown_connections 2024-04-25T16:52:27.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.175+0000 7fd9d9c3f640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9d415ebd0 0x7fd9d4156940 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.175+0000 7fd9d9c3f640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd9c006f430 0x7fd9c0071940 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.175+0000 7fd9d9c3f640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9d415e7c0 0x7fd9d409c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.175+0000 7fd9d9c3f640 1 --2- 172.21.15.94:0/3460409919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d4070080 0x7fd9d409aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.175+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/3460409919 >> 172.21.15.94:0/3460409919 conn(0x7fd9d40b47f0 msgr2=0x7fd9d406c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:27.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.175+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/3460409919 shutdown_connections 2024-04-25T16:52:27.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.175+0000 7fd9d9c3f640 1 -- 172.21.15.94:0/3460409919 wait complete. 2024-04-25T16:52:27.192 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:52:27.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f075ac80640 1 Processor -- start 2024-04-25T16:52:27.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f075ac80640 1 -- start start 2024-04-25T16:52:27.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f075ac80640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f075406de20 0x7f07540af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f075ac80640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0754074a20 con 0x7f0754074610 2024-04-25T16:52:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f075ac80640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f075415d350 con 0x7f075415cf40 2024-04-25T16:52:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f075ac80640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0754160760 con 0x7f075406de20 2024-04-25T16:52:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f0758a1c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f0754074610 0x7f0754160bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45902/0 (socket says 172.21.15.94:45902) 2024-04-25T16:52:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f0758a1c640 1 -- 172.21.15.94:0/1106170769 learned_addr learned my addr 172.21.15.94:0/1106170769 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f075921d640 1 --2- 172.21.15.94:0/1106170769 >> v2:172.21.15.138:3300/0 conn(0x7f075406de20 0x7f07540af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4191076210 0 0) 0x7f0744001120 con 0x7f0754074610 2024-04-25T16:52:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f073c003530 con 0x7f0754074610 2024-04-25T16:52:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 923190682 0 0) 0x7f073c003530 con 0x7f0754074610 2024-04-25T16:52:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 >> v2:172.21.15.138:3300/0 conn(0x7f075406de20 msgr2=0x7f07540af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 --2- 172.21.15.94:0/1106170769 >> v2:172.21.15.138:3300/0 conn(0x7f075406de20 0x7f07540af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 >> v1:172.21.15.138:6789/0 conn(0x7f075415cf40 legacy=0x7f075406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075415d620 con 0x7f0754074610 2024-04-25T16:52:27.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f0744001850 con 0x7f0754074610 2024-04-25T16:52:27.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 >> v1:172.21.15.94:6789/0 conn(0x7f0754074610 legacy=0x7f0754160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 --2- 172.21.15.94:0/1106170769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f073c0039c0 0x7f073c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 --2- 172.21.15.94:0/1106170769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f073c004810 0x7f073c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 --2- 172.21.15.94:0/1106170769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f073c0051d0 0x7f073c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f073c00c140 con 0x7f073c004810 2024-04-25T16:52:27.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f073c00c2f0 con 0x7f073c0051d0 2024-04-25T16:52:27.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f073c00c4f0 con 0x7f073c0039c0 2024-04-25T16:52:27.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f0753fff640 1 --2- 172.21.15.94:0/1106170769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f073c004810 0x7f073c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f075921d640 1 --2- 172.21.15.94:0/1106170769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f073c0051d0 0x7f073c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.339+0000 7f0758a1c640 1 --2- 172.21.15.94:0/1106170769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f073c0039c0 0x7f073c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f0753fff640 1 -- 172.21.15.94:0/1106170769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f073c0039c0 msgr2=0x7f073c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:27.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f0753fff640 1 --2- 172.21.15.94:0/1106170769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f073c0039c0 0x7f073c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f0753fff640 1 -- 172.21.15.94:0/1106170769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f073c0051d0 msgr2=0x7f073c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:27.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f0753fff640 1 --2- 172.21.15.94:0/1106170769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f073c0051d0 0x7f073c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f0753fff640 1 -- 172.21.15.94:0/1106170769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c003530 con 0x7f073c004810 2024-04-25T16:52:27.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f075921d640 1 -- 172.21.15.94:0/1106170769 reap_dead start 2024-04-25T16:52:27.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f0758a1c640 1 --2- 172.21.15.94:0/1106170769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f073c0039c0 0x7f073c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:27.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f0753fff640 1 --2- 172.21.15.94:0/1106170769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f073c004810 0x7f073c004c20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0748001780 tx=0x7f074801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5f4445bd7d01cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:27.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07480013a0 con 0x7f073c004810 2024-04-25T16:52:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07480052f0 con 0x7f073c004810 2024-04-25T16:52:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f07537fe640 1 -- 172.21.15.94:0/1106170769 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748003840 con 0x7f073c004810 2024-04-25T16:52:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f075ac80640 1 -- 172.21.15.94:0/1106170769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f073c004810 msgr2=0x7f073c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f075ac80640 1 --2- 172.21.15.94:0/1106170769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f073c004810 0x7f073c004c20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0748001780 tx=0x7f074801c400 comp rx=0 tx=0).stop 2024-04-25T16:52:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f075ac80640 1 -- 172.21.15.94:0/1106170769 shutdown_connections 2024-04-25T16:52:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f075ac80640 1 --2- 172.21.15.94:0/1106170769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f073c004810 0x7f073c004c20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f075ac80640 1 -- 172.21.15.94:0/1106170769 >> 172.21.15.94:0/1106170769 conn(0x7f07540b47f0 msgr2=0x7f07540b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f075ac80640 1 -- 172.21.15.94:0/1106170769 shutdown_connections 2024-04-25T16:52:27.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.343+0000 7f075ac80640 1 -- 172.21.15.94:0/1106170769 wait complete. 2024-04-25T16:52:27.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075ac80640 1 Processor -- start 2024-04-25T16:52:27.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075ac80640 1 -- start start 2024-04-25T16:52:27.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075ac80640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f073c004810 0x7f07540b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075ac80640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0754148140 0x7f0754158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075ac80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07541590e0 0x7f0754153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075ac80640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f075415dbe0 con 0x7f07541590e0 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075ac80640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f075415dec0 con 0x7f073c004810 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075ac80640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f075406fea0 con 0x7f0754148140 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075921d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07541590e0 0x7f0754153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075921d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07541590e0 0x7f0754153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37854/0 (socket says 172.21.15.94:37854) 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075921d640 1 -- 172.21.15.94:0/91028851 learned_addr learned my addr 172.21.15.94:0/91028851 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f0758a1c640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f073c004810 0x7f07540b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f0753fff640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0754148140 0x7f0754158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075921d640 1 -- 172.21.15.94:0/91028851 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0754148140 msgr2=0x7f0754158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075921d640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0754148140 0x7f0754158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075921d640 1 -- 172.21.15.94:0/91028851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f073c004810 msgr2=0x7f07540b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075921d640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f073c004810 0x7f07540b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075921d640 1 -- 172.21.15.94:0/91028851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07540b0f00 con 0x7f07541590e0 2024-04-25T16:52:27.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f0758a1c640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f073c004810 0x7f07540b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:27.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075921d640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07541590e0 0x7f0754153f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f07540b0d30 tx=0x7f074c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:27.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f07517fa640 1 -- 172.21.15.94:0/91028851 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c011de0 con 0x7f07541590e0 2024-04-25T16:52:27.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f07517fa640 1 -- 172.21.15.94:0/91028851 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f074c01b070 con 0x7f07541590e0 2024-04-25T16:52:27.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075ac80640 1 -- 172.21.15.94:0/91028851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07540b1ed0 con 0x7f07541590e0 2024-04-25T16:52:27.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f07517fa640 1 -- 172.21.15.94:0/91028851 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c0039a0 con 0x7f07541590e0 2024-04-25T16:52:27.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.347+0000 7f075ac80640 1 -- 172.21.15.94:0/91028851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07540b2490 con 0x7f07541590e0 2024-04-25T16:52:27.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.351+0000 7f075ac80640 1 -- 172.21.15.94:0/91028851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0754076830 con 0x7f07541590e0 2024-04-25T16:52:27.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.351+0000 7f07517fa640 1 -- 172.21.15.94:0/91028851 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f074c003b40 con 0x7f07541590e0 2024-04-25T16:52:27.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.351+0000 7f07517fa640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f074006f430 0x7f0740071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.351+0000 7f07517fa640 1 -- 172.21.15.94:0/91028851 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f074c08fd20 con 0x7f07541590e0 2024-04-25T16:52:27.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.351+0000 7f0758a1c640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f074006f430 0x7f0740071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.351+0000 7f0758a1c640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f074006f430 0x7f0740071940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0744001a40 tx=0x7f0744001220 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:27.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.355+0000 7f07517fa640 1 -- 172.21.15.94:0/91028851 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f074c05cb10 con 0x7f07541590e0 2024-04-25T16:52:27.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.515+0000 7f075ac80640 1 -- 172.21.15.94:0/91028851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f075406fa50 con 0x7f07541590e0 2024-04-25T16:52:27.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.519+0000 7f07517fa640 1 -- 172.21.15.94:0/91028851 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16967 (secure 0 0 0) 0x7f074c05c410 con 0x7f07541590e0 2024-04-25T16:52:27.516 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:27.516 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":37,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:52:26.175287+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:52:23.840281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T16:52:25.130347+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:52:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.519+0000 7f075ac80640 1 -- 172.21.15.94:0/91028851 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f074006f430 msgr2=0x7f0740071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.519+0000 7f075ac80640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f074006f430 0x7f0740071940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0744001a40 tx=0x7f0744001220 comp rx=0 tx=0).stop 2024-04-25T16:52:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.519+0000 7f075ac80640 1 -- 172.21.15.94:0/91028851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07541590e0 msgr2=0x7f0754153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.519+0000 7f075ac80640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07541590e0 0x7f0754153f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f07540b0d30 tx=0x7f074c0115a0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.519+0000 7f075ac80640 1 -- 172.21.15.94:0/91028851 shutdown_connections 2024-04-25T16:52:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.519+0000 7f075ac80640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07541590e0 0x7f0754153f90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.519+0000 7f075ac80640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f074006f430 0x7f0740071940 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.519+0000 7f075ac80640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0754148140 0x7f0754158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.519+0000 7f075ac80640 1 --2- 172.21.15.94:0/91028851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f073c004810 0x7f07540b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.519+0000 7f075ac80640 1 -- 172.21.15.94:0/91028851 >> 172.21.15.94:0/91028851 conn(0x7f07540b47f0 msgr2=0x7f07540af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.519+0000 7f075ac80640 1 -- 172.21.15.94:0/91028851 shutdown_connections 2024-04-25T16:52:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.519+0000 7f075ac80640 1 -- 172.21.15.94:0/91028851 wait complete. 2024-04-25T16:52:27.537 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T16:52:27.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f518ab6d640 1 Processor -- start 2024-04-25T16:52:27.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f518ab6d640 1 -- start start 2024-04-25T16:52:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f518ab6d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f51840b0420 0x7f518406ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f518ab6d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f518415ec30 0x7f518406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f518ab6d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5184070630 con 0x7f518406f2a0 2024-04-25T16:52:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f518ab6d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f51840aeaa0 con 0x7f51840b0420 2024-04-25T16:52:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f518ab6d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f51840703e0 con 0x7f518415ec30 2024-04-25T16:52:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f5183fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f518406f2a0 0x7f518406c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45906/0 (socket says 172.21.15.94:45906) 2024-04-25T16:52:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f5183fff640 1 -- 172.21.15.94:0/3858683520 learned_addr learned my addr 172.21.15.94:0/3858683520 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f5188909640 1 --2- 172.21.15.94:0/3858683520 >> v2:172.21.15.138:3300/0 conn(0x7f51840b0420 0x7f518406ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f518910a640 1 --2- 172.21.15.94:0/3858683520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f518415ec30 0x7f518406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 18563609 0 0) 0x7f5184070630 con 0x7f518406f2a0 2024-04-25T16:52:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f516c003530 con 0x7f518406f2a0 2024-04-25T16:52:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3995587755 0 0) 0x7f516c003530 con 0x7f518406f2a0 2024-04-25T16:52:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f518415ec30 msgr2=0x7f518406cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:27.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 --2- 172.21.15.94:0/3858683520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f518415ec30 0x7f518406cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 >> v2:172.21.15.138:3300/0 conn(0x7f51840b0420 msgr2=0x7f518406ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:27.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 --2- 172.21.15.94:0/3858683520 >> v2:172.21.15.138:3300/0 conn(0x7f51840b0420 0x7f518406ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518406f6b0 con 0x7f518406f2a0 2024-04-25T16:52:27.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f518910a640 1 --2- 172.21.15.94:0/3858683520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f518415ec30 0x7f518406cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:27.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f5178001680 con 0x7f518406f2a0 2024-04-25T16:52:27.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f5188909640 1 --2- 172.21.15.94:0/3858683520 >> v2:172.21.15.138:3300/0 conn(0x7f51840b0420 0x7f518406ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:27.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 >> v1:172.21.15.94:6789/0 conn(0x7f518406f2a0 legacy=0x7f518406c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 --2- 172.21.15.94:0/3858683520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f516c0039c0 0x7f516c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 --2- 172.21.15.94:0/3858683520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f518415ec30 0x7f516c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 --2- 172.21.15.94:0/3858683520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c004d50 0x7f516c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51840aeaa0 con 0x7f516c004d50 2024-04-25T16:52:27.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51840703e0 con 0x7f516c0039c0 2024-04-25T16:52:27.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f516c00fde0 con 0x7f518415ec30 2024-04-25T16:52:27.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.695+0000 7f518910a640 1 --2- 172.21.15.94:0/3858683520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c004d50 0x7f516c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f5188909640 1 --2- 172.21.15.94:0/3858683520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f516c0039c0 0x7f516c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f5183fff640 1 --2- 172.21.15.94:0/3858683520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f518415ec30 0x7f516c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f518910a640 1 -- 172.21.15.94:0/3858683520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f518415ec30 msgr2=0x7f516c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:27.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f518910a640 1 --2- 172.21.15.94:0/3858683520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f518415ec30 0x7f516c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f518910a640 1 -- 172.21.15.94:0/3858683520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f516c0039c0 msgr2=0x7f516c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:27.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f518910a640 1 --2- 172.21.15.94:0/3858683520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f516c0039c0 0x7f516c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f518910a640 1 -- 172.21.15.94:0/3858683520 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516c003530 con 0x7f516c004d50 2024-04-25T16:52:27.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f518910a640 1 --2- 172.21.15.94:0/3858683520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c004d50 0x7f516c005160 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f517c00e350 tx=0x7f517c01cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a73061ae188a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:27.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c009de0 con 0x7f516c004d50 2024-04-25T16:52:27.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f517c0a5df0 con 0x7f516c004d50 2024-04-25T16:52:27.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f51837fe640 1 -- 172.21.15.94:0/3858683520 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c003ba0 con 0x7f516c004d50 2024-04-25T16:52:27.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f518ab6d640 1 -- 172.21.15.94:0/3858683520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c004d50 msgr2=0x7f516c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f518ab6d640 1 --2- 172.21.15.94:0/3858683520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c004d50 0x7f516c005160 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f517c00e350 tx=0x7f517c01cde0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f518910a640 1 -- 172.21.15.94:0/3858683520 reap_dead start 2024-04-25T16:52:27.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f518ab6d640 1 -- 172.21.15.94:0/3858683520 shutdown_connections 2024-04-25T16:52:27.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f518ab6d640 1 -- 172.21.15.94:0/3858683520 >> 172.21.15.94:0/3858683520 conn(0x7f51840b47f0 msgr2=0x7f51840b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:27.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f518ab6d640 1 -- 172.21.15.94:0/3858683520 shutdown_connections 2024-04-25T16:52:27.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.699+0000 7f518ab6d640 1 -- 172.21.15.94:0/3858683520 wait complete. 2024-04-25T16:52:27.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518ab6d640 1 Processor -- start 2024-04-25T16:52:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518ab6d640 1 -- start start 2024-04-25T16:52:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518ab6d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5184156a80 0x7f51841604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518ab6d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f518409a7f0 0x7f518415f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518ab6d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f518415f580 0x7f51841f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518ab6d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51840ae730 con 0x7f518415f580 2024-04-25T16:52:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518ab6d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51840ae450 con 0x7f518409a7f0 2024-04-25T16:52:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518ab6d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f51840aef30 con 0x7f5184156a80 2024-04-25T16:52:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518910a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f518415f580 0x7f51841f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518910a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f518415f580 0x7f51841f2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37878/0 (socket says 172.21.15.94:37878) 2024-04-25T16:52:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518910a640 1 -- 172.21.15.94:0/2290519803 learned_addr learned my addr 172.21.15.94:0/2290519803 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f5183fff640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f518409a7f0 0x7f518415f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f5188909640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5184156a80 0x7f51841604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518910a640 1 -- 172.21.15.94:0/2290519803 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5184156a80 msgr2=0x7f51841604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518910a640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5184156a80 0x7f51841604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518910a640 1 -- 172.21.15.94:0/2290519803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f518409a7f0 msgr2=0x7f518415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518910a640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f518409a7f0 0x7f518415f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518910a640 1 -- 172.21.15.94:0/2290519803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518415f990 con 0x7f518415f580 2024-04-25T16:52:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f5183fff640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f518409a7f0 0x7f518415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f5188909640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5184156a80 0x7f51841604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518910a640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f518415f580 0x7f51841f2070 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f517c00e350 tx=0x7f517c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f51817fa640 1 -- 172.21.15.94:0/2290519803 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c009de0 con 0x7f518415f580 2024-04-25T16:52:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f51817fa640 1 -- 172.21.15.94:0/2290519803 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f517c009a50 con 0x7f518415f580 2024-04-25T16:52:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518ab6d640 1 -- 172.21.15.94:0/2290519803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5184160a10 con 0x7f518415f580 2024-04-25T16:52:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f51817fa640 1 -- 172.21.15.94:0/2290519803 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c0add60 con 0x7f518415f580 2024-04-25T16:52:27.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.703+0000 7f518ab6d640 1 -- 172.21.15.94:0/2290519803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51841571b0 con 0x7f518415f580 2024-04-25T16:52:27.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.707+0000 7f518ab6d640 1 -- 172.21.15.94:0/2290519803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5184076830 con 0x7f518415f580 2024-04-25T16:52:27.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.707+0000 7f51817fa640 1 -- 172.21.15.94:0/2290519803 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f517c0c0030 con 0x7f518415f580 2024-04-25T16:52:27.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.707+0000 7f51817fa640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f517006f400 0x7f5170071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:27.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.707+0000 7f51817fa640 1 -- 172.21.15.94:0/2290519803 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f517c12a820 con 0x7f518415f580 2024-04-25T16:52:27.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.707+0000 7f5188909640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f517006f400 0x7f5170071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:27.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.707+0000 7f5188909640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f517006f400 0x7f5170071910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f51841600c0 tx=0x7f5174008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:27.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.711+0000 7f51817fa640 1 -- 172.21.15.94:0/2290519803 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f518415f990 con 0x7f518415f580 2024-04-25T16:52:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:27.915+0000 7f518ab6d640 1 -- 172.21.15.94:0/2290519803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f51840cd0d0 con 0x7f518415f580 2024-04-25T16:52:28.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:28.823+0000 7f51817fa640 1 -- 172.21.15.94:0/2290519803 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f517c016b30 con 0x7f518415f580 2024-04-25T16:52:28.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:28.827+0000 7f518ab6d640 1 -- 172.21.15.94:0/2290519803 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f517006f400 msgr2=0x7f5170071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:28.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:28.827+0000 7f518ab6d640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f517006f400 0x7f5170071910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f51841600c0 tx=0x7f5174008040 comp rx=0 tx=0).stop 2024-04-25T16:52:28.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:28.827+0000 7f518ab6d640 1 -- 172.21.15.94:0/2290519803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f518415f580 msgr2=0x7f51841f2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:28.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:28.827+0000 7f518ab6d640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f518415f580 0x7f51841f2070 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f517c00e350 tx=0x7f517c009750 comp rx=0 tx=0).stop 2024-04-25T16:52:28.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:28.827+0000 7f518ab6d640 1 -- 172.21.15.94:0/2290519803 shutdown_connections 2024-04-25T16:52:28.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:28.827+0000 7f518ab6d640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f518415f580 0x7f51841f2070 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:28.827+0000 7f518ab6d640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f518409a7f0 0x7f518415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:28.827+0000 7f518ab6d640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f517006f400 0x7f5170071910 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:28.827+0000 7f518ab6d640 1 --2- 172.21.15.94:0/2290519803 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5184156a80 0x7f51841604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:28.827+0000 7f518ab6d640 1 -- 172.21.15.94:0/2290519803 >> 172.21.15.94:0/2290519803 conn(0x7f51840b47f0 msgr2=0x7f5184073170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:28.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:28.827+0000 7f518ab6d640 1 -- 172.21.15.94:0/2290519803 shutdown_connections 2024-04-25T16:52:28.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:28.827+0000 7f518ab6d640 1 -- 172.21.15.94:0/2290519803 wait complete. 2024-04-25T16:52:28.852 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T16:52:28.852 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T16:52:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f27399fa640 1 Processor -- start 2024-04-25T16:52:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f27399fa640 1 -- start start 2024-04-25T16:52:29.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f27399fa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f273406d710 0x7f273406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:29.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f27399fa640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2734074380 con 0x7f273415c7e0 2024-04-25T16:52:29.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f27399fa640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f273406e060 con 0x7f273406e7a0 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f27399fa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f27340727e0 con 0x7f273406d710 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2732ffd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f273415c7e0 0x7f27340afdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56814/0 (socket says 172.21.15.94:56814) 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2732ffd640 1 -- 172.21.15.94:0/4045211532 learned_addr learned my addr 172.21.15.94:0/4045211532 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f27327fc640 1 --2- 172.21.15.94:0/4045211532 >> v2:172.21.15.138:3300/0 conn(0x7f273406d710 0x7f273406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2214762326 0 0) 0x7f272c0014a0 con 0x7f273415c7e0 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f271c003530 con 0x7f273415c7e0 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 703688597 0 0) 0x7f271c003530 con 0x7f273415c7e0 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 >> v2:172.21.15.138:3300/0 conn(0x7f273406d710 msgr2=0x7f273406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 --2- 172.21.15.94:0/4045211532 >> v2:172.21.15.138:3300/0 conn(0x7f273406d710 0x7f273406db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 >> v1:172.21.15.138:6789/0 conn(0x7f273406e7a0 legacy=0x7f273406ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2734072ac0 con 0x7f273415c7e0 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f272c001bd0 con 0x7f273415c7e0 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 >> v1:172.21.15.94:6789/0 conn(0x7f273415c7e0 legacy=0x7f27340afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 --2- 172.21.15.94:0/4045211532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f271c0039c0 0x7f271c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 --2- 172.21.15.94:0/4045211532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f271c004810 0x7f271c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 --2- 172.21.15.94:0/4045211532 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f271c005260 0x7f271c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27340727e0 con 0x7f271c004810 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f271c00c1f0 con 0x7f271c0039c0 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.015+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f271c00c3f0 con 0x7f271c005260 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.019+0000 7f27327fc640 1 --2- 172.21.15.94:0/4045211532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f271c004810 0x7f271c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.019+0000 7f2732ffd640 1 --2- 172.21.15.94:0/4045211532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f271c0039c0 0x7f271c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.019+0000 7f27337fe640 1 --2- 172.21.15.94:0/4045211532 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f271c005260 0x7f271c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.019+0000 7f27327fc640 1 -- 172.21.15.94:0/4045211532 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f271c005260 msgr2=0x7f271c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.019+0000 7f27327fc640 1 --2- 172.21.15.94:0/4045211532 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f271c005260 0x7f271c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.019+0000 7f27327fc640 1 -- 172.21.15.94:0/4045211532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f271c0039c0 msgr2=0x7f271c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.019+0000 7f27327fc640 1 --2- 172.21.15.94:0/4045211532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f271c0039c0 0x7f271c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.019+0000 7f27337fe640 1 -- 172.21.15.94:0/4045211532 reap_dead start 2024-04-25T16:52:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.019+0000 7f27327fc640 1 -- 172.21.15.94:0/4045211532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c003530 con 0x7f271c004810 2024-04-25T16:52:29.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.023+0000 7f27327fc640 1 --2- 172.21.15.94:0/4045211532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f271c004810 0x7f271c004c20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2728009ab0 tx=0x7f272801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f623eadb88fc93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:29.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.023+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27280096b0 con 0x7f271c004810 2024-04-25T16:52:29.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.023+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27280a6510 con 0x7f271c004810 2024-04-25T16:52:29.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.023+0000 7f2731ffb640 1 -- 172.21.15.94:0/4045211532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27280a6900 con 0x7f271c004810 2024-04-25T16:52:29.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.023+0000 7f27399fa640 1 -- 172.21.15.94:0/4045211532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f271c004810 msgr2=0x7f271c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:29.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.023+0000 7f27399fa640 1 --2- 172.21.15.94:0/4045211532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f271c004810 0x7f271c004c20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2728009ab0 tx=0x7f272801ca20 comp rx=0 tx=0).stop 2024-04-25T16:52:29.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.023+0000 7f27399fa640 1 -- 172.21.15.94:0/4045211532 shutdown_connections 2024-04-25T16:52:29.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.023+0000 7f27399fa640 1 --2- 172.21.15.94:0/4045211532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f271c004810 0x7f271c004c20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.023+0000 7f27399fa640 1 -- 172.21.15.94:0/4045211532 >> 172.21.15.94:0/4045211532 conn(0x7f27340b47f0 msgr2=0x7f27340b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:29.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.023+0000 7f27399fa640 1 -- 172.21.15.94:0/4045211532 shutdown_connections 2024-04-25T16:52:29.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.023+0000 7f27399fa640 1 -- 172.21.15.94:0/4045211532 wait complete. 2024-04-25T16:52:29.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f27399fa640 1 Processor -- start 2024-04-25T16:52:29.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f27399fa640 1 -- start start 2024-04-25T16:52:29.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f27399fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f271c004810 0x7f273415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:29.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f27399fa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f273406fd10 0x7f273413d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:29.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f27399fa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2734070120 0x7f273413d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:29.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f27399fa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f273415cbf0 con 0x7f271c004810 2024-04-25T16:52:29.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f27399fa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2734073ca0 con 0x7f2734070120 2024-04-25T16:52:29.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f27399fa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f27340afdf0 con 0x7f273406fd10 2024-04-25T16:52:29.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f271c004810 0x7f273415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:29.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f271c004810 0x7f273415cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58936/0 (socket says 172.21.15.94:58936) 2024-04-25T16:52:29.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f2732ffd640 1 -- 172.21.15.94:0/3687461356 learned_addr learned my addr 172.21.15.94:0/3687461356 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:29.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f27327fc640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f273406fd10 0x7f273413d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:29.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f27337fe640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2734070120 0x7f273413d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:29.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f2732ffd640 1 -- 172.21.15.94:0/3687461356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f273406fd10 msgr2=0x7f273413d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:29.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f2732ffd640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f273406fd10 0x7f273413d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f2732ffd640 1 -- 172.21.15.94:0/3687461356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2734070120 msgr2=0x7f273413d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:29.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f2732ffd640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2734070120 0x7f273413d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f2732ffd640 1 -- 172.21.15.94:0/3687461356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273415fd20 con 0x7f271c004810 2024-04-25T16:52:29.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f27337fe640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2734070120 0x7f273413d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:29.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f2732ffd640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f271c004810 0x7f273415cd70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f272c00aab0 tx=0x7f272c00af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:29.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f271bfff640 1 -- 172.21.15.94:0/3687461356 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c017020 con 0x7f271c004810 2024-04-25T16:52:29.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f271bfff640 1 -- 172.21.15.94:0/3687461356 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c007a30 con 0x7f271c004810 2024-04-25T16:52:29.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f27399fa640 1 -- 172.21.15.94:0/3687461356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273415ec80 con 0x7f271c004810 2024-04-25T16:52:29.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f271bfff640 1 -- 172.21.15.94:0/3687461356 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c0135f0 con 0x7f271c004810 2024-04-25T16:52:29.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.027+0000 7f27399fa640 1 -- 172.21.15.94:0/3687461356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273415f240 con 0x7f271c004810 2024-04-25T16:52:29.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.031+0000 7f27399fa640 1 -- 172.21.15.94:0/3687461356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2734076830 con 0x7f271c004810 2024-04-25T16:52:29.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.031+0000 7f271bfff640 1 -- 172.21.15.94:0/3687461356 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f272c01a020 con 0x7f271c004810 2024-04-25T16:52:29.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.031+0000 7f271bfff640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f272006f430 0x7f2720071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:29.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.031+0000 7f271bfff640 1 -- 172.21.15.94:0/3687461356 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f272c090490 con 0x7f271c004810 2024-04-25T16:52:29.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.031+0000 7f27327fc640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f272006f430 0x7f2720071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:29.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.031+0000 7f27327fc640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f272006f430 0x7f2720071940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f27280af000 tx=0x7f27280046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:29.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.035+0000 7f271bfff640 1 -- 172.21.15.94:0/3687461356 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f272c021020 con 0x7f271c004810 2024-04-25T16:52:29.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.227+0000 7f27399fa640 1 -- 172.21.15.94:0/3687461356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f27341601d0 con 0x7f271c004810 2024-04-25T16:52:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.827+0000 7f271bfff640 1 -- 172.21.15.94:0/3687461356 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f2734074380 con 0x7f271c004810 2024-04-25T16:52:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.831+0000 7f27399fa640 1 -- 172.21.15.94:0/3687461356 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f272006f430 msgr2=0x7f2720071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.831+0000 7f27399fa640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f272006f430 0x7f2720071940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f27280af000 tx=0x7f27280046d0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.831+0000 7f27399fa640 1 -- 172.21.15.94:0/3687461356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f271c004810 msgr2=0x7f273415cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.831+0000 7f27399fa640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f271c004810 0x7f273415cd70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f272c00aab0 tx=0x7f272c00af70 comp rx=0 tx=0).stop 2024-04-25T16:52:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.831+0000 7f27399fa640 1 -- 172.21.15.94:0/3687461356 shutdown_connections 2024-04-25T16:52:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.831+0000 7f27399fa640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2734070120 0x7f273413d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.831+0000 7f27399fa640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f272006f430 0x7f2720071940 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.831+0000 7f27399fa640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f273406fd10 0x7f273413d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.831+0000 7f27399fa640 1 --2- 172.21.15.94:0/3687461356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f271c004810 0x7f273415cd70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.831+0000 7f27399fa640 1 -- 172.21.15.94:0/3687461356 >> 172.21.15.94:0/3687461356 conn(0x7f27340b47f0 msgr2=0x7f27340b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:29.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.831+0000 7f27399fa640 1 -- 172.21.15.94:0/3687461356 shutdown_connections 2024-04-25T16:52:29.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.831+0000 7f27399fa640 1 -- 172.21.15.94:0/3687461356 wait complete. 2024-04-25T16:52:29.853 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T16:52:29.853 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:52:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.999+0000 7fec756df640 1 Processor -- start 2024-04-25T16:52:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.999+0000 7fec756df640 1 -- start start 2024-04-25T16:52:30.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.999+0000 7fec756df640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec700b74f0 0x7fec700b7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.999+0000 7fec756df640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec700d2120 con 0x7fec700cdcb0 2024-04-25T16:52:30.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.999+0000 7fec756df640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec700d2400 con 0x7fec70072e50 2024-04-25T16:52:30.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.999+0000 7fec756df640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fec700d26e0 con 0x7fec700b74f0 2024-04-25T16:52:30.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.999+0000 7fec6effd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fec700cdcb0 0x7fec70072740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56828/0 (socket says 172.21.15.94:56828) 2024-04-25T16:52:30.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:29.999+0000 7fec6effd640 1 -- 172.21.15.94:0/19474174 learned_addr learned my addr 172.21.15.94:0/19474174 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:30.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6f7fe640 1 --2- 172.21.15.94:0/19474174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec700b74f0 0x7fec700b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3462495454 0 0) 0x7fec700d2120 con 0x7fec700cdcb0 2024-04-25T16:52:30.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec5c003530 con 0x7fec700cdcb0 2024-04-25T16:52:30.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2097314077 0 0) 0x7fec5c003530 con 0x7fec700cdcb0 2024-04-25T16:52:30.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec700b74f0 msgr2=0x7fec700b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:30.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 --2- 172.21.15.94:0/19474174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec700b74f0 0x7fec700b7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 >> v1:172.21.15.138:6789/0 conn(0x7fec70072e50 legacy=0x7fec700b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec700b68c0 con 0x7fec700cdcb0 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fec60001680 con 0x7fec700cdcb0 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 >> v1:172.21.15.94:6789/0 conn(0x7fec700cdcb0 legacy=0x7fec70072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 --2- 172.21.15.94:0/19474174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec5c0039f0 0x7fec5c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 --2- 172.21.15.94:0/19474174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c004810 0x7fec5c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 --2- 172.21.15.94:0/19474174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec700b74f0 0x7fec5c0052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fec700d26e0 con 0x7fec5c0039f0 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c00bb90 con 0x7fec5c004810 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fec5c00bd90 con 0x7fec700b74f0 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6effd640 1 --2- 172.21.15.94:0/19474174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec5c0039f0 0x7fec5c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6f7fe640 1 --2- 172.21.15.94:0/19474174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec700b74f0 0x7fec5c0052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6effd640 1 -- 172.21.15.94:0/19474174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec700b74f0 msgr2=0x7fec5c0052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6effd640 1 --2- 172.21.15.94:0/19474174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec700b74f0 0x7fec5c0052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6effd640 1 -- 172.21.15.94:0/19474174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c004810 msgr2=0x7fec5c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:30.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6e7fc640 1 --2- 172.21.15.94:0/19474174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c004810 0x7fec5c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6effd640 1 --2- 172.21.15.94:0/19474174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c004810 0x7fec5c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6effd640 1 -- 172.21.15.94:0/19474174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5c003530 con 0x7fec5c0039f0 2024-04-25T16:52:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6e7fc640 1 --2- 172.21.15.94:0/19474174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c004810 0x7fec5c004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6effd640 1 --2- 172.21.15.94:0/19474174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec5c0039f0 0x7fec5c003e60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c240 tx=0x7fec6001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e488a0e7fa3692 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60007d20 con 0x7fec5c0039f0 2024-04-25T16:52:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec600a9070 con 0x7fec5c0039f0 2024-04-25T16:52:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6dffb640 1 -- 172.21.15.94:0/19474174 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec700b8110 con 0x7fec5c0039f0 2024-04-25T16:52:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec756df640 1 -- 172.21.15.94:0/19474174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec5c0039f0 msgr2=0x7fec5c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec756df640 1 --2- 172.21.15.94:0/19474174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec5c0039f0 0x7fec5c003e60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c240 tx=0x7fec6001cc40 comp rx=0 tx=0).stop 2024-04-25T16:52:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.003+0000 7fec6f7fe640 1 -- 172.21.15.94:0/19474174 reap_dead start 2024-04-25T16:52:30.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec756df640 1 -- 172.21.15.94:0/19474174 shutdown_connections 2024-04-25T16:52:30.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec756df640 1 -- 172.21.15.94:0/19474174 >> 172.21.15.94:0/19474174 conn(0x7fec700d3850 msgr2=0x7fec700d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:30.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec756df640 1 -- 172.21.15.94:0/19474174 shutdown_connections 2024-04-25T16:52:30.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec756df640 1 -- 172.21.15.94:0/19474174 wait complete. 2024-04-25T16:52:30.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec756df640 1 Processor -- start 2024-04-25T16:52:30.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec756df640 1 -- start start 2024-04-25T16:52:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec756df640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c0039f0 0x7fec70155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec756df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec700ca4d0 0x7fec701f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec756df640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec70155820 0x7fec700cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec756df640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fec700ce240 con 0x7fec700ca4d0 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec756df640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec700ce0c0 con 0x7fec5c0039f0 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec756df640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fec701696f0 con 0x7fec70155820 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec6e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec700ca4d0 0x7fec701f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec6e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec700ca4d0 0x7fec701f6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58956/0 (socket says 172.21.15.94:58956) 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec6e7fc640 1 -- 172.21.15.94:0/2021378316 learned_addr learned my addr 172.21.15.94:0/2021378316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec6effd640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c0039f0 0x7fec70155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec6f7fe640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec70155820 0x7fec700cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec6e7fc640 1 -- 172.21.15.94:0/2021378316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec70155820 msgr2=0x7fec700cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec6e7fc640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec70155820 0x7fec700cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec6e7fc640 1 -- 172.21.15.94:0/2021378316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c0039f0 msgr2=0x7fec70155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec6e7fc640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c0039f0 0x7fec70155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec6e7fc640 1 -- 172.21.15.94:0/2021378316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec700cabb0 con 0x7fec700ca4d0 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.007+0000 7fec6e7fc640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec700ca4d0 0x7fec701f6820 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fec5800d770 tx=0x7fec5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.011+0000 7fec677fe640 1 -- 172.21.15.94:0/2021378316 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58005270 con 0x7fec700ca4d0 2024-04-25T16:52:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.011+0000 7fec677fe640 1 -- 172.21.15.94:0/2021378316 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec58017070 con 0x7fec700ca4d0 2024-04-25T16:52:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.011+0000 7fec677fe640 1 -- 172.21.15.94:0/2021378316 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58003620 con 0x7fec700ca4d0 2024-04-25T16:52:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.011+0000 7fec756df640 1 -- 172.21.15.94:0/2021378316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec700cae90 con 0x7fec700ca4d0 2024-04-25T16:52:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.011+0000 7fec756df640 1 -- 172.21.15.94:0/2021378316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec700cb450 con 0x7fec700ca4d0 2024-04-25T16:52:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.011+0000 7fec756df640 1 -- 172.21.15.94:0/2021378316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7006a730 con 0x7fec700ca4d0 2024-04-25T16:52:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.011+0000 7fec677fe640 1 -- 172.21.15.94:0/2021378316 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fec580037c0 con 0x7fec700ca4d0 2024-04-25T16:52:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.011+0000 7fec677fe640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fec5006f430 0x7fec50071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.011+0000 7fec6effd640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fec5006f430 0x7fec50071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.011+0000 7fec677fe640 1 -- 172.21.15.94:0/2021378316 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fec5808f200 con 0x7fec700ca4d0 2024-04-25T16:52:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.011+0000 7fec6effd640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fec5006f430 0x7fec50071940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fec600ab000 tx=0x7fec600b1040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.015+0000 7fec677fe640 1 -- 172.21.15.94:0/2021378316 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fec5805bf60 con 0x7fec700ca4d0 2024-04-25T16:52:30.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.211+0000 7fec756df640 1 -- 172.21.15.94:0/2021378316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec700ca8e0 con 0x7fec700ca4d0 2024-04-25T16:52:30.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.215+0000 7fec677fe640 1 -- 172.21.15.94:0/2021378316 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5184 (secure 0 0 0) 0x7fec5805b860 con 0x7fec700ca4d0 2024-04-25T16:52:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:30.212 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/478421533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":478421533},{"type":"v1","addr":"172.21.15.138:6833","nonce":478421533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2461866660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2461866660},{"type":"v1","addr":"172.21.15.138:6835","nonce":2461866660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5439,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2566493915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2566493915},{"type":"v1","addr":"172.21.15.94:6835","nonce":2566493915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:52:26.175197+0000","modified":"2024-04-25T16:52:29.233625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5442":{"gid":5442,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/2235039329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2235039329},{"type":"v1","addr":"172.21.15.94:6837","nonce":2235039329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5442,"qdb_cluster":[5442]},"id":2}]} 2024-04-25T16:52:30.212 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-04-25T16:52:30.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.215+0000 7fec756df640 1 -- 172.21.15.94:0/2021378316 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fec5006f430 msgr2=0x7fec50071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.215+0000 7fec756df640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fec5006f430 0x7fec50071940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fec600ab000 tx=0x7fec600b1040 comp rx=0 tx=0).stop 2024-04-25T16:52:30.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.215+0000 7fec756df640 1 -- 172.21.15.94:0/2021378316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec700ca4d0 msgr2=0x7fec701f6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.215+0000 7fec756df640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec700ca4d0 0x7fec701f6820 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fec5800d770 tx=0x7fec5800dc30 comp rx=0 tx=0).stop 2024-04-25T16:52:30.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.215+0000 7fec756df640 1 -- 172.21.15.94:0/2021378316 shutdown_connections 2024-04-25T16:52:30.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.215+0000 7fec756df640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fec5006f430 0x7fec50071940 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.215+0000 7fec756df640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec70155820 0x7fec700cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.215+0000 7fec756df640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec700ca4d0 0x7fec701f6820 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.215+0000 7fec756df640 1 --2- 172.21.15.94:0/2021378316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c0039f0 0x7fec70155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.215+0000 7fec756df640 1 -- 172.21.15.94:0/2021378316 >> 172.21.15.94:0/2021378316 conn(0x7fec700d3850 msgr2=0x7fec70070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:30.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.215+0000 7fec756df640 1 -- 172.21.15.94:0/2021378316 shutdown_connections 2024-04-25T16:52:30.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.215+0000 7fec756df640 1 -- 172.21.15.94:0/2021378316 wait complete. 2024-04-25T16:52:30.233 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:52:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f582a9de640 1 Processor -- start 2024-04-25T16:52:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f582a9de640 1 -- start start 2024-04-25T16:52:30.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f582a9de640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f582415e990 0x7f5824165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f582a9de640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5824165b90 0x7f5824167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f582a9de640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f582406d610 con 0x7f5824070080 2024-04-25T16:52:30.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f582a9de640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f5824158550 con 0x7f5824165b90 2024-04-25T16:52:30.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f582a9de640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f582406f7d0 con 0x7f582415e990 2024-04-25T16:52:30.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f5828f7b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5824165b90 0x7f5824167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f5828f7b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5824165b90 0x7f5824167fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58970/0 (socket says 172.21.15.94:58970) 2024-04-25T16:52:30.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f5828f7b640 1 -- 172.21.15.94:0/89016383 learned_addr learned my addr 172.21.15.94:0/89016383 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:30.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f58237fe640 1 --2- 172.21.15.94:0/89016383 >> v2:172.21.15.138:3300/0 conn(0x7f582415e990 0x7f5824165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f5828f7b640 1 -- 172.21.15.94:0/89016383 >> v2:172.21.15.138:3300/0 conn(0x7f582415e990 msgr2=0x7f5824165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:30.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f5828f7b640 1 --2- 172.21.15.94:0/89016383 >> v2:172.21.15.138:3300/0 conn(0x7f582415e990 0x7f5824165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f5828f7b640 1 -- 172.21.15.94:0/89016383 >> v1:172.21.15.94:6789/0 conn(0x7f5824070080 legacy=0x7f582415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.387+0000 7f5828f7b640 1 -- 172.21.15.94:0/89016383 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582406fc60 con 0x7f5824165b90 2024-04-25T16:52:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.391+0000 7f5828f7b640 1 --2- 172.21.15.94:0/89016383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5824165b90 0x7f5824167fb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f58240b3e90 tx=0x7f581c01c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507075bb4cc253cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.391+0000 7f5822ffd640 1 -- 172.21.15.94:0/89016383 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c0a56a0 con 0x7f5824165b90 2024-04-25T16:52:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.391+0000 7f5822ffd640 1 -- 172.21.15.94:0/89016383 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f581c0a5840 con 0x7f5824165b90 2024-04-25T16:52:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.391+0000 7f5822ffd640 1 -- 172.21.15.94:0/89016383 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c0a5c10 con 0x7f5824165b90 2024-04-25T16:52:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.391+0000 7f582a9de640 1 -- 172.21.15.94:0/89016383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5824165b90 msgr2=0x7f5824167fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.391+0000 7f582a9de640 1 --2- 172.21.15.94:0/89016383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5824165b90 0x7f5824167fb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f58240b3e90 tx=0x7f581c01c860 comp rx=0 tx=0).stop 2024-04-25T16:52:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.391+0000 7f582a9de640 1 -- 172.21.15.94:0/89016383 shutdown_connections 2024-04-25T16:52:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.391+0000 7f582a9de640 1 --2- 172.21.15.94:0/89016383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5824165b90 0x7f5824167fb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.391+0000 7f582a9de640 1 --2- 172.21.15.94:0/89016383 >> v2:172.21.15.138:3300/0 conn(0x7f582415e990 0x7f5824165650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.391+0000 7f582a9de640 1 -- 172.21.15.94:0/89016383 >> 172.21.15.94:0/89016383 conn(0x7f58240b47f0 msgr2=0x7f58240b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.391+0000 7f582a9de640 1 -- 172.21.15.94:0/89016383 shutdown_connections 2024-04-25T16:52:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.391+0000 7f582a9de640 1 -- 172.21.15.94:0/89016383 wait complete. 2024-04-25T16:52:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f582a9de640 1 Processor -- start 2024-04-25T16:52:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f582a9de640 1 -- start start 2024-04-25T16:52:30.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f582a9de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5824070080 0x7f58240cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f582a9de640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f582415e990 0x7f582414dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f582a9de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5824165b90 0x7f582415f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f582a9de640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58240ae6c0 con 0x7f5824070080 2024-04-25T16:52:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f582a9de640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f582406f460 con 0x7f5824165b90 2024-04-25T16:52:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f582a9de640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5824070490 con 0x7f582415e990 2024-04-25T16:52:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5823fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5824070080 0x7f58240cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5823fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5824070080 0x7f58240cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58976/0 (socket says 172.21.15.94:58976) 2024-04-25T16:52:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5823fff640 1 -- 172.21.15.94:0/274150375 learned_addr learned my addr 172.21.15.94:0/274150375 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5828f7b640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5824165b90 0x7f582415f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f58237fe640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f582415e990 0x7f582414dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5823fff640 1 -- 172.21.15.94:0/274150375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f582415e990 msgr2=0x7f582414dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5823fff640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f582415e990 0x7f582414dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5823fff640 1 -- 172.21.15.94:0/274150375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5824165b90 msgr2=0x7f582415f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5823fff640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5824165b90 0x7f582415f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5823fff640 1 -- 172.21.15.94:0/274150375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58240b1880 con 0x7f5824070080 2024-04-25T16:52:30.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5828f7b640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5824165b90 0x7f582415f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:30.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5823fff640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5824070080 0x7f58240cd670 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f581400c710 tx=0x7f581400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:30.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5820ff9640 1 -- 172.21.15.94:0/274150375 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814017070 con 0x7f5824070080 2024-04-25T16:52:30.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5820ff9640 1 -- 172.21.15.94:0/274150375 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5814005480 con 0x7f5824070080 2024-04-25T16:52:30.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f5820ff9640 1 -- 172.21.15.94:0/274150375 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581400a880 con 0x7f5824070080 2024-04-25T16:52:30.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f582a9de640 1 -- 172.21.15.94:0/274150375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58241522f0 con 0x7f5824070080 2024-04-25T16:52:30.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f582a9de640 1 -- 172.21.15.94:0/274150375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5824152010 con 0x7f5824070080 2024-04-25T16:52:30.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.395+0000 7f582a9de640 1 -- 172.21.15.94:0/274150375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5824076830 con 0x7f5824070080 2024-04-25T16:52:30.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.399+0000 7f5820ff9640 1 -- 172.21.15.94:0/274150375 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f581401a070 con 0x7f5824070080 2024-04-25T16:52:30.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.399+0000 7f5820ff9640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f581006f430 0x7f5810071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.399+0000 7f5820ff9640 1 -- 172.21.15.94:0/274150375 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f581408e7f0 con 0x7f5824070080 2024-04-25T16:52:30.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.399+0000 7f58237fe640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f581006f430 0x7f5810071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.399+0000 7f58237fe640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f581006f430 0x7f5810071940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f58241516a0 tx=0x7f5818008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:30.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.403+0000 7f5820ff9640 1 -- 172.21.15.94:0/274150375 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f582406d610 con 0x7f5824070080 2024-04-25T16:52:30.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.551+0000 7f582a9de640 1 -- 172.21.15.94:0/274150375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5824075cc0 con 0x7f5824070080 2024-04-25T16:52:30.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.555+0000 7f5820ff9640 1 -- 172.21.15.94:0/274150375 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16954 (secure 0 0 0) 0x7f581400ce20 con 0x7f5824070080 2024-04-25T16:52:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:30.552 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":38,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:52:27.766947+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:52:23.840281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T16:52:25.130347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:52:30.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.555+0000 7f582a9de640 1 -- 172.21.15.94:0/274150375 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f581006f430 msgr2=0x7f5810071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.555+0000 7f582a9de640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f581006f430 0x7f5810071940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f58241516a0 tx=0x7f5818008040 comp rx=0 tx=0).stop 2024-04-25T16:52:30.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.555+0000 7f582a9de640 1 -- 172.21.15.94:0/274150375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5824070080 msgr2=0x7f58240cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.555+0000 7f582a9de640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5824070080 0x7f58240cd670 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f581400c710 tx=0x7f581400cbd0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.555+0000 7f582a9de640 1 -- 172.21.15.94:0/274150375 shutdown_connections 2024-04-25T16:52:30.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.555+0000 7f582a9de640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5824165b90 0x7f582415f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.555+0000 7f582a9de640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f581006f430 0x7f5810071940 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.555+0000 7f582a9de640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f582415e990 0x7f582414dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.555+0000 7f582a9de640 1 --2- 172.21.15.94:0/274150375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5824070080 0x7f58240cd670 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.555+0000 7f582a9de640 1 -- 172.21.15.94:0/274150375 >> 172.21.15.94:0/274150375 conn(0x7f58240b47f0 msgr2=0x7f5824167910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.555+0000 7f582a9de640 1 -- 172.21.15.94:0/274150375 shutdown_connections 2024-04-25T16:52:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.555+0000 7f582a9de640 1 -- 172.21.15.94:0/274150375 wait complete. 2024-04-25T16:52:30.573 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T16:52:30.573 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:52:30.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdbafd6640 1 Processor -- start 2024-04-25T16:52:30.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdbafd6640 1 -- start start 2024-04-25T16:52:30.722 INFO:tasks.ceph.mds.c.smithi094.stderr:2024-04-25T16:52:30.723+0000 7f3de5729640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T16:52:30.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdbafd6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbdb415e990 0x7fbdb4165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdbafd6640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbdb4165b90 0x7fbdb4167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdbafd6640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb406d610 con 0x7fbdb4070080 2024-04-25T16:52:30.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdbafd6640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb4158550 con 0x7fbdb4165b90 2024-04-25T16:52:30.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdbafd6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb406f7d0 con 0x7fbdb415e990 2024-04-25T16:52:30.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdb9573640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbdb4165b90 0x7fbdb4167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdb9573640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbdb4165b90 0x7fbdb4167fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58984/0 (socket says 172.21.15.94:58984) 2024-04-25T16:52:30.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdb9573640 1 -- 172.21.15.94:0/268617075 learned_addr learned my addr 172.21.15.94:0/268617075 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:30.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdb3fff640 1 --2- 172.21.15.94:0/268617075 >> v2:172.21.15.138:3300/0 conn(0x7fbdb415e990 0x7fbdb4165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdb9573640 1 -- 172.21.15.94:0/268617075 >> v2:172.21.15.138:3300/0 conn(0x7fbdb415e990 msgr2=0x7fbdb4165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:30.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdb9573640 1 --2- 172.21.15.94:0/268617075 >> v2:172.21.15.138:3300/0 conn(0x7fbdb415e990 0x7fbdb4165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdb9573640 1 -- 172.21.15.94:0/268617075 >> v1:172.21.15.138:6789/0 conn(0x7fbdb4070080 legacy=0x7fbdb415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:30.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.723+0000 7fbdb9573640 1 -- 172.21.15.94:0/268617075 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb406fc60 con 0x7fbdb4165b90 2024-04-25T16:52:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.727+0000 7fbdb9573640 1 --2- 172.21.15.94:0/268617075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb4165b90 0x7fbdb4167fb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40b3e90 tx=0x7fbdac01c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cbec1546c4baeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.727+0000 7fbdb37fe640 1 -- 172.21.15.94:0/268617075 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac0a5670 con 0x7fbdb4165b90 2024-04-25T16:52:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.727+0000 7fbdb37fe640 1 -- 172.21.15.94:0/268617075 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdac0a5810 con 0x7fbdb4165b90 2024-04-25T16:52:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.727+0000 7fbdb37fe640 1 -- 172.21.15.94:0/268617075 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac0a5be0 con 0x7fbdb4165b90 2024-04-25T16:52:30.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.727+0000 7fbdbafd6640 1 -- 172.21.15.94:0/268617075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb4165b90 msgr2=0x7fbdb4167fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.727+0000 7fbdbafd6640 1 --2- 172.21.15.94:0/268617075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb4165b90 0x7fbdb4167fb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40b3e90 tx=0x7fbdac01c7b0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.727+0000 7fbdbafd6640 1 -- 172.21.15.94:0/268617075 shutdown_connections 2024-04-25T16:52:30.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.727+0000 7fbdbafd6640 1 --2- 172.21.15.94:0/268617075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb4165b90 0x7fbdb4167fb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.727+0000 7fbdbafd6640 1 --2- 172.21.15.94:0/268617075 >> v2:172.21.15.138:3300/0 conn(0x7fbdb415e990 0x7fbdb4165650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.727+0000 7fbdbafd6640 1 -- 172.21.15.94:0/268617075 >> 172.21.15.94:0/268617075 conn(0x7fbdb40b47f0 msgr2=0x7fbdb40b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:30.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.727+0000 7fbdbafd6640 1 -- 172.21.15.94:0/268617075 shutdown_connections 2024-04-25T16:52:30.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.727+0000 7fbdbafd6640 1 -- 172.21.15.94:0/268617075 wait complete. 2024-04-25T16:52:30.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdbafd6640 1 Processor -- start 2024-04-25T16:52:30.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdbafd6640 1 -- start start 2024-04-25T16:52:30.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdbafd6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdb4070080 0x7fbdb40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdbafd6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb415e990 0x7fbdb414dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdbafd6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdb4165b90 0x7fbdb415f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdbafd6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb40ae6c0 con 0x7fbdb415e990 2024-04-25T16:52:30.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdbafd6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb406f460 con 0x7fbdb4070080 2024-04-25T16:52:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdbafd6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb4070490 con 0x7fbdb4165b90 2024-04-25T16:52:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb415e990 0x7fbdb414dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb415e990 0x7fbdb414dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58998/0 (socket says 172.21.15.94:58998) 2024-04-25T16:52:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb3fff640 1 -- 172.21.15.94:0/3606898977 learned_addr learned my addr 172.21.15.94:0/3606898977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb8d72640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdb4070080 0x7fbdb40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb9573640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdb4165b90 0x7fbdb415f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb3fff640 1 -- 172.21.15.94:0/3606898977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdb4165b90 msgr2=0x7fbdb415f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb3fff640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdb4165b90 0x7fbdb415f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb3fff640 1 -- 172.21.15.94:0/3606898977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdb4070080 msgr2=0x7fbdb40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb3fff640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdb4070080 0x7fbdb40cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb3fff640 1 -- 172.21.15.94:0/3606898977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb40b1880 con 0x7fbdb415e990 2024-04-25T16:52:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb3fff640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb415e990 0x7fbdb414dbe0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbda800d5d0 tx=0x7fbda800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb17fa640 1 -- 172.21.15.94:0/3606898977 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda80125f0 con 0x7fbdb415e990 2024-04-25T16:52:30.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb17fa640 1 -- 172.21.15.94:0/3606898977 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda8019070 con 0x7fbdb415e990 2024-04-25T16:52:30.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.731+0000 7fbdb17fa640 1 -- 172.21.15.94:0/3606898977 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda80129b0 con 0x7fbdb415e990 2024-04-25T16:52:30.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.735+0000 7fbdbafd6640 1 -- 172.21.15.94:0/3606898977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb41522f0 con 0x7fbdb415e990 2024-04-25T16:52:30.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.735+0000 7fbdbafd6640 1 -- 172.21.15.94:0/3606898977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb4152010 con 0x7fbdb415e990 2024-04-25T16:52:30.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.735+0000 7fbdbafd6640 1 -- 172.21.15.94:0/3606898977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb4076830 con 0x7fbdb415e990 2024-04-25T16:52:30.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.735+0000 7fbdb17fa640 1 -- 172.21.15.94:0/3606898977 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbda8012b50 con 0x7fbdb415e990 2024-04-25T16:52:30.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.735+0000 7fbdb17fa640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbda006f430 0x7fbda0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:30.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.735+0000 7fbdb8d72640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbda006f430 0x7fbda0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:30.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.735+0000 7fbdb17fa640 1 -- 172.21.15.94:0/3606898977 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fbda808f6d0 con 0x7fbdb415e990 2024-04-25T16:52:30.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.735+0000 7fbdb8d72640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbda006f430 0x7fbda0071940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40b50c0 tx=0x7fbda4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:30.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.739+0000 7fbdb17fa640 1 -- 172.21.15.94:0/3606898977 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbda805c550 con 0x7fbdb415e990 2024-04-25T16:52:30.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.899+0000 7fbdbafd6640 1 -- 172.21.15.94:0/3606898977 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbdb406db30 con 0x7fbda006f430 2024-04-25T16:52:30.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.915+0000 7fbdb17fa640 1 -- 172.21.15.94:0/3606898977 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240020 (secure 0 0 0) 0x7fbdb406db30 con 0x7fbda006f430 2024-04-25T16:52:30.918 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:30.920 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:52:30.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.923+0000 7fbdbafd6640 1 -- 172.21.15.94:0/3606898977 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbda006f430 msgr2=0x7fbda0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.923+0000 7fbdbafd6640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbda006f430 0x7fbda0071940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40b50c0 tx=0x7fbda4008040 comp rx=0 tx=0).stop 2024-04-25T16:52:30.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.923+0000 7fbdbafd6640 1 -- 172.21.15.94:0/3606898977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb415e990 msgr2=0x7fbdb414dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:30.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.923+0000 7fbdbafd6640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb415e990 0x7fbdb414dbe0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbda800d5d0 tx=0x7fbda800da90 comp rx=0 tx=0).stop 2024-04-25T16:52:30.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.923+0000 7fbdbafd6640 1 -- 172.21.15.94:0/3606898977 shutdown_connections 2024-04-25T16:52:30.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.923+0000 7fbdbafd6640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fbda006f430 0x7fbda0071940 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.923+0000 7fbdbafd6640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbdb4165b90 0x7fbdb415f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.923+0000 7fbdbafd6640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb415e990 0x7fbdb414dbe0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.923+0000 7fbdbafd6640 1 --2- 172.21.15.94:0/3606898977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbdb4070080 0x7fbdb40cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:30.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.923+0000 7fbdbafd6640 1 -- 172.21.15.94:0/3606898977 >> 172.21.15.94:0/3606898977 conn(0x7fbdb40b47f0 msgr2=0x7fbdb4167910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:30.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.923+0000 7fbdbafd6640 1 -- 172.21.15.94:0/3606898977 shutdown_connections 2024-04-25T16:52:30.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:30.923+0000 7fbdbafd6640 1 -- 172.21.15.94:0/3606898977 wait complete. 2024-04-25T16:52:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-04-25T16:52:29.693358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249576,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754725144,"statfs":{"total":773094113280,"available":772838547456,"internally_reserved":0,"allocated":4399104,"data_stored":2536752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.940818"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.755772+0000","last_change":"2024-04-25T16:52:24.755772+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.755772+0000","last_undegraded":"2024-04-25T16:52:24.755772+0000","last_fullsized":"2024-04-25T16:52:24.755772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.759549+0000","last_change":"2024-04-25T16:52:24.759549+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.759549+0000","last_undegraded":"2024-04-25T16:52:24.759549+0000","last_fullsized":"2024-04-25T16:52:24.759549+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757571+0000","last_change":"2024-04-25T16:52:24.757571+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757571+0000","last_undegraded":"2024-04-25T16:52:24.757571+0000","last_fullsized":"2024-04-25T16:52:24.757571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.762520+0000","last_change":"2024-04-25T16:52:24.762520+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.762520+0000","last_undegraded":"2024-04-25T16:52:24.762520+0000","last_fullsized":"2024-04-25T16:52:24.762520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.755757+0000","last_change":"2024-04-25T16:52:24.755757+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.755757+0000","last_undegraded":"2024-04-25T16:52:24.755757+0000","last_fullsized":"2024-04-25T16:52:24.755757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.763524+0000","last_change":"2024-04-25T16:52:24.763524+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.763524+0000","last_undegraded":"2024-04-25T16:52:24.763524+0000","last_fullsized":"2024-04-25T16:52:24.763524+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757570+0000","last_change":"2024-04-25T16:52:24.757570+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757570+0000","last_undegraded":"2024-04-25T16:52:24.757570+0000","last_fullsized":"2024-04-25T16:52:24.757570+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.762412+0000","last_change":"2024-04-25T16:52:24.762412+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.762412+0000","last_undegraded":"2024-04-25T16:52:24.762412+0000","last_fullsized":"2024-04-25T16:52:24.762412+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.760991+0000","last_change":"2024-04-25T16:52:24.760991+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.760991+0000","last_undegraded":"2024-04-25T16:52:24.760991+0000","last_fullsized":"2024-04-25T16:52:24.760991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757037+0000","last_change":"2024-04-25T16:52:24.757037+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757037+0000","last_undegraded":"2024-04-25T16:52:24.757037+0000","last_fullsized":"2024-04-25T16:52:24.757037+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.755751+0000","last_change":"2024-04-25T16:52:24.755751+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.755751+0000","last_undegraded":"2024-04-25T16:52:24.755751+0000","last_fullsized":"2024-04-25T16:52:24.755751+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.762054+0000","last_change":"2024-04-25T16:52:24.762054+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.762054+0000","last_undegraded":"2024-04-25T16:52:24.762054+0000","last_fullsized":"2024-04-25T16:52:24.762054+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.761704+0000","last_change":"2024-04-25T16:52:24.761704+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.761704+0000","last_undegraded":"2024-04-25T16:52:24.761704+0000","last_fullsized":"2024-04-25T16:52:24.761704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757937+0000","last_change":"2024-04-25T16:52:24.757937+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757937+0000","last_undegraded":"2024-04-25T16:52:24.757937+0000","last_fullsized":"2024-04-25T16:52:24.757937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757321+0000","last_change":"2024-04-25T16:52:24.757321+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757321+0000","last_undegraded":"2024-04-25T16:52:24.757321+0000","last_fullsized":"2024-04-25T16:52:24.757321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.764879+0000","last_change":"2024-04-25T16:52:24.764879+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.764879+0000","last_undegraded":"2024-04-25T16:52:24.764879+0000","last_fullsized":"2024-04-25T16:52:24.764879+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757170+0000","last_change":"2024-04-25T16:52:24.757170+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757170+0000","last_undegraded":"2024-04-25T16:52:24.757170+0000","last_fullsized":"2024-04-25T16:52:24.757170+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.762409+0000","last_change":"2024-04-25T16:52:24.762409+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.762409+0000","last_undegraded":"2024-04-25T16:52:24.762409+0000","last_fullsized":"2024-04-25T16:52:24.762409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.769465+0000","last_change":"2024-04-25T16:52:24.769465+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.769465+0000","last_undegraded":"2024-04-25T16:52:24.769465+0000","last_fullsized":"2024-04-25T16:52:24.769465+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757539+0000","last_change":"2024-04-25T16:52:24.757539+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757539+0000","last_undegraded":"2024-04-25T16:52:24.757539+0000","last_fullsized":"2024-04-25T16:52:24.757539+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.769427+0000","last_change":"2024-04-25T16:52:24.769427+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.769427+0000","last_undegraded":"2024-04-25T16:52:24.769427+0000","last_fullsized":"2024-04-25T16:52:24.769427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.769427+0000","last_change":"2024-04-25T16:52:24.769427+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.769427+0000","last_undegraded":"2024-04-25T16:52:24.769427+0000","last_fullsized":"2024-04-25T16:52:24.769427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.764235+0000","last_change":"2024-04-25T16:52:24.764235+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.764235+0000","last_undegraded":"2024-04-25T16:52:24.764235+0000","last_fullsized":"2024-04-25T16:52:24.764235+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.769701+0000","last_change":"2024-04-25T16:52:24.769701+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.769701+0000","last_undegraded":"2024-04-25T16:52:24.769701+0000","last_fullsized":"2024-04-25T16:52:24.769701+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.755786+0000","last_change":"2024-04-25T16:52:24.755786+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.755786+0000","last_undegraded":"2024-04-25T16:52:24.755786+0000","last_fullsized":"2024-04-25T16:52:24.755786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.769366+0000","last_change":"2024-04-25T16:52:24.769366+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.769366+0000","last_undegraded":"2024-04-25T16:52:24.769366+0000","last_fullsized":"2024-04-25T16:52:24.769366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.759671+0000","last_change":"2024-04-25T16:52:24.759671+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.759671+0000","last_undegraded":"2024-04-25T16:52:24.759671+0000","last_fullsized":"2024-04-25T16:52:24.759671+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.759677+0000","last_change":"2024-04-25T16:52:24.759677+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.759677+0000","last_undegraded":"2024-04-25T16:52:24.759677+0000","last_fullsized":"2024-04-25T16:52:24.759677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.758781+0000","last_change":"2024-04-25T16:52:24.758781+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.758781+0000","last_undegraded":"2024-04-25T16:52:24.758781+0000","last_fullsized":"2024-04-25T16:52:24.758781+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.762183+0000","last_change":"2024-04-25T16:52:24.762183+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.762183+0000","last_undegraded":"2024-04-25T16:52:24.762183+0000","last_fullsized":"2024-04-25T16:52:24.762183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.762044+0000","last_change":"2024-04-25T16:52:24.762044+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.762044+0000","last_undegraded":"2024-04-25T16:52:24.762044+0000","last_fullsized":"2024-04-25T16:52:24.762044+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.764416+0000","last_change":"2024-04-25T16:52:24.764416+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.764416+0000","last_undegraded":"2024-04-25T16:52:24.764416+0000","last_fullsized":"2024-04-25T16:52:24.764416+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":137,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T16:52:24.766850+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:52:24.766850+0000","last_peered":"2024-04-25T16:52:24.766850+0000","last_clean":"2024-04-25T16:52:24.766850+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:52:24.766850+0000","last_undegraded":"2024-04-25T16:52:24.766850+0000","last_fullsized":"2024-04-25T16:52:24.766850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":759870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35732,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336108,"statfs":{"total":96636764160,"available":96600174592,"internally_reserved":0,"allocated":401408,"data_stored":169502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":995328,"data_stored":759870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:52:30.945 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:52:31.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.131+0000 7ff654306640 1 Processor -- start 2024-04-25T16:52:31.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.131+0000 7ff654306640 1 -- start start 2024-04-25T16:52:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff654306640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff64c15d640 0x7ff64c160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff654306640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff64c0d0ef0 con 0x7ff64c0b57b0 2024-04-25T16:52:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff654306640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff64c0d11d0 con 0x7ff64c0cf2c0 2024-04-25T16:52:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff654306640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff64c0d14b0 con 0x7ff64c15d640 2024-04-25T16:52:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6528a3640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ff64c0b57b0 0x7ff64c0b5bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56836/0 (socket says 172.21.15.94:56836) 2024-04-25T16:52:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6528a3640 1 -- 172.21.15.94:0/2066002560 learned_addr learned my addr 172.21.15.94:0/2066002560 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6520a2640 1 --2- 172.21.15.94:0/2066002560 >> v2:172.21.15.138:3300/0 conn(0x7ff64c15d640 0x7ff64c160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2939680738 0 0) 0x7ff64c0d0ef0 con 0x7ff64c0b57b0 2024-04-25T16:52:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff638003530 con 0x7ff64c0b57b0 2024-04-25T16:52:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 86602883 0 0) 0x7ff638003530 con 0x7ff64c0b57b0 2024-04-25T16:52:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 >> v2:172.21.15.138:3300/0 conn(0x7ff64c15d640 msgr2=0x7ff64c160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 --2- 172.21.15.94:0/2066002560 >> v2:172.21.15.138:3300/0 conn(0x7ff64c15d640 0x7ff64c160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 >> v1:172.21.15.138:6789/0 conn(0x7ff64c0cf2c0 legacy=0x7ff64c0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c0d1790 con 0x7ff64c0b57b0 2024-04-25T16:52:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7ff644001a00 con 0x7ff64c0b57b0 2024-04-25T16:52:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 >> v1:172.21.15.94:6789/0 conn(0x7ff64c0b57b0 legacy=0x7ff64c0b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 --2- 172.21.15.94:0/2066002560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6380039c0 0x7ff638003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 --2- 172.21.15.94:0/2066002560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff638004810 0x7ff638004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 --2- 172.21.15.94:0/2066002560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff638005360 0x7ff63800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c0d14b0 con 0x7ff638005360 2024-04-25T16:52:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff63800c290 con 0x7ff6380039c0 2024-04-25T16:52:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff63800c490 con 0x7ff638004810 2024-04-25T16:52:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6528a3640 1 --2- 172.21.15.94:0/2066002560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff638005360 0x7ff63800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6520a2640 1 --2- 172.21.15.94:0/2066002560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6380039c0 0x7ff638003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6528a3640 1 -- 172.21.15.94:0/2066002560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff638004810 msgr2=0x7ff638004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6528a3640 1 --2- 172.21.15.94:0/2066002560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff638004810 0x7ff638004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6528a3640 1 -- 172.21.15.94:0/2066002560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6380039c0 msgr2=0x7ff638003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6528a3640 1 --2- 172.21.15.94:0/2066002560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6380039c0 0x7ff638003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6528a3640 1 -- 172.21.15.94:0/2066002560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff638003530 con 0x7ff638005360 2024-04-25T16:52:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.135+0000 7ff6520a2640 1 --2- 172.21.15.94:0/2066002560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6380039c0 0x7ff638003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.139+0000 7ff6528a3640 1 -- 172.21.15.94:0/2066002560 reap_dead start 2024-04-25T16:52:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.139+0000 7ff6528a3640 1 --2- 172.21.15.94:0/2066002560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff638005360 0x7ff63800b9d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff64400a840 tx=0x7ff64401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a5113a2d8b060a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:31.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.139+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64400a460 con 0x7ff638005360 2024-04-25T16:52:31.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.139+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6440ac640 con 0x7ff638005360 2024-04-25T16:52:31.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.139+0000 7ff6510a0640 1 -- 172.21.15.94:0/2066002560 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c0b6ae0 con 0x7ff638005360 2024-04-25T16:52:31.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.139+0000 7ff654306640 1 -- 172.21.15.94:0/2066002560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff638005360 msgr2=0x7ff63800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.139+0000 7ff654306640 1 --2- 172.21.15.94:0/2066002560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff638005360 0x7ff63800b9d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff64400a840 tx=0x7ff64401d360 comp rx=0 tx=0).stop 2024-04-25T16:52:31.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.139+0000 7ff654306640 1 -- 172.21.15.94:0/2066002560 shutdown_connections 2024-04-25T16:52:31.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.139+0000 7ff654306640 1 --2- 172.21.15.94:0/2066002560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff638005360 0x7ff63800b9d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.139+0000 7ff654306640 1 -- 172.21.15.94:0/2066002560 >> 172.21.15.94:0/2066002560 conn(0x7ff64c071150 msgr2=0x7ff64c071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:31.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.139+0000 7ff654306640 1 -- 172.21.15.94:0/2066002560 shutdown_connections 2024-04-25T16:52:31.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.139+0000 7ff654306640 1 -- 172.21.15.94:0/2066002560 wait complete. 2024-04-25T16:52:31.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.139+0000 7ff654306640 1 Processor -- start 2024-04-25T16:52:31.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff654306640 1 -- start start 2024-04-25T16:52:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff654306640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff638005360 0x7ff64c06d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff654306640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff64c06faa0 0x7ff64c15cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff654306640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff64c1623f0 0x7ff64c162800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff654306640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c160f50 con 0x7ff64c06faa0 2024-04-25T16:52:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff654306640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c160dd0 con 0x7ff64c1623f0 2024-04-25T16:52:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff654306640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff64c0b5bc0 con 0x7ff638005360 2024-04-25T16:52:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff6518a1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff64c06faa0 0x7ff64c15cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff6518a1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff64c06faa0 0x7ff64c15cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59010/0 (socket says 172.21.15.94:59010) 2024-04-25T16:52:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff6518a1640 1 -- 172.21.15.94:0/3821162995 learned_addr learned my addr 172.21.15.94:0/3821162995 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff6520a2640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff638005360 0x7ff64c06d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff6528a3640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff64c1623f0 0x7ff64c162800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff6518a1640 1 -- 172.21.15.94:0/3821162995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff638005360 msgr2=0x7ff64c06d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff6518a1640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff638005360 0x7ff64c06d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff6518a1640 1 -- 172.21.15.94:0/3821162995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff64c1623f0 msgr2=0x7ff64c162800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff6518a1640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff64c1623f0 0x7ff64c162800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff6518a1640 1 -- 172.21.15.94:0/3821162995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c162d40 con 0x7ff64c06faa0 2024-04-25T16:52:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff6518a1640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff64c06faa0 0x7ff64c15cee0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00c500 tx=0x7ff63c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff642ffd640 1 -- 172.21.15.94:0/3821162995 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c017070 con 0x7ff64c06faa0 2024-04-25T16:52:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff642ffd640 1 -- 172.21.15.94:0/3821162995 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff63c0052d0 con 0x7ff64c06faa0 2024-04-25T16:52:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff654306640 1 -- 172.21.15.94:0/3821162995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64c06feb0 con 0x7ff64c06faa0 2024-04-25T16:52:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff642ffd640 1 -- 172.21.15.94:0/3821162995 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c003840 con 0x7ff64c06faa0 2024-04-25T16:52:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff654306640 1 -- 172.21.15.94:0/3821162995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64c06e0d0 con 0x7ff64c06faa0 2024-04-25T16:52:31.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff654306640 1 -- 172.21.15.94:0/3821162995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64c06edb0 con 0x7ff64c06faa0 2024-04-25T16:52:31.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.143+0000 7ff642ffd640 1 -- 172.21.15.94:0/3821162995 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff63c01a070 con 0x7ff64c06faa0 2024-04-25T16:52:31.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.147+0000 7ff642ffd640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff63006f430 0x7ff630071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.147+0000 7ff642ffd640 1 -- 172.21.15.94:0/3821162995 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7ff63c08e7b0 con 0x7ff64c06faa0 2024-04-25T16:52:31.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.147+0000 7ff6520a2640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff63006f430 0x7ff630071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.147+0000 7ff6520a2640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff63006f430 0x7ff630071940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0d07d0 tx=0x7ff648008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:31.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.151+0000 7ff642ffd640 1 -- 172.21.15.94:0/3821162995 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff64c162d40 con 0x7ff64c06faa0 2024-04-25T16:52:31.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.303+0000 7ff654306640 1 -- 172.21.15.94:0/3821162995 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff64c06a730 con 0x7ff63006f430 2024-04-25T16:52:31.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.311+0000 7ff642ffd640 1 -- 172.21.15.94:0/3821162995 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240020 (secure 0 0 0) 0x7ff64c06a730 con 0x7ff63006f430 2024-04-25T16:52:31.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:31.311 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:52:31.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.311+0000 7ff654306640 1 -- 172.21.15.94:0/3821162995 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff63006f430 msgr2=0x7ff630071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.311+0000 7ff654306640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff63006f430 0x7ff630071940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0d07d0 tx=0x7ff648008040 comp rx=0 tx=0).stop 2024-04-25T16:52:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.311+0000 7ff654306640 1 -- 172.21.15.94:0/3821162995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff64c06faa0 msgr2=0x7ff64c15cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.311+0000 7ff654306640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff64c06faa0 0x7ff64c15cee0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00c500 tx=0x7ff63c00c9c0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.311+0000 7ff654306640 1 -- 172.21.15.94:0/3821162995 shutdown_connections 2024-04-25T16:52:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.311+0000 7ff654306640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff64c1623f0 0x7ff64c162800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.311+0000 7ff654306640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff64c06faa0 0x7ff64c15cee0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.311+0000 7ff654306640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff63006f430 0x7ff630071940 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.311+0000 7ff654306640 1 --2- 172.21.15.94:0/3821162995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff638005360 0x7ff64c06d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.311+0000 7ff654306640 1 -- 172.21.15.94:0/3821162995 >> 172.21.15.94:0/3821162995 conn(0x7ff64c071150 msgr2=0x7ff64c0738f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.311+0000 7ff654306640 1 -- 172.21.15.94:0/3821162995 shutdown_connections 2024-04-25T16:52:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.315+0000 7ff654306640 1 -- 172.21.15.94:0/3821162995 wait complete. 2024-04-25T16:52:31.329 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-04-25T16:52:29.693358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249576,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754725144,"statfs":{"total":773094113280,"available":772838547456,"internally_reserved":0,"allocated":4399104,"data_stored":2536752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.940818"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.755772+0000","last_change":"2024-04-25T16:52:24.755772+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.755772+0000","last_undegraded":"2024-04-25T16:52:24.755772+0000","last_fullsized":"2024-04-25T16:52:24.755772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.759549+0000","last_change":"2024-04-25T16:52:24.759549+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.759549+0000","last_undegraded":"2024-04-25T16:52:24.759549+0000","last_fullsized":"2024-04-25T16:52:24.759549+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757571+0000","last_change":"2024-04-25T16:52:24.757571+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757571+0000","last_undegraded":"2024-04-25T16:52:24.757571+0000","last_fullsized":"2024-04-25T16:52:24.757571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.762520+0000","last_change":"2024-04-25T16:52:24.762520+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.762520+0000","last_undegraded":"2024-04-25T16:52:24.762520+0000","last_fullsized":"2024-04-25T16:52:24.762520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.755757+0000","last_change":"2024-04-25T16:52:24.755757+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.755757+0000","last_undegraded":"2024-04-25T16:52:24.755757+0000","last_fullsized":"2024-04-25T16:52:24.755757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.763524+0000","last_change":"2024-04-25T16:52:24.763524+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.763524+0000","last_undegraded":"2024-04-25T16:52:24.763524+0000","last_fullsized":"2024-04-25T16:52:24.763524+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757570+0000","last_change":"2024-04-25T16:52:24.757570+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757570+0000","last_undegraded":"2024-04-25T16:52:24.757570+0000","last_fullsized":"2024-04-25T16:52:24.757570+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.762412+0000","last_change":"2024-04-25T16:52:24.762412+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.762412+0000","last_undegraded":"2024-04-25T16:52:24.762412+0000","last_fullsized":"2024-04-25T16:52:24.762412+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.760991+0000","last_change":"2024-04-25T16:52:24.760991+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.760991+0000","last_undegraded":"2024-04-25T16:52:24.760991+0000","last_fullsized":"2024-04-25T16:52:24.760991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757037+0000","last_change":"2024-04-25T16:52:24.757037+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757037+0000","last_undegraded":"2024-04-25T16:52:24.757037+0000","last_fullsized":"2024-04-25T16:52:24.757037+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.755751+0000","last_change":"2024-04-25T16:52:24.755751+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.755751+0000","last_undegraded":"2024-04-25T16:52:24.755751+0000","last_fullsized":"2024-04-25T16:52:24.755751+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.762054+0000","last_change":"2024-04-25T16:52:24.762054+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.762054+0000","last_undegraded":"2024-04-25T16:52:24.762054+0000","last_fullsized":"2024-04-25T16:52:24.762054+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.761704+0000","last_change":"2024-04-25T16:52:24.761704+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.761704+0000","last_undegraded":"2024-04-25T16:52:24.761704+0000","last_fullsized":"2024-04-25T16:52:24.761704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757937+0000","last_change":"2024-04-25T16:52:24.757937+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757937+0000","last_undegraded":"2024-04-25T16:52:24.757937+0000","last_fullsized":"2024-04-25T16:52:24.757937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757321+0000","last_change":"2024-04-25T16:52:24.757321+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757321+0000","last_undegraded":"2024-04-25T16:52:24.757321+0000","last_fullsized":"2024-04-25T16:52:24.757321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.764879+0000","last_change":"2024-04-25T16:52:24.764879+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.764879+0000","last_undegraded":"2024-04-25T16:52:24.764879+0000","last_fullsized":"2024-04-25T16:52:24.764879+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757170+0000","last_change":"2024-04-25T16:52:24.757170+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757170+0000","last_undegraded":"2024-04-25T16:52:24.757170+0000","last_fullsized":"2024-04-25T16:52:24.757170+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.762409+0000","last_change":"2024-04-25T16:52:24.762409+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.762409+0000","last_undegraded":"2024-04-25T16:52:24.762409+0000","last_fullsized":"2024-04-25T16:52:24.762409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.769465+0000","last_change":"2024-04-25T16:52:24.769465+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.769465+0000","last_undegraded":"2024-04-25T16:52:24.769465+0000","last_fullsized":"2024-04-25T16:52:24.769465+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.757539+0000","last_change":"2024-04-25T16:52:24.757539+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.757539+0000","last_undegraded":"2024-04-25T16:52:24.757539+0000","last_fullsized":"2024-04-25T16:52:24.757539+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.769427+0000","last_change":"2024-04-25T16:52:24.769427+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.769427+0000","last_undegraded":"2024-04-25T16:52:24.769427+0000","last_fullsized":"2024-04-25T16:52:24.769427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.769427+0000","last_change":"2024-04-25T16:52:24.769427+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.769427+0000","last_undegraded":"2024-04-25T16:52:24.769427+0000","last_fullsized":"2024-04-25T16:52:24.769427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.764235+0000","last_change":"2024-04-25T16:52:24.764235+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.764235+0000","last_undegraded":"2024-04-25T16:52:24.764235+0000","last_fullsized":"2024-04-25T16:52:24.764235+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.769701+0000","last_change":"2024-04-25T16:52:24.769701+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.769701+0000","last_undegraded":"2024-04-25T16:52:24.769701+0000","last_fullsized":"2024-04-25T16:52:24.769701+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.755786+0000","last_change":"2024-04-25T16:52:24.755786+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.755786+0000","last_undegraded":"2024-04-25T16:52:24.755786+0000","last_fullsized":"2024-04-25T16:52:24.755786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.769366+0000","last_change":"2024-04-25T16:52:24.769366+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.769366+0000","last_undegraded":"2024-04-25T16:52:24.769366+0000","last_fullsized":"2024-04-25T16:52:24.769366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.759671+0000","last_change":"2024-04-25T16:52:24.759671+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.759671+0000","last_undegraded":"2024-04-25T16:52:24.759671+0000","last_fullsized":"2024-04-25T16:52:24.759671+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.759677+0000","last_change":"2024-04-25T16:52:24.759677+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.759677+0000","last_undegraded":"2024-04-25T16:52:24.759677+0000","last_fullsized":"2024-04-25T16:52:24.759677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.758781+0000","last_change":"2024-04-25T16:52:24.758781+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.758781+0000","last_undegraded":"2024-04-25T16:52:24.758781+0000","last_fullsized":"2024-04-25T16:52:24.758781+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.762183+0000","last_change":"2024-04-25T16:52:24.762183+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.762183+0000","last_undegraded":"2024-04-25T16:52:24.762183+0000","last_fullsized":"2024-04-25T16:52:24.762183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.762044+0000","last_change":"2024-04-25T16:52:24.762044+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.762044+0000","last_undegraded":"2024-04-25T16:52:24.762044+0000","last_fullsized":"2024-04-25T16:52:24.762044+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T16:52:25.752315+0000","last_change":"2024-04-25T16:52:25.752315+0000","last_active":"2024-04-25T16:52:25.752315+0000","last_peered":"2024-04-25T16:52:25.752315+0000","last_clean":"2024-04-25T16:52:25.752315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:25.752315+0000","last_undegraded":"2024-04-25T16:52:25.752315+0000","last_fullsized":"2024-04-25T16:52:25.752315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-25T16:52:24.764416+0000","last_change":"2024-04-25T16:52:24.764416+0000","last_active":"2024-04-25T16:52:24.741701+0000","last_peered":"2024-04-25T16:52:24.741701+0000","last_clean":"2024-04-25T16:52:24.741701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T16:52:24.764416+0000","last_undegraded":"2024-04-25T16:52:24.764416+0000","last_fullsized":"2024-04-25T16:52:24.764416+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":137,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T16:52:24.766850+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:52:24.766850+0000","last_peered":"2024-04-25T16:52:24.766850+0000","last_clean":"2024-04-25T16:52:24.766850+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:52:24.766850+0000","last_undegraded":"2024-04-25T16:52:24.766850+0000","last_fullsized":"2024-04-25T16:52:24.766850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":759870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35732,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336108,"statfs":{"total":96636764160,"available":96600174592,"internally_reserved":0,"allocated":401408,"data_stored":169502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":995328,"data_stored":759870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:52:31.333 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:52:31.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd358220640 1 Processor -- start 2024-04-25T16:52:31.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd358220640 1 -- start start 2024-04-25T16:52:31.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd358220640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd350156790 0x7fd3500738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd358220640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd35006fbe0 0x7fd35006fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd358220640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd350070530 con 0x7fd350072740 2024-04-25T16:52:31.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd358220640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd350156ba0 con 0x7fd35006fbe0 2024-04-25T16:52:31.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd358220640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd35015fd80 con 0x7fd350156790 2024-04-25T16:52:31.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd3557bb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd35006fbe0 0x7fd35006fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd3557bb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd35006fbe0 0x7fd35006fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59024/0 (socket says 172.21.15.94:59024) 2024-04-25T16:52:31.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd3557bb640 1 -- 172.21.15.94:0/604058896 learned_addr learned my addr 172.21.15.94:0/604058896 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd355fbc640 1 --2- 172.21.15.94:0/604058896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd350156790 0x7fd3500738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd3557bb640 1 -- 172.21.15.94:0/604058896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd350156790 msgr2=0x7fd3500738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd3557bb640 1 --2- 172.21.15.94:0/604058896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd350156790 0x7fd3500738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd3557bb640 1 -- 172.21.15.94:0/604058896 >> v1:172.21.15.138:6789/0 conn(0x7fd350072740 legacy=0x7fd350072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd3557bb640 1 -- 172.21.15.94:0/604058896 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd350160060 con 0x7fd35006fbe0 2024-04-25T16:52:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.523+0000 7fd355fbc640 1 --2- 172.21.15.94:0/604058896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd350156790 0x7fd3500738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.527+0000 7fd3557bb640 1 --2- 172.21.15.94:0/604058896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd35006fbe0 0x7fd35006fff0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd340009870 tx=0x7fd34001c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b5169afaeb9bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:31.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.527+0000 7fd354fba640 1 -- 172.21.15.94:0/604058896 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3400b0070 con 0x7fd35006fbe0 2024-04-25T16:52:31.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.527+0000 7fd354fba640 1 -- 172.21.15.94:0/604058896 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3400ab600 con 0x7fd35006fbe0 2024-04-25T16:52:31.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.527+0000 7fd354fba640 1 -- 172.21.15.94:0/604058896 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3400ab9d0 con 0x7fd35006fbe0 2024-04-25T16:52:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.527+0000 7fd358220640 1 -- 172.21.15.94:0/604058896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd35006fbe0 msgr2=0x7fd35006fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.527+0000 7fd358220640 1 --2- 172.21.15.94:0/604058896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd35006fbe0 0x7fd35006fff0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd340009870 tx=0x7fd34001c2b0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.527+0000 7fd358220640 1 -- 172.21.15.94:0/604058896 shutdown_connections 2024-04-25T16:52:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.527+0000 7fd358220640 1 --2- 172.21.15.94:0/604058896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd35006fbe0 0x7fd35006fff0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.527+0000 7fd358220640 1 --2- 172.21.15.94:0/604058896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd350156790 0x7fd3500738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.527+0000 7fd358220640 1 -- 172.21.15.94:0/604058896 >> 172.21.15.94:0/604058896 conn(0x7fd3500b47f0 msgr2=0x7fd3500b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.527+0000 7fd358220640 1 -- 172.21.15.94:0/604058896 shutdown_connections 2024-04-25T16:52:31.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.527+0000 7fd358220640 1 -- 172.21.15.94:0/604058896 wait complete. 2024-04-25T16:52:31.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd358220640 1 Processor -- start 2024-04-25T16:52:31.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd358220640 1 -- start start 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd358220640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd35006fbe0 0x7fd3501472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd358220640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd350072740 0x7fd3500b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd358220640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd350156790 0x7fd3500b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd358220640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3500afaf0 con 0x7fd35006fbe0 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd358220640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd350074100 con 0x7fd350072740 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd358220640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd3500afc70 con 0x7fd350156790 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd355fbc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd35006fbe0 0x7fd3501472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd355fbc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd35006fbe0 0x7fd3501472e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59040/0 (socket says 172.21.15.94:59040) 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd355fbc640 1 -- 172.21.15.94:0/2059070582 learned_addr learned my addr 172.21.15.94:0/2059070582 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd3567bd640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd350156790 0x7fd3500b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd355fbc640 1 -- 172.21.15.94:0/2059070582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd350156790 msgr2=0x7fd3500b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd3557bb640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd350072740 0x7fd3500b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd355fbc640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd350156790 0x7fd3500b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd355fbc640 1 -- 172.21.15.94:0/2059070582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd350072740 msgr2=0x7fd3500b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd355fbc640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd350072740 0x7fd3500b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd355fbc640 1 -- 172.21.15.94:0/2059070582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd350168d40 con 0x7fd35006fbe0 2024-04-25T16:52:31.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd3567bd640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd350156790 0x7fd3500b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:31.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd355fbc640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd35006fbe0 0x7fd3501472e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00c7f0 tx=0x7fd34c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:31.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd3467fc640 1 -- 172.21.15.94:0/2059070582 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c015070 con 0x7fd35006fbe0 2024-04-25T16:52:31.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd3467fc640 1 -- 172.21.15.94:0/2059070582 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c007820 con 0x7fd35006fbe0 2024-04-25T16:52:31.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd358220640 1 -- 172.21.15.94:0/2059070582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd350168780 con 0x7fd35006fbe0 2024-04-25T16:52:31.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd3467fc640 1 -- 172.21.15.94:0/2059070582 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c007bd0 con 0x7fd35006fbe0 2024-04-25T16:52:31.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd358220640 1 -- 172.21.15.94:0/2059070582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35016c7e0 con 0x7fd35006fbe0 2024-04-25T16:52:31.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.531+0000 7fd358220640 1 -- 172.21.15.94:0/2059070582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd350073850 con 0x7fd35006fbe0 2024-04-25T16:52:31.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.535+0000 7fd3467fc640 1 -- 172.21.15.94:0/2059070582 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd34c018070 con 0x7fd35006fbe0 2024-04-25T16:52:31.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.535+0000 7fd3467fc640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd33406f400 0x7fd334071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.535+0000 7fd3467fc640 1 -- 172.21.15.94:0/2059070582 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fd34c08f580 con 0x7fd35006fbe0 2024-04-25T16:52:31.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.535+0000 7fd3557bb640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd33406f400 0x7fd334071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.535+0000 7fd3557bb640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd33406f400 0x7fd334071910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd3400ad000 tx=0x7fd340008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:31.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.539+0000 7fd3467fc640 1 -- 172.21.15.94:0/2059070582 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd34c05c370 con 0x7fd35006fbe0 2024-04-25T16:52:31.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.695+0000 7fd358220640 1 -- 172.21.15.94:0/2059070582 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd350076830 con 0x7fd33406f400 2024-04-25T16:52:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.715+0000 7fd3467fc640 1 -- 172.21.15.94:0/2059070582 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7fd350076830 con 0x7fd33406f400 2024-04-25T16:52:31.715 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:52:31.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.719+0000 7fd358220640 1 -- 172.21.15.94:0/2059070582 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd33406f400 msgr2=0x7fd334071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.719+0000 7fd358220640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd33406f400 0x7fd334071910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd3400ad000 tx=0x7fd340008040 comp rx=0 tx=0).stop 2024-04-25T16:52:31.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.719+0000 7fd358220640 1 -- 172.21.15.94:0/2059070582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd35006fbe0 msgr2=0x7fd3501472e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.719+0000 7fd358220640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd35006fbe0 0x7fd3501472e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00c7f0 tx=0x7fd34c00ccb0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.719+0000 7fd358220640 1 -- 172.21.15.94:0/2059070582 shutdown_connections 2024-04-25T16:52:31.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.719+0000 7fd358220640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd33406f400 0x7fd334071910 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.719+0000 7fd358220640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd350156790 0x7fd3500b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.719+0000 7fd358220640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd350072740 0x7fd3500b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.719+0000 7fd358220640 1 --2- 172.21.15.94:0/2059070582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd35006fbe0 0x7fd3501472e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.719+0000 7fd358220640 1 -- 172.21.15.94:0/2059070582 >> 172.21.15.94:0/2059070582 conn(0x7fd3500b47f0 msgr2=0x7fd3500b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:31.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.719+0000 7fd358220640 1 -- 172.21.15.94:0/2059070582 shutdown_connections 2024-04-25T16:52:31.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.719+0000 7fd358220640 1 -- 172.21.15.94:0/2059070582 wait complete. 2024-04-25T16:52:31.736 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-04-25T16:52:31.694027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258276,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716444,"statfs":{"total":773094113280,"available":772829638656,"internally_reserved":0,"allocated":4759552,"data_stored":2705780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001852"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830121+0000","last_change":"2024-04-25T16:52:26.825748+0000","last_active":"2024-04-25T16:52:27.830121+0000","last_peered":"2024-04-25T16:52:27.830121+0000","last_clean":"2024-04-25T16:52:27.830121+0000","last_became_active":"2024-04-25T16:52:26.825333+0000","last_became_peered":"2024-04-25T16:52:26.825333+0000","last_unstale":"2024-04-25T16:52:27.830121+0000","last_undegraded":"2024-04-25T16:52:27.830121+0000","last_fullsized":"2024-04-25T16:52:27.830121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:19.084563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829420+0000","last_change":"2024-04-25T16:52:26.795964+0000","last_active":"2024-04-25T16:52:27.829420+0000","last_peered":"2024-04-25T16:52:27.829420+0000","last_clean":"2024-04-25T16:52:27.829420+0000","last_became_active":"2024-04-25T16:52:26.794884+0000","last_became_peered":"2024-04-25T16:52:26.794884+0000","last_unstale":"2024-04-25T16:52:27.829420+0000","last_undegraded":"2024-04-25T16:52:27.829420+0000","last_fullsized":"2024-04-25T16:52:27.829420+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832064+0000","last_change":"2024-04-25T16:52:26.802075+0000","last_active":"2024-04-25T16:52:27.832064+0000","last_peered":"2024-04-25T16:52:27.832064+0000","last_clean":"2024-04-25T16:52:27.832064+0000","last_became_active":"2024-04-25T16:52:26.801203+0000","last_became_peered":"2024-04-25T16:52:26.801203+0000","last_unstale":"2024-04-25T16:52:27.832064+0000","last_undegraded":"2024-04-25T16:52:27.832064+0000","last_fullsized":"2024-04-25T16:52:27.832064+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782213+0000","last_change":"2024-04-25T16:52:26.793541+0000","last_active":"2024-04-25T16:52:27.782213+0000","last_peered":"2024-04-25T16:52:27.782213+0000","last_clean":"2024-04-25T16:52:27.782213+0000","last_became_active":"2024-04-25T16:52:26.789101+0000","last_became_peered":"2024-04-25T16:52:26.789101+0000","last_unstale":"2024-04-25T16:52:27.782213+0000","last_undegraded":"2024-04-25T16:52:27.782213+0000","last_fullsized":"2024-04-25T16:52:27.782213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:38.013058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781321+0000","last_change":"2024-04-25T16:52:26.784467+0000","last_active":"2024-04-25T16:52:27.781321+0000","last_peered":"2024-04-25T16:52:27.781321+0000","last_clean":"2024-04-25T16:52:27.781321+0000","last_became_active":"2024-04-25T16:52:26.783673+0000","last_became_peered":"2024-04-25T16:52:26.783673+0000","last_unstale":"2024-04-25T16:52:27.781321+0000","last_undegraded":"2024-04-25T16:52:27.781321+0000","last_fullsized":"2024-04-25T16:52:27.781321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:36.583897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831274+0000","last_change":"2024-04-25T16:52:26.823413+0000","last_active":"2024-04-25T16:52:27.831274+0000","last_peered":"2024-04-25T16:52:27.831274+0000","last_clean":"2024-04-25T16:52:27.831274+0000","last_became_active":"2024-04-25T16:52:26.823059+0000","last_became_peered":"2024-04-25T16:52:26.823059+0000","last_unstale":"2024-04-25T16:52:27.831274+0000","last_undegraded":"2024-04-25T16:52:27.831274+0000","last_fullsized":"2024-04-25T16:52:27.831274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.821862+0000","last_change":"2024-04-25T16:52:26.808661+0000","last_active":"2024-04-25T16:52:27.821862+0000","last_peered":"2024-04-25T16:52:27.821862+0000","last_clean":"2024-04-25T16:52:27.821862+0000","last_became_active":"2024-04-25T16:52:26.805296+0000","last_became_peered":"2024-04-25T16:52:26.805296+0000","last_unstale":"2024-04-25T16:52:27.821862+0000","last_undegraded":"2024-04-25T16:52:27.821862+0000","last_fullsized":"2024-04-25T16:52:27.821862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829292+0000","last_change":"2024-04-25T16:52:26.820861+0000","last_active":"2024-04-25T16:52:27.829292+0000","last_peered":"2024-04-25T16:52:27.829292+0000","last_clean":"2024-04-25T16:52:27.829292+0000","last_became_active":"2024-04-25T16:52:26.820528+0000","last_became_peered":"2024-04-25T16:52:26.820528+0000","last_unstale":"2024-04-25T16:52:27.829292+0000","last_undegraded":"2024-04-25T16:52:27.829292+0000","last_fullsized":"2024-04-25T16:52:27.829292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:33.962706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832345+0000","last_change":"2024-04-25T16:52:26.803363+0000","last_active":"2024-04-25T16:52:27.832345+0000","last_peered":"2024-04-25T16:52:27.832345+0000","last_clean":"2024-04-25T16:52:27.832345+0000","last_became_active":"2024-04-25T16:52:26.802539+0000","last_became_peered":"2024-04-25T16:52:26.802539+0000","last_unstale":"2024-04-25T16:52:27.832345+0000","last_undegraded":"2024-04-25T16:52:27.832345+0000","last_fullsized":"2024-04-25T16:52:27.832345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822740+0000","last_change":"2024-04-25T16:52:26.842728+0000","last_active":"2024-04-25T16:52:27.822740+0000","last_peered":"2024-04-25T16:52:27.822740+0000","last_clean":"2024-04-25T16:52:27.822740+0000","last_became_active":"2024-04-25T16:52:26.842041+0000","last_became_peered":"2024-04-25T16:52:26.842041+0000","last_unstale":"2024-04-25T16:52:27.822740+0000","last_undegraded":"2024-04-25T16:52:27.822740+0000","last_fullsized":"2024-04-25T16:52:27.822740+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:33.962706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823850+0000","last_change":"2024-04-25T16:52:26.819329+0000","last_active":"2024-04-25T16:52:27.823850+0000","last_peered":"2024-04-25T16:52:27.823850+0000","last_clean":"2024-04-25T16:52:27.823850+0000","last_became_active":"2024-04-25T16:52:26.818528+0000","last_became_peered":"2024-04-25T16:52:26.818528+0000","last_unstale":"2024-04-25T16:52:27.823850+0000","last_undegraded":"2024-04-25T16:52:27.823850+0000","last_fullsized":"2024-04-25T16:52:27.823850+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832046+0000","last_change":"2024-04-25T16:52:26.795942+0000","last_active":"2024-04-25T16:52:27.832046+0000","last_peered":"2024-04-25T16:52:27.832046+0000","last_clean":"2024-04-25T16:52:27.832046+0000","last_became_active":"2024-04-25T16:52:26.795124+0000","last_became_peered":"2024-04-25T16:52:26.795124+0000","last_unstale":"2024-04-25T16:52:27.832046+0000","last_undegraded":"2024-04-25T16:52:27.832046+0000","last_fullsized":"2024-04-25T16:52:27.832046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822028+0000","last_change":"2024-04-25T16:52:26.814749+0000","last_active":"2024-04-25T16:52:27.822028+0000","last_peered":"2024-04-25T16:52:27.822028+0000","last_clean":"2024-04-25T16:52:27.822028+0000","last_became_active":"2024-04-25T16:52:26.814020+0000","last_became_peered":"2024-04-25T16:52:26.814020+0000","last_unstale":"2024-04-25T16:52:27.822028+0000","last_undegraded":"2024-04-25T16:52:27.822028+0000","last_fullsized":"2024-04-25T16:52:27.822028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832107+0000","last_change":"2024-04-25T16:52:26.813118+0000","last_active":"2024-04-25T16:52:27.832107+0000","last_peered":"2024-04-25T16:52:27.832107+0000","last_clean":"2024-04-25T16:52:27.832107+0000","last_became_active":"2024-04-25T16:52:26.812400+0000","last_became_peered":"2024-04-25T16:52:26.812400+0000","last_unstale":"2024-04-25T16:52:27.832107+0000","last_undegraded":"2024-04-25T16:52:27.832107+0000","last_fullsized":"2024-04-25T16:52:27.832107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:16.169654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822023+0000","last_change":"2024-04-25T16:52:26.795864+0000","last_active":"2024-04-25T16:52:27.822023+0000","last_peered":"2024-04-25T16:52:27.822023+0000","last_clean":"2024-04-25T16:52:27.822023+0000","last_became_active":"2024-04-25T16:52:26.794738+0000","last_became_peered":"2024-04-25T16:52:26.794738+0000","last_unstale":"2024-04-25T16:52:27.822023+0000","last_undegraded":"2024-04-25T16:52:27.822023+0000","last_fullsized":"2024-04-25T16:52:27.822023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781621+0000","last_change":"2024-04-25T16:52:26.836640+0000","last_active":"2024-04-25T16:52:27.781621+0000","last_peered":"2024-04-25T16:52:27.781621+0000","last_clean":"2024-04-25T16:52:27.781621+0000","last_became_active":"2024-04-25T16:52:26.833071+0000","last_became_peered":"2024-04-25T16:52:26.833071+0000","last_unstale":"2024-04-25T16:52:27.781621+0000","last_undegraded":"2024-04-25T16:52:27.781621+0000","last_fullsized":"2024-04-25T16:52:27.781621+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.783331+0000","last_change":"2024-04-25T16:52:26.841207+0000","last_active":"2024-04-25T16:52:27.783331+0000","last_peered":"2024-04-25T16:52:27.783331+0000","last_clean":"2024-04-25T16:52:27.783331+0000","last_became_active":"2024-04-25T16:52:26.838750+0000","last_became_peered":"2024-04-25T16:52:26.838750+0000","last_unstale":"2024-04-25T16:52:27.783331+0000","last_undegraded":"2024-04-25T16:52:27.783331+0000","last_fullsized":"2024-04-25T16:52:27.783331+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780940+0000","last_change":"2024-04-25T16:52:26.795939+0000","last_active":"2024-04-25T16:52:27.780940+0000","last_peered":"2024-04-25T16:52:27.780940+0000","last_clean":"2024-04-25T16:52:27.780940+0000","last_became_active":"2024-04-25T16:52:26.795084+0000","last_became_peered":"2024-04-25T16:52:26.795084+0000","last_unstale":"2024-04-25T16:52:27.780940+0000","last_undegraded":"2024-04-25T16:52:27.780940+0000","last_fullsized":"2024-04-25T16:52:27.780940+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781139+0000","last_change":"2024-04-25T16:52:26.818025+0000","last_active":"2024-04-25T16:52:27.781139+0000","last_peered":"2024-04-25T16:52:27.781139+0000","last_clean":"2024-04-25T16:52:27.781139+0000","last_became_active":"2024-04-25T16:52:26.817652+0000","last_became_peered":"2024-04-25T16:52:26.817652+0000","last_unstale":"2024-04-25T16:52:27.781139+0000","last_undegraded":"2024-04-25T16:52:27.781139+0000","last_fullsized":"2024-04-25T16:52:27.781139+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:46.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781002+0000","last_change":"2024-04-25T16:52:26.796384+0000","last_active":"2024-04-25T16:52:27.781002+0000","last_peered":"2024-04-25T16:52:27.781002+0000","last_clean":"2024-04-25T16:52:27.781002+0000","last_became_active":"2024-04-25T16:52:26.795507+0000","last_became_peered":"2024-04-25T16:52:26.795507+0000","last_unstale":"2024-04-25T16:52:27.781002+0000","last_undegraded":"2024-04-25T16:52:27.781002+0000","last_fullsized":"2024-04-25T16:52:27.781002+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822359+0000","last_change":"2024-04-25T16:52:26.809988+0000","last_active":"2024-04-25T16:52:27.822359+0000","last_peered":"2024-04-25T16:52:27.822359+0000","last_clean":"2024-04-25T16:52:27.822359+0000","last_became_active":"2024-04-25T16:52:26.809107+0000","last_became_peered":"2024-04-25T16:52:26.809107+0000","last_unstale":"2024-04-25T16:52:27.822359+0000","last_undegraded":"2024-04-25T16:52:27.822359+0000","last_fullsized":"2024-04-25T16:52:27.822359+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831788+0000","last_change":"2024-04-25T16:52:26.784466+0000","last_active":"2024-04-25T16:52:27.831788+0000","last_peered":"2024-04-25T16:52:27.831788+0000","last_clean":"2024-04-25T16:52:27.831788+0000","last_became_active":"2024-04-25T16:52:26.783550+0000","last_became_peered":"2024-04-25T16:52:26.783550+0000","last_unstale":"2024-04-25T16:52:27.831788+0000","last_undegraded":"2024-04-25T16:52:27.831788+0000","last_fullsized":"2024-04-25T16:52:27.831788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781970+0000","last_change":"2024-04-25T16:52:26.832452+0000","last_active":"2024-04-25T16:52:27.781970+0000","last_peered":"2024-04-25T16:52:27.781970+0000","last_clean":"2024-04-25T16:52:27.781970+0000","last_became_active":"2024-04-25T16:52:26.831753+0000","last_became_peered":"2024-04-25T16:52:26.831753+0000","last_unstale":"2024-04-25T16:52:27.781970+0000","last_undegraded":"2024-04-25T16:52:27.781970+0000","last_fullsized":"2024-04-25T16:52:27.781970+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780752+0000","last_change":"2024-04-25T16:52:26.805633+0000","last_active":"2024-04-25T16:52:27.780752+0000","last_peered":"2024-04-25T16:52:27.780752+0000","last_clean":"2024-04-25T16:52:27.780752+0000","last_became_active":"2024-04-25T16:52:26.804789+0000","last_became_peered":"2024-04-25T16:52:26.804789+0000","last_unstale":"2024-04-25T16:52:27.780752+0000","last_undegraded":"2024-04-25T16:52:27.780752+0000","last_fullsized":"2024-04-25T16:52:27.780752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:08.931465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780486+0000","last_change":"2024-04-25T16:52:26.798693+0000","last_active":"2024-04-25T16:52:27.780486+0000","last_peered":"2024-04-25T16:52:27.780486+0000","last_clean":"2024-04-25T16:52:27.780486+0000","last_became_active":"2024-04-25T16:52:26.797852+0000","last_became_peered":"2024-04-25T16:52:26.797852+0000","last_unstale":"2024-04-25T16:52:27.780486+0000","last_undegraded":"2024-04-25T16:52:27.780486+0000","last_fullsized":"2024-04-25T16:52:27.780486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831376+0000","last_change":"2024-04-25T16:52:26.819630+0000","last_active":"2024-04-25T16:52:27.831376+0000","last_peered":"2024-04-25T16:52:27.831376+0000","last_clean":"2024-04-25T16:52:27.831376+0000","last_became_active":"2024-04-25T16:52:26.819274+0000","last_became_peered":"2024-04-25T16:52:26.819274+0000","last_unstale":"2024-04-25T16:52:27.831376+0000","last_undegraded":"2024-04-25T16:52:27.831376+0000","last_fullsized":"2024-04-25T16:52:27.831376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781126+0000","last_change":"2024-04-25T16:52:26.791433+0000","last_active":"2024-04-25T16:52:27.781126+0000","last_peered":"2024-04-25T16:52:27.781126+0000","last_clean":"2024-04-25T16:52:27.781126+0000","last_became_active":"2024-04-25T16:52:26.790483+0000","last_became_peered":"2024-04-25T16:52:26.790483+0000","last_unstale":"2024-04-25T16:52:27.781126+0000","last_undegraded":"2024-04-25T16:52:27.781126+0000","last_fullsized":"2024-04-25T16:52:27.781126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822953+0000","last_change":"2024-04-25T16:52:26.804860+0000","last_active":"2024-04-25T16:52:27.822953+0000","last_peered":"2024-04-25T16:52:27.822953+0000","last_clean":"2024-04-25T16:52:27.822953+0000","last_became_active":"2024-04-25T16:52:26.804065+0000","last_became_peered":"2024-04-25T16:52:26.804065+0000","last_unstale":"2024-04-25T16:52:27.822953+0000","last_undegraded":"2024-04-25T16:52:27.822953+0000","last_fullsized":"2024-04-25T16:52:27.822953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780617+0000","last_change":"2024-04-25T16:52:26.806705+0000","last_active":"2024-04-25T16:52:27.780617+0000","last_peered":"2024-04-25T16:52:27.780617+0000","last_clean":"2024-04-25T16:52:27.780617+0000","last_became_active":"2024-04-25T16:52:26.805889+0000","last_became_peered":"2024-04-25T16:52:26.805889+0000","last_unstale":"2024-04-25T16:52:27.780617+0000","last_undegraded":"2024-04-25T16:52:27.780617+0000","last_fullsized":"2024-04-25T16:52:27.780617+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780149+0000","last_change":"2024-04-25T16:52:26.798903+0000","last_active":"2024-04-25T16:52:27.780149+0000","last_peered":"2024-04-25T16:52:27.780149+0000","last_clean":"2024-04-25T16:52:27.780149+0000","last_became_active":"2024-04-25T16:52:26.798011+0000","last_became_peered":"2024-04-25T16:52:26.798011+0000","last_unstale":"2024-04-25T16:52:27.780149+0000","last_undegraded":"2024-04-25T16:52:27.780149+0000","last_fullsized":"2024-04-25T16:52:27.780149+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782037+0000","last_change":"2024-04-25T16:52:26.841037+0000","last_active":"2024-04-25T16:52:27.782037+0000","last_peered":"2024-04-25T16:52:27.782037+0000","last_clean":"2024-04-25T16:52:27.782037+0000","last_became_active":"2024-04-25T16:52:26.838194+0000","last_became_peered":"2024-04-25T16:52:26.838194+0000","last_unstale":"2024-04-25T16:52:27.782037+0000","last_undegraded":"2024-04-25T16:52:27.782037+0000","last_fullsized":"2024-04-25T16:52:27.782037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830883+0000","last_change":"2024-04-25T16:52:26.813243+0000","last_active":"2024-04-25T16:52:27.830883+0000","last_peered":"2024-04-25T16:52:27.830883+0000","last_clean":"2024-04-25T16:52:27.830883+0000","last_became_active":"2024-04-25T16:52:26.811807+0000","last_became_peered":"2024-04-25T16:52:26.811807+0000","last_unstale":"2024-04-25T16:52:27.830883+0000","last_undegraded":"2024-04-25T16:52:27.830883+0000","last_fullsized":"2024-04-25T16:52:27.830883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:42.553053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782541+0000","last_change":"2024-04-25T16:52:26.841017+0000","last_active":"2024-04-25T16:52:27.782541+0000","last_peered":"2024-04-25T16:52:27.782541+0000","last_clean":"2024-04-25T16:52:27.782541+0000","last_became_active":"2024-04-25T16:52:26.837543+0000","last_became_peered":"2024-04-25T16:52:26.837543+0000","last_unstale":"2024-04-25T16:52:27.782541+0000","last_undegraded":"2024-04-25T16:52:27.782541+0000","last_fullsized":"2024-04-25T16:52:27.782541+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780599+0000","last_change":"2024-04-25T16:52:25.818069+0000","last_active":"2024-04-25T16:52:27.780599+0000","last_peered":"2024-04-25T16:52:27.780599+0000","last_clean":"2024-04-25T16:52:27.780599+0000","last_became_active":"2024-04-25T16:52:25.817362+0000","last_became_peered":"2024-04-25T16:52:25.817362+0000","last_unstale":"2024-04-25T16:52:27.780599+0000","last_undegraded":"2024-04-25T16:52:27.780599+0000","last_fullsized":"2024-04-25T16:52:27.780599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:08.488136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833310+0000","last_change":"2024-04-25T16:52:26.829008+0000","last_active":"2024-04-25T16:52:27.833310+0000","last_peered":"2024-04-25T16:52:27.833310+0000","last_clean":"2024-04-25T16:52:27.833310+0000","last_became_active":"2024-04-25T16:52:26.828706+0000","last_became_peered":"2024-04-25T16:52:26.828706+0000","last_unstale":"2024-04-25T16:52:27.833310+0000","last_undegraded":"2024-04-25T16:52:27.833310+0000","last_fullsized":"2024-04-25T16:52:27.833310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832100+0000","last_change":"2024-04-25T16:52:25.795967+0000","last_active":"2024-04-25T16:52:27.832100+0000","last_peered":"2024-04-25T16:52:27.832100+0000","last_clean":"2024-04-25T16:52:27.832100+0000","last_became_active":"2024-04-25T16:52:25.795695+0000","last_became_peered":"2024-04-25T16:52:25.795695+0000","last_unstale":"2024-04-25T16:52:27.832100+0000","last_undegraded":"2024-04-25T16:52:27.832100+0000","last_fullsized":"2024-04-25T16:52:27.832100+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831737+0000","last_change":"2024-04-25T16:52:26.789430+0000","last_active":"2024-04-25T16:52:27.831737+0000","last_peered":"2024-04-25T16:52:27.831737+0000","last_clean":"2024-04-25T16:52:27.831737+0000","last_became_active":"2024-04-25T16:52:26.782243+0000","last_became_peered":"2024-04-25T16:52:26.782243+0000","last_unstale":"2024-04-25T16:52:27.831737+0000","last_undegraded":"2024-04-25T16:52:27.831737+0000","last_fullsized":"2024-04-25T16:52:27.831737+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833428+0000","last_change":"2024-04-25T16:52:25.819162+0000","last_active":"2024-04-25T16:52:27.833428+0000","last_peered":"2024-04-25T16:52:27.833428+0000","last_clean":"2024-04-25T16:52:27.833428+0000","last_became_active":"2024-04-25T16:52:25.818859+0000","last_became_peered":"2024-04-25T16:52:25.818859+0000","last_unstale":"2024-04-25T16:52:27.833428+0000","last_undegraded":"2024-04-25T16:52:27.833428+0000","last_fullsized":"2024-04-25T16:52:27.833428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829867+0000","last_change":"2024-04-25T16:52:26.805376+0000","last_active":"2024-04-25T16:52:27.829867+0000","last_peered":"2024-04-25T16:52:27.829867+0000","last_clean":"2024-04-25T16:52:27.829867+0000","last_became_active":"2024-04-25T16:52:26.804542+0000","last_became_peered":"2024-04-25T16:52:26.804542+0000","last_unstale":"2024-04-25T16:52:27.829867+0000","last_undegraded":"2024-04-25T16:52:27.829867+0000","last_fullsized":"2024-04-25T16:52:27.829867+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.620224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831272+0000","last_change":"2024-04-25T16:52:25.815751+0000","last_active":"2024-04-25T16:52:27.831272+0000","last_peered":"2024-04-25T16:52:27.831272+0000","last_clean":"2024-04-25T16:52:27.831272+0000","last_became_active":"2024-04-25T16:52:25.814927+0000","last_became_peered":"2024-04-25T16:52:25.814927+0000","last_unstale":"2024-04-25T16:52:27.831272+0000","last_undegraded":"2024-04-25T16:52:27.831272+0000","last_fullsized":"2024-04-25T16:52:27.831272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831681+0000","last_change":"2024-04-25T16:52:26.814640+0000","last_active":"2024-04-25T16:52:27.831681+0000","last_peered":"2024-04-25T16:52:27.831681+0000","last_clean":"2024-04-25T16:52:27.831681+0000","last_became_active":"2024-04-25T16:52:26.814016+0000","last_became_peered":"2024-04-25T16:52:26.814016+0000","last_unstale":"2024-04-25T16:52:27.831681+0000","last_undegraded":"2024-04-25T16:52:27.831681+0000","last_fullsized":"2024-04-25T16:52:27.831681+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822321+0000","last_change":"2024-04-25T16:52:25.805323+0000","last_active":"2024-04-25T16:52:27.822321+0000","last_peered":"2024-04-25T16:52:27.822321+0000","last_clean":"2024-04-25T16:52:27.822321+0000","last_became_active":"2024-04-25T16:52:25.796444+0000","last_became_peered":"2024-04-25T16:52:25.796444+0000","last_unstale":"2024-04-25T16:52:27.822321+0000","last_undegraded":"2024-04-25T16:52:27.822321+0000","last_fullsized":"2024-04-25T16:52:27.822321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:28.996351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822686+0000","last_change":"2024-04-25T16:52:26.821648+0000","last_active":"2024-04-25T16:52:27.822686+0000","last_peered":"2024-04-25T16:52:27.822686+0000","last_clean":"2024-04-25T16:52:27.822686+0000","last_became_active":"2024-04-25T16:52:26.820835+0000","last_became_peered":"2024-04-25T16:52:26.820835+0000","last_unstale":"2024-04-25T16:52:27.822686+0000","last_undegraded":"2024-04-25T16:52:27.822686+0000","last_fullsized":"2024-04-25T16:52:27.822686+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781761+0000","last_change":"2024-04-25T16:52:25.816250+0000","last_active":"2024-04-25T16:52:27.781761+0000","last_peered":"2024-04-25T16:52:27.781761+0000","last_clean":"2024-04-25T16:52:27.781761+0000","last_became_active":"2024-04-25T16:52:25.815656+0000","last_became_peered":"2024-04-25T16:52:25.815656+0000","last_unstale":"2024-04-25T16:52:27.781761+0000","last_undegraded":"2024-04-25T16:52:27.781761+0000","last_fullsized":"2024-04-25T16:52:27.781761+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831178+0000","last_change":"2024-04-25T16:52:26.813425+0000","last_active":"2024-04-25T16:52:27.831178+0000","last_peered":"2024-04-25T16:52:27.831178+0000","last_clean":"2024-04-25T16:52:27.831178+0000","last_became_active":"2024-04-25T16:52:26.812713+0000","last_became_peered":"2024-04-25T16:52:26.812713+0000","last_unstale":"2024-04-25T16:52:27.831178+0000","last_undegraded":"2024-04-25T16:52:27.831178+0000","last_fullsized":"2024-04-25T16:52:27.831178+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831072+0000","last_change":"2024-04-25T16:52:25.803265+0000","last_active":"2024-04-25T16:52:27.831072+0000","last_peered":"2024-04-25T16:52:27.831072+0000","last_clean":"2024-04-25T16:52:27.831072+0000","last_became_active":"2024-04-25T16:52:25.802617+0000","last_became_peered":"2024-04-25T16:52:25.802617+0000","last_unstale":"2024-04-25T16:52:27.831072+0000","last_undegraded":"2024-04-25T16:52:27.831072+0000","last_fullsized":"2024-04-25T16:52:27.831072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829740+0000","last_change":"2024-04-25T16:52:26.819267+0000","last_active":"2024-04-25T16:52:27.829740+0000","last_peered":"2024-04-25T16:52:27.829740+0000","last_clean":"2024-04-25T16:52:27.829740+0000","last_became_active":"2024-04-25T16:52:26.818740+0000","last_became_peered":"2024-04-25T16:52:26.818740+0000","last_unstale":"2024-04-25T16:52:27.829740+0000","last_undegraded":"2024-04-25T16:52:27.829740+0000","last_fullsized":"2024-04-25T16:52:27.829740+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830189+0000","last_change":"2024-04-25T16:52:25.816663+0000","last_active":"2024-04-25T16:52:27.830189+0000","last_peered":"2024-04-25T16:52:27.830189+0000","last_clean":"2024-04-25T16:52:27.830189+0000","last_became_active":"2024-04-25T16:52:25.815420+0000","last_became_peered":"2024-04-25T16:52:25.815420+0000","last_unstale":"2024-04-25T16:52:27.830189+0000","last_undegraded":"2024-04-25T16:52:27.830189+0000","last_fullsized":"2024-04-25T16:52:27.830189+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:30.288538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782313+0000","last_change":"2024-04-25T16:52:26.789927+0000","last_active":"2024-04-25T16:52:27.782313+0000","last_peered":"2024-04-25T16:52:27.782313+0000","last_clean":"2024-04-25T16:52:27.782313+0000","last_became_active":"2024-04-25T16:52:26.788014+0000","last_became_peered":"2024-04-25T16:52:26.788014+0000","last_unstale":"2024-04-25T16:52:27.782313+0000","last_undegraded":"2024-04-25T16:52:27.782313+0000","last_fullsized":"2024-04-25T16:52:27.782313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:47.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830050+0000","last_change":"2024-04-25T16:52:25.798974+0000","last_active":"2024-04-25T16:52:27.830050+0000","last_peered":"2024-04-25T16:52:27.830050+0000","last_clean":"2024-04-25T16:52:27.830050+0000","last_became_active":"2024-04-25T16:52:25.798314+0000","last_became_peered":"2024-04-25T16:52:25.798314+0000","last_unstale":"2024-04-25T16:52:27.830050+0000","last_undegraded":"2024-04-25T16:52:27.830050+0000","last_fullsized":"2024-04-25T16:52:27.830050+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.348530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782542+0000","last_change":"2024-04-25T16:52:26.842485+0000","last_active":"2024-04-25T16:52:27.782542+0000","last_peered":"2024-04-25T16:52:27.782542+0000","last_clean":"2024-04-25T16:52:27.782542+0000","last_became_active":"2024-04-25T16:52:26.838813+0000","last_became_peered":"2024-04-25T16:52:26.838813+0000","last_unstale":"2024-04-25T16:52:27.782542+0000","last_undegraded":"2024-04-25T16:52:27.782542+0000","last_fullsized":"2024-04-25T16:52:27.782542+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823330+0000","last_change":"2024-04-25T16:52:25.803289+0000","last_active":"2024-04-25T16:52:27.823330+0000","last_peered":"2024-04-25T16:52:27.823330+0000","last_clean":"2024-04-25T16:52:27.823330+0000","last_became_active":"2024-04-25T16:52:25.801415+0000","last_became_peered":"2024-04-25T16:52:25.801415+0000","last_unstale":"2024-04-25T16:52:27.823330+0000","last_undegraded":"2024-04-25T16:52:27.823330+0000","last_fullsized":"2024-04-25T16:52:27.823330+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:45.310089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832151+0000","last_change":"2024-04-25T16:52:26.840379+0000","last_active":"2024-04-25T16:52:27.832151+0000","last_peered":"2024-04-25T16:52:27.832151+0000","last_clean":"2024-04-25T16:52:27.832151+0000","last_became_active":"2024-04-25T16:52:26.840062+0000","last_became_peered":"2024-04-25T16:52:26.840062+0000","last_unstale":"2024-04-25T16:52:27.832151+0000","last_undegraded":"2024-04-25T16:52:27.832151+0000","last_fullsized":"2024-04-25T16:52:27.832151+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823961+0000","last_change":"2024-04-25T16:52:25.809391+0000","last_active":"2024-04-25T16:52:27.823961+0000","last_peered":"2024-04-25T16:52:27.823961+0000","last_clean":"2024-04-25T16:52:27.823961+0000","last_became_active":"2024-04-25T16:52:25.808455+0000","last_became_peered":"2024-04-25T16:52:25.808455+0000","last_unstale":"2024-04-25T16:52:27.823961+0000","last_undegraded":"2024-04-25T16:52:27.823961+0000","last_fullsized":"2024-04-25T16:52:27.823961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831993+0000","last_change":"2024-04-25T16:52:26.782954+0000","last_active":"2024-04-25T16:52:27.831993+0000","last_peered":"2024-04-25T16:52:27.831993+0000","last_clean":"2024-04-25T16:52:27.831993+0000","last_became_active":"2024-04-25T16:52:26.782108+0000","last_became_peered":"2024-04-25T16:52:26.782108+0000","last_unstale":"2024-04-25T16:52:27.831993+0000","last_undegraded":"2024-04-25T16:52:27.831993+0000","last_fullsized":"2024-04-25T16:52:27.831993+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830290+0000","last_change":"2024-04-25T16:52:25.795020+0000","last_active":"2024-04-25T16:52:27.830290+0000","last_peered":"2024-04-25T16:52:27.830290+0000","last_clean":"2024-04-25T16:52:27.830290+0000","last_became_active":"2024-04-25T16:52:25.794633+0000","last_became_peered":"2024-04-25T16:52:25.794633+0000","last_unstale":"2024-04-25T16:52:27.830290+0000","last_undegraded":"2024-04-25T16:52:27.830290+0000","last_fullsized":"2024-04-25T16:52:27.830290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:38.694859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831678+0000","last_change":"2024-04-25T16:52:26.821081+0000","last_active":"2024-04-25T16:52:27.831678+0000","last_peered":"2024-04-25T16:52:27.831678+0000","last_clean":"2024-04-25T16:52:27.831678+0000","last_became_active":"2024-04-25T16:52:26.819337+0000","last_became_peered":"2024-04-25T16:52:26.819337+0000","last_unstale":"2024-04-25T16:52:27.831678+0000","last_undegraded":"2024-04-25T16:52:27.831678+0000","last_fullsized":"2024-04-25T16:52:27.831678+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830009+0000","last_change":"2024-04-25T16:52:25.793039+0000","last_active":"2024-04-25T16:52:27.830009+0000","last_peered":"2024-04-25T16:52:27.830009+0000","last_clean":"2024-04-25T16:52:27.830009+0000","last_became_active":"2024-04-25T16:52:25.792552+0000","last_became_peered":"2024-04-25T16:52:25.792552+0000","last_unstale":"2024-04-25T16:52:27.830009+0000","last_undegraded":"2024-04-25T16:52:27.830009+0000","last_fullsized":"2024-04-25T16:52:27.830009+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:30.954700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829446+0000","last_change":"2024-04-25T16:52:26.823643+0000","last_active":"2024-04-25T16:52:27.829446+0000","last_peered":"2024-04-25T16:52:27.829446+0000","last_clean":"2024-04-25T16:52:27.829446+0000","last_became_active":"2024-04-25T16:52:26.823348+0000","last_became_peered":"2024-04-25T16:52:26.823348+0000","last_unstale":"2024-04-25T16:52:27.829446+0000","last_undegraded":"2024-04-25T16:52:27.829446+0000","last_fullsized":"2024-04-25T16:52:27.829446+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830926+0000","last_change":"2024-04-25T16:52:25.813706+0000","last_active":"2024-04-25T16:52:27.830926+0000","last_peered":"2024-04-25T16:52:27.830926+0000","last_clean":"2024-04-25T16:52:27.830926+0000","last_became_active":"2024-04-25T16:52:25.813387+0000","last_became_peered":"2024-04-25T16:52:25.813387+0000","last_unstale":"2024-04-25T16:52:27.830926+0000","last_undegraded":"2024-04-25T16:52:27.830926+0000","last_fullsized":"2024-04-25T16:52:27.830926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781928+0000","last_change":"2024-04-25T16:52:26.817332+0000","last_active":"2024-04-25T16:52:27.781928+0000","last_peered":"2024-04-25T16:52:27.781928+0000","last_clean":"2024-04-25T16:52:27.781928+0000","last_became_active":"2024-04-25T16:52:26.816020+0000","last_became_peered":"2024-04-25T16:52:26.816020+0000","last_unstale":"2024-04-25T16:52:27.781928+0000","last_undegraded":"2024-04-25T16:52:27.781928+0000","last_fullsized":"2024-04-25T16:52:27.781928+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.779458+0000","last_change":"2024-04-25T16:52:25.815545+0000","last_active":"2024-04-25T16:52:27.779458+0000","last_peered":"2024-04-25T16:52:27.779458+0000","last_clean":"2024-04-25T16:52:27.779458+0000","last_became_active":"2024-04-25T16:52:25.814711+0000","last_became_peered":"2024-04-25T16:52:25.814711+0000","last_unstale":"2024-04-25T16:52:27.779458+0000","last_undegraded":"2024-04-25T16:52:27.779458+0000","last_fullsized":"2024-04-25T16:52:27.779458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829829+0000","last_change":"2024-04-25T16:52:26.826567+0000","last_active":"2024-04-25T16:52:27.829829+0000","last_peered":"2024-04-25T16:52:27.829829+0000","last_clean":"2024-04-25T16:52:27.829829+0000","last_became_active":"2024-04-25T16:52:26.826224+0000","last_became_peered":"2024-04-25T16:52:26.826224+0000","last_unstale":"2024-04-25T16:52:27.829829+0000","last_undegraded":"2024-04-25T16:52:27.829829+0000","last_fullsized":"2024-04-25T16:52:27.829829+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:06.823841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782735+0000","last_change":"2024-04-25T16:52:25.815153+0000","last_active":"2024-04-25T16:52:27.782735+0000","last_peered":"2024-04-25T16:52:27.782735+0000","last_clean":"2024-04-25T16:52:27.782735+0000","last_became_active":"2024-04-25T16:52:25.814360+0000","last_became_peered":"2024-04-25T16:52:25.814360+0000","last_unstale":"2024-04-25T16:52:27.782735+0000","last_undegraded":"2024-04-25T16:52:27.782735+0000","last_fullsized":"2024-04-25T16:52:27.782735+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830252+0000","last_change":"2024-04-25T16:52:26.804704+0000","last_active":"2024-04-25T16:52:27.830252+0000","last_peered":"2024-04-25T16:52:27.830252+0000","last_clean":"2024-04-25T16:52:27.830252+0000","last_became_active":"2024-04-25T16:52:26.803888+0000","last_became_peered":"2024-04-25T16:52:26.803888+0000","last_unstale":"2024-04-25T16:52:27.830252+0000","last_undegraded":"2024-04-25T16:52:27.830252+0000","last_fullsized":"2024-04-25T16:52:27.830252+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.640204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823002+0000","last_change":"2024-04-25T16:52:25.818880+0000","last_active":"2024-04-25T16:52:27.823002+0000","last_peered":"2024-04-25T16:52:27.823002+0000","last_clean":"2024-04-25T16:52:27.823002+0000","last_became_active":"2024-04-25T16:52:25.817841+0000","last_became_peered":"2024-04-25T16:52:25.817841+0000","last_unstale":"2024-04-25T16:52:27.823002+0000","last_undegraded":"2024-04-25T16:52:27.823002+0000","last_fullsized":"2024-04-25T16:52:27.823002+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:26.817110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823891+0000","last_change":"2024-04-25T16:52:26.809554+0000","last_active":"2024-04-25T16:52:27.823891+0000","last_peered":"2024-04-25T16:52:27.823891+0000","last_clean":"2024-04-25T16:52:27.823891+0000","last_became_active":"2024-04-25T16:52:26.808709+0000","last_became_peered":"2024-04-25T16:52:26.808709+0000","last_unstale":"2024-04-25T16:52:27.823891+0000","last_undegraded":"2024-04-25T16:52:27.823891+0000","last_fullsized":"2024-04-25T16:52:27.823891+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831719+0000","last_change":"2024-04-25T16:52:25.812269+0000","last_active":"2024-04-25T16:52:27.831719+0000","last_peered":"2024-04-25T16:52:27.831719+0000","last_clean":"2024-04-25T16:52:27.831719+0000","last_became_active":"2024-04-25T16:52:25.810831+0000","last_became_peered":"2024-04-25T16:52:25.810831+0000","last_unstale":"2024-04-25T16:52:27.831719+0000","last_undegraded":"2024-04-25T16:52:27.831719+0000","last_fullsized":"2024-04-25T16:52:27.831719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831522+0000","last_change":"2024-04-25T16:52:26.806939+0000","last_active":"2024-04-25T16:52:27.831522+0000","last_peered":"2024-04-25T16:52:27.831522+0000","last_clean":"2024-04-25T16:52:27.831522+0000","last_became_active":"2024-04-25T16:52:26.806053+0000","last_became_peered":"2024-04-25T16:52:26.806053+0000","last_unstale":"2024-04-25T16:52:27.831522+0000","last_undegraded":"2024-04-25T16:52:27.831522+0000","last_fullsized":"2024-04-25T16:52:27.831522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781168+0000","last_change":"2024-04-25T16:52:25.783759+0000","last_active":"2024-04-25T16:52:27.781168+0000","last_peered":"2024-04-25T16:52:27.781168+0000","last_clean":"2024-04-25T16:52:27.781168+0000","last_became_active":"2024-04-25T16:52:25.783353+0000","last_became_peered":"2024-04-25T16:52:25.783353+0000","last_unstale":"2024-04-25T16:52:27.781168+0000","last_undegraded":"2024-04-25T16:52:27.781168+0000","last_fullsized":"2024-04-25T16:52:27.781168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:34.281502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780945+0000","last_change":"2024-04-25T16:52:26.798434+0000","last_active":"2024-04-25T16:52:27.780945+0000","last_peered":"2024-04-25T16:52:27.780945+0000","last_clean":"2024-04-25T16:52:27.780945+0000","last_became_active":"2024-04-25T16:52:26.797631+0000","last_became_peered":"2024-04-25T16:52:26.797631+0000","last_unstale":"2024-04-25T16:52:27.780945+0000","last_undegraded":"2024-04-25T16:52:27.780945+0000","last_fullsized":"2024-04-25T16:52:27.780945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:16.248387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831612+0000","last_change":"2024-04-25T16:52:25.819136+0000","last_active":"2024-04-25T16:52:27.831612+0000","last_peered":"2024-04-25T16:52:27.831612+0000","last_clean":"2024-04-25T16:52:27.831612+0000","last_became_active":"2024-04-25T16:52:25.818809+0000","last_became_peered":"2024-04-25T16:52:25.818809+0000","last_unstale":"2024-04-25T16:52:27.831612+0000","last_undegraded":"2024-04-25T16:52:27.831612+0000","last_fullsized":"2024-04-25T16:52:27.831612+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782782+0000","last_change":"2024-04-25T16:52:26.819612+0000","last_active":"2024-04-25T16:52:27.782782+0000","last_peered":"2024-04-25T16:52:27.782782+0000","last_clean":"2024-04-25T16:52:27.782782+0000","last_became_active":"2024-04-25T16:52:26.818874+0000","last_became_peered":"2024-04-25T16:52:26.818874+0000","last_unstale":"2024-04-25T16:52:27.782782+0000","last_undegraded":"2024-04-25T16:52:27.782782+0000","last_fullsized":"2024-04-25T16:52:27.782782+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781486+0000","last_change":"2024-04-25T16:52:25.780874+0000","last_active":"2024-04-25T16:52:27.781486+0000","last_peered":"2024-04-25T16:52:27.781486+0000","last_clean":"2024-04-25T16:52:27.781486+0000","last_became_active":"2024-04-25T16:52:25.780529+0000","last_became_peered":"2024-04-25T16:52:25.780529+0000","last_unstale":"2024-04-25T16:52:27.781486+0000","last_undegraded":"2024-04-25T16:52:27.781486+0000","last_fullsized":"2024-04-25T16:52:27.781486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:57.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831873+0000","last_change":"2024-04-25T16:52:26.845027+0000","last_active":"2024-04-25T16:52:27.831873+0000","last_peered":"2024-04-25T16:52:27.831873+0000","last_clean":"2024-04-25T16:52:27.831873+0000","last_became_active":"2024-04-25T16:52:26.844620+0000","last_became_peered":"2024-04-25T16:52:26.844620+0000","last_unstale":"2024-04-25T16:52:27.831873+0000","last_undegraded":"2024-04-25T16:52:27.831873+0000","last_fullsized":"2024-04-25T16:52:27.831873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780947+0000","last_change":"2024-04-25T16:52:25.821889+0000","last_active":"2024-04-25T16:52:27.780947+0000","last_peered":"2024-04-25T16:52:27.780947+0000","last_clean":"2024-04-25T16:52:27.780947+0000","last_became_active":"2024-04-25T16:52:25.821643+0000","last_became_peered":"2024-04-25T16:52:25.821643+0000","last_unstale":"2024-04-25T16:52:27.780947+0000","last_undegraded":"2024-04-25T16:52:27.780947+0000","last_fullsized":"2024-04-25T16:52:27.780947+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:27.786669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823650+0000","last_change":"2024-04-25T16:52:26.802864+0000","last_active":"2024-04-25T16:52:27.823650+0000","last_peered":"2024-04-25T16:52:27.823650+0000","last_clean":"2024-04-25T16:52:27.823650+0000","last_became_active":"2024-04-25T16:52:26.801913+0000","last_became_peered":"2024-04-25T16:52:26.801913+0000","last_unstale":"2024-04-25T16:52:27.823650+0000","last_undegraded":"2024-04-25T16:52:27.823650+0000","last_fullsized":"2024-04-25T16:52:27.823650+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782146+0000","last_change":"2024-04-25T16:52:25.806124+0000","last_active":"2024-04-25T16:52:27.782146+0000","last_peered":"2024-04-25T16:52:27.782146+0000","last_clean":"2024-04-25T16:52:27.782146+0000","last_became_active":"2024-04-25T16:52:25.805306+0000","last_became_peered":"2024-04-25T16:52:25.805306+0000","last_unstale":"2024-04-25T16:52:27.782146+0000","last_undegraded":"2024-04-25T16:52:27.782146+0000","last_fullsized":"2024-04-25T16:52:27.782146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829948+0000","last_change":"2024-04-25T16:52:26.819372+0000","last_active":"2024-04-25T16:52:27.829948+0000","last_peered":"2024-04-25T16:52:27.829948+0000","last_clean":"2024-04-25T16:52:27.829948+0000","last_became_active":"2024-04-25T16:52:26.807182+0000","last_became_peered":"2024-04-25T16:52:26.807182+0000","last_unstale":"2024-04-25T16:52:27.829948+0000","last_undegraded":"2024-04-25T16:52:27.829948+0000","last_fullsized":"2024-04-25T16:52:27.829948+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:14.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780505+0000","last_change":"2024-04-25T16:52:25.788959+0000","last_active":"2024-04-25T16:52:27.780505+0000","last_peered":"2024-04-25T16:52:27.780505+0000","last_clean":"2024-04-25T16:52:27.780505+0000","last_became_active":"2024-04-25T16:52:25.788261+0000","last_became_peered":"2024-04-25T16:52:25.788261+0000","last_unstale":"2024-04-25T16:52:27.780505+0000","last_undegraded":"2024-04-25T16:52:27.780505+0000","last_fullsized":"2024-04-25T16:52:27.780505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:13.943131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831765+0000","last_change":"2024-04-25T16:52:26.819641+0000","last_active":"2024-04-25T16:52:27.831765+0000","last_peered":"2024-04-25T16:52:27.831765+0000","last_clean":"2024-04-25T16:52:27.831765+0000","last_became_active":"2024-04-25T16:52:26.819314+0000","last_became_peered":"2024-04-25T16:52:26.819314+0000","last_unstale":"2024-04-25T16:52:27.831765+0000","last_undegraded":"2024-04-25T16:52:27.831765+0000","last_fullsized":"2024-04-25T16:52:27.831765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823426+0000","last_change":"2024-04-25T16:52:25.806269+0000","last_active":"2024-04-25T16:52:27.823426+0000","last_peered":"2024-04-25T16:52:27.823426+0000","last_clean":"2024-04-25T16:52:27.823426+0000","last_became_active":"2024-04-25T16:52:25.804183+0000","last_became_peered":"2024-04-25T16:52:25.804183+0000","last_unstale":"2024-04-25T16:52:27.823426+0000","last_undegraded":"2024-04-25T16:52:27.823426+0000","last_fullsized":"2024-04-25T16:52:27.823426+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:07.902516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.783237+0000","last_change":"2024-04-25T16:52:26.841753+0000","last_active":"2024-04-25T16:52:27.783237+0000","last_peered":"2024-04-25T16:52:27.783237+0000","last_clean":"2024-04-25T16:52:27.783237+0000","last_became_active":"2024-04-25T16:52:26.841112+0000","last_became_peered":"2024-04-25T16:52:26.841112+0000","last_unstale":"2024-04-25T16:52:27.783237+0000","last_undegraded":"2024-04-25T16:52:27.783237+0000","last_fullsized":"2024-04-25T16:52:27.783237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781212+0000","last_change":"2024-04-25T16:52:25.803168+0000","last_active":"2024-04-25T16:52:27.781212+0000","last_peered":"2024-04-25T16:52:27.781212+0000","last_clean":"2024-04-25T16:52:27.781212+0000","last_became_active":"2024-04-25T16:52:25.802880+0000","last_became_peered":"2024-04-25T16:52:25.802880+0000","last_unstale":"2024-04-25T16:52:27.781212+0000","last_undegraded":"2024-04-25T16:52:27.781212+0000","last_fullsized":"2024-04-25T16:52:27.781212+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.210482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829616+0000","last_change":"2024-04-25T16:52:26.819781+0000","last_active":"2024-04-25T16:52:27.829616+0000","last_peered":"2024-04-25T16:52:27.829616+0000","last_clean":"2024-04-25T16:52:27.829616+0000","last_became_active":"2024-04-25T16:52:26.819467+0000","last_became_peered":"2024-04-25T16:52:26.819467+0000","last_unstale":"2024-04-25T16:52:27.829616+0000","last_undegraded":"2024-04-25T16:52:27.829616+0000","last_fullsized":"2024-04-25T16:52:27.829616+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832139+0000","last_change":"2024-04-25T16:52:25.799953+0000","last_active":"2024-04-25T16:52:27.832139+0000","last_peered":"2024-04-25T16:52:27.832139+0000","last_clean":"2024-04-25T16:52:27.832139+0000","last_became_active":"2024-04-25T16:52:25.799689+0000","last_became_peered":"2024-04-25T16:52:25.799689+0000","last_unstale":"2024-04-25T16:52:27.832139+0000","last_undegraded":"2024-04-25T16:52:27.832139+0000","last_fullsized":"2024-04-25T16:52:27.832139+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831852+0000","last_change":"2024-04-25T16:52:26.799367+0000","last_active":"2024-04-25T16:52:27.831852+0000","last_peered":"2024-04-25T16:52:27.831852+0000","last_clean":"2024-04-25T16:52:27.831852+0000","last_became_active":"2024-04-25T16:52:26.797337+0000","last_became_peered":"2024-04-25T16:52:26.797337+0000","last_unstale":"2024-04-25T16:52:27.831852+0000","last_undegraded":"2024-04-25T16:52:27.831852+0000","last_fullsized":"2024-04-25T16:52:27.831852+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832932+0000","last_change":"2024-04-25T16:52:25.806956+0000","last_active":"2024-04-25T16:52:27.832932+0000","last_peered":"2024-04-25T16:52:27.832932+0000","last_clean":"2024-04-25T16:52:27.832932+0000","last_became_active":"2024-04-25T16:52:25.806604+0000","last_became_peered":"2024-04-25T16:52:25.806604+0000","last_unstale":"2024-04-25T16:52:27.832932+0000","last_undegraded":"2024-04-25T16:52:27.832932+0000","last_fullsized":"2024-04-25T16:52:27.832932+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831008+0000","last_change":"2024-04-25T16:52:26.820845+0000","last_active":"2024-04-25T16:52:27.831008+0000","last_peered":"2024-04-25T16:52:27.831008+0000","last_clean":"2024-04-25T16:52:27.831008+0000","last_became_active":"2024-04-25T16:52:26.820484+0000","last_became_peered":"2024-04-25T16:52:26.820484+0000","last_unstale":"2024-04-25T16:52:27.831008+0000","last_undegraded":"2024-04-25T16:52:27.831008+0000","last_fullsized":"2024-04-25T16:52:27.831008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:44.359144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.779900+0000","last_change":"2024-04-25T16:52:25.785723+0000","last_active":"2024-04-25T16:52:27.779900+0000","last_peered":"2024-04-25T16:52:27.779900+0000","last_clean":"2024-04-25T16:52:27.779900+0000","last_became_active":"2024-04-25T16:52:25.784930+0000","last_became_peered":"2024-04-25T16:52:25.784930+0000","last_unstale":"2024-04-25T16:52:27.779900+0000","last_undegraded":"2024-04-25T16:52:27.779900+0000","last_fullsized":"2024-04-25T16:52:27.779900+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830715+0000","last_change":"2024-04-25T16:52:26.799277+0000","last_active":"2024-04-25T16:52:27.830715+0000","last_peered":"2024-04-25T16:52:27.830715+0000","last_clean":"2024-04-25T16:52:27.830715+0000","last_became_active":"2024-04-25T16:52:26.797817+0000","last_became_peered":"2024-04-25T16:52:26.797817+0000","last_unstale":"2024-04-25T16:52:27.830715+0000","last_undegraded":"2024-04-25T16:52:27.830715+0000","last_fullsized":"2024-04-25T16:52:27.830715+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830490+0000","last_change":"2024-04-25T16:52:25.810628+0000","last_active":"2024-04-25T16:52:27.830490+0000","last_peered":"2024-04-25T16:52:27.830490+0000","last_clean":"2024-04-25T16:52:27.830490+0000","last_became_active":"2024-04-25T16:52:25.809669+0000","last_became_peered":"2024-04-25T16:52:25.809669+0000","last_unstale":"2024-04-25T16:52:27.830490+0000","last_undegraded":"2024-04-25T16:52:27.830490+0000","last_fullsized":"2024-04-25T16:52:27.830490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:36.638682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782480+0000","last_change":"2024-04-25T16:52:26.832124+0000","last_active":"2024-04-25T16:52:27.782480+0000","last_peered":"2024-04-25T16:52:27.782480+0000","last_clean":"2024-04-25T16:52:27.782480+0000","last_became_active":"2024-04-25T16:52:26.831437+0000","last_became_peered":"2024-04-25T16:52:26.831437+0000","last_unstale":"2024-04-25T16:52:27.782480+0000","last_undegraded":"2024-04-25T16:52:27.782480+0000","last_fullsized":"2024-04-25T16:52:27.782480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829356+0000","last_change":"2024-04-25T16:52:25.793944+0000","last_active":"2024-04-25T16:52:27.829356+0000","last_peered":"2024-04-25T16:52:27.829356+0000","last_clean":"2024-04-25T16:52:27.829356+0000","last_became_active":"2024-04-25T16:52:25.793522+0000","last_became_peered":"2024-04-25T16:52:25.793522+0000","last_unstale":"2024-04-25T16:52:27.829356+0000","last_undegraded":"2024-04-25T16:52:27.829356+0000","last_fullsized":"2024-04-25T16:52:27.829356+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:41.542439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822709+0000","last_change":"2024-04-25T16:52:26.804643+0000","last_active":"2024-04-25T16:52:27.822709+0000","last_peered":"2024-04-25T16:52:27.822709+0000","last_clean":"2024-04-25T16:52:27.822709+0000","last_became_active":"2024-04-25T16:52:26.802297+0000","last_became_peered":"2024-04-25T16:52:26.802297+0000","last_unstale":"2024-04-25T16:52:27.822709+0000","last_undegraded":"2024-04-25T16:52:27.822709+0000","last_fullsized":"2024-04-25T16:52:27.822709+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782714+0000","last_change":"2024-04-25T16:52:25.796075+0000","last_active":"2024-04-25T16:52:27.782714+0000","last_peered":"2024-04-25T16:52:27.782714+0000","last_clean":"2024-04-25T16:52:27.782714+0000","last_became_active":"2024-04-25T16:52:25.795226+0000","last_became_peered":"2024-04-25T16:52:25.795226+0000","last_unstale":"2024-04-25T16:52:27.782714+0000","last_undegraded":"2024-04-25T16:52:27.782714+0000","last_fullsized":"2024-04-25T16:52:27.782714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":143,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782982+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:52:27.782982+0000","last_peered":"2024-04-25T16:52:27.782982+0000","last_clean":"2024-04-25T16:52:27.782982+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:52:27.782982+0000","last_undegraded":"2024-04-25T16:52:27.782982+0000","last_fullsized":"2024-04-25T16:52:27.782982+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":190610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":446464,"data_stored":190632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35788,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336052,"statfs":{"total":96636764160,"available":96600117248,"internally_reserved":0,"allocated":454656,"data_stored":191674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":190098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35776,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336064,"statfs":{"total":96636764160,"available":96600129536,"internally_reserved":0,"allocated":442368,"data_stored":190118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335484,"statfs":{"total":96636764160,"available":96599535616,"internally_reserved":0,"allocated":1036288,"data_stored":780430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:52:31.740 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T16:52:31.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.923+0000 7faad66d2640 1 Processor -- start 2024-04-25T16:52:31.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.923+0000 7faad66d2640 1 -- start start 2024-04-25T16:52:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.923+0000 7faad66d2640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faad006f6c0 0x7faad00d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.923+0000 7faad66d2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faad015ff50 0x7faad00d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad66d2640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faad00b7050 0x7faad00b7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad66d2640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7faad00d2fd0 con 0x7faad00b7050 2024-04-25T16:52:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad66d2640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7faad00d2070 con 0x7faad006f6c0 2024-04-25T16:52:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad66d2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faad00d2aa0 con 0x7faad015ff50 2024-04-25T16:52:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad4c6f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faad00b7050 0x7faad00b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad4c6f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faad00b7050 0x7faad00b7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59054/0 (socket says 172.21.15.94:59054) 2024-04-25T16:52:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad4c6f640 1 -- 172.21.15.94:0/296044842 learned_addr learned my addr 172.21.15.94:0/296044842 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faacffff640 1 --2- 172.21.15.94:0/296044842 >> v2:172.21.15.138:3300/0 conn(0x7faad006f6c0 0x7faad00d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faacf7fe640 1 --2- 172.21.15.94:0/296044842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faad015ff50 0x7faad00d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad4c6f640 1 -- 172.21.15.94:0/296044842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faad015ff50 msgr2=0x7faad00d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:31.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad4c6f640 1 --2- 172.21.15.94:0/296044842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faad015ff50 0x7faad00d3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad4c6f640 1 -- 172.21.15.94:0/296044842 >> v2:172.21.15.138:3300/0 conn(0x7faad006f6c0 msgr2=0x7faad00d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:31.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad4c6f640 1 --2- 172.21.15.94:0/296044842 >> v2:172.21.15.138:3300/0 conn(0x7faad006f6c0 0x7faad00d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad4c6f640 1 -- 172.21.15.94:0/296044842 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad00d1d40 con 0x7faad00b7050 2024-04-25T16:52:31.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faacffff640 1 --2- 172.21.15.94:0/296044842 >> v2:172.21.15.138:3300/0 conn(0x7faad006f6c0 0x7faad00d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:31.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad4c6f640 1 --2- 172.21.15.94:0/296044842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad00b7050 0x7faad00b7460 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faad00d1f10 tx=0x7faac801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f76b32a55fff24e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:31.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faaceffd640 1 -- 172.21.15.94:0/296044842 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac80aa660 con 0x7faad00b7050 2024-04-25T16:52:31.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faaceffd640 1 -- 172.21.15.94:0/296044842 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac80af070 con 0x7faad00b7050 2024-04-25T16:52:31.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faaceffd640 1 -- 172.21.15.94:0/296044842 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac80aaa20 con 0x7faad00b7050 2024-04-25T16:52:31.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad66d2640 1 -- 172.21.15.94:0/296044842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad00b7050 msgr2=0x7faad00b7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad66d2640 1 --2- 172.21.15.94:0/296044842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad00b7050 0x7faad00b7460 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faad00d1f10 tx=0x7faac801c520 comp rx=0 tx=0).stop 2024-04-25T16:52:31.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad66d2640 1 -- 172.21.15.94:0/296044842 shutdown_connections 2024-04-25T16:52:31.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad66d2640 1 --2- 172.21.15.94:0/296044842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad00b7050 0x7faad00b7460 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad66d2640 1 --2- 172.21.15.94:0/296044842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faad015ff50 0x7faad00d3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad66d2640 1 --2- 172.21.15.94:0/296044842 >> v2:172.21.15.138:3300/0 conn(0x7faad006f6c0 0x7faad00d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad66d2640 1 -- 172.21.15.94:0/296044842 >> 172.21.15.94:0/296044842 conn(0x7faad009e950 msgr2=0x7faad008d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:31.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad66d2640 1 -- 172.21.15.94:0/296044842 shutdown_connections 2024-04-25T16:52:31.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.927+0000 7faad66d2640 1 -- 172.21.15.94:0/296044842 wait complete. 2024-04-25T16:52:31.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faad66d2640 1 Processor -- start 2024-04-25T16:52:31.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faad66d2640 1 -- start start 2024-04-25T16:52:31.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faad66d2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faad006f6c0 0x7faad0150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faad66d2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad00b7050 0x7faad00cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faad66d2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faad015ff50 0x7faad0155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faad66d2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faad006fd90 con 0x7faad00b7050 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faad66d2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faad006fc10 con 0x7faad006f6c0 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faad66d2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faad00d5600 con 0x7faad015ff50 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faacf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad00b7050 0x7faad00cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faacf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad00b7050 0x7faad00cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59068/0 (socket says 172.21.15.94:59068) 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faacf7fe640 1 -- 172.21.15.94:0/1433519391 learned_addr learned my addr 172.21.15.94:0/1433519391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faacffff640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faad006f6c0 0x7faad0150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faad4c6f640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faad015ff50 0x7faad0155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faacf7fe640 1 -- 172.21.15.94:0/1433519391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faad015ff50 msgr2=0x7faad0155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faacf7fe640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faad015ff50 0x7faad0155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faacf7fe640 1 -- 172.21.15.94:0/1433519391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faad006f6c0 msgr2=0x7faad0150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faacf7fe640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faad006f6c0 0x7faad0150d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faacf7fe640 1 -- 172.21.15.94:0/1433519391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad0152980 con 0x7faad00b7050 2024-04-25T16:52:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faacf7fe640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad00b7050 0x7faad00cc870 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faac400d620 tx=0x7faac400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:31.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faaccff9640 1 -- 172.21.15.94:0/1433519391 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac400b620 con 0x7faad00b7050 2024-04-25T16:52:31.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faaccff9640 1 -- 172.21.15.94:0/1433519391 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac4019070 con 0x7faad00b7050 2024-04-25T16:52:31.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faad66d2640 1 -- 172.21.15.94:0/1433519391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad00cefa0 con 0x7faad00b7050 2024-04-25T16:52:31.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faaccff9640 1 -- 172.21.15.94:0/1433519391 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac400b9e0 con 0x7faad00b7050 2024-04-25T16:52:31.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.931+0000 7faad66d2640 1 -- 172.21.15.94:0/1433519391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad00cd0a0 con 0x7faad00b7050 2024-04-25T16:52:31.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.935+0000 7faad66d2640 1 -- 172.21.15.94:0/1433519391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad0076830 con 0x7faad00b7050 2024-04-25T16:52:31.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.935+0000 7faaccff9640 1 -- 172.21.15.94:0/1433519391 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faac400bb80 con 0x7faad00b7050 2024-04-25T16:52:31.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.935+0000 7faaccff9640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faabc06f430 0x7faabc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:31.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.935+0000 7faaccff9640 1 -- 172.21.15.94:0/1433519391 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7faac408f8b0 con 0x7faad00b7050 2024-04-25T16:52:31.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.935+0000 7faacffff640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faabc06f430 0x7faabc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:31.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.935+0000 7faacffff640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faabc06f430 0x7faabc071940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faad00d0280 tx=0x7faac000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:31.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:31.939+0000 7faaccff9640 1 -- 172.21.15.94:0/1433519391 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faac405c610 con 0x7faad00b7050 2024-04-25T16:52:32.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.163+0000 7faad66d2640 1 -- 172.21.15.94:0/1433519391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7faad0075cc0 con 0x7faad00b7050 2024-04-25T16:52:32.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.163+0000 7faaccff9640 1 -- 172.21.15.94:0/1433519391 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1013 (secure 0 0 0) 0x7faac405bf10 con 0x7faad00b7050 2024-04-25T16:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:32.162 INFO:teuthology.orchestra.run.smithi094.stdout:{"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":123,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":38,"num_osds":8,"num_up_osds":8,"osd_up_since":1714063840,"num_in_osds":8,"osd_in_since":1714063831,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":264474624,"bytes_avail":772829638656,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5442}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-25T16:51:59.684427+0000","services":{}},"progress_events":{}} 2024-04-25T16:52:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.167+0000 7faad66d2640 1 -- 172.21.15.94:0/1433519391 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faabc06f430 msgr2=0x7faabc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.167+0000 7faad66d2640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faabc06f430 0x7faabc071940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faad00d0280 tx=0x7faac000b040 comp rx=0 tx=0).stop 2024-04-25T16:52:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.167+0000 7faad66d2640 1 -- 172.21.15.94:0/1433519391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad00b7050 msgr2=0x7faad00cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.167+0000 7faad66d2640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad00b7050 0x7faad00cc870 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faac400d620 tx=0x7faac400dae0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.167+0000 7faad66d2640 1 -- 172.21.15.94:0/1433519391 shutdown_connections 2024-04-25T16:52:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.167+0000 7faad66d2640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faabc06f430 0x7faabc071940 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.167+0000 7faad66d2640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faad015ff50 0x7faad0155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.167+0000 7faad66d2640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad00b7050 0x7faad00cc870 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.167+0000 7faad66d2640 1 --2- 172.21.15.94:0/1433519391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faad006f6c0 0x7faad0150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.167+0000 7faad66d2640 1 -- 172.21.15.94:0/1433519391 >> 172.21.15.94:0/1433519391 conn(0x7faad009e950 msgr2=0x7faad0160480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.167+0000 7faad66d2640 1 -- 172.21.15.94:0/1433519391 shutdown_connections 2024-04-25T16:52:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.167+0000 7faad66d2640 1 -- 172.21.15.94:0/1433519391 wait complete. 2024-04-25T16:52:32.182 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T16:52:32.183 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:52:32.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe592ea3640 1 Processor -- start 2024-04-25T16:52:32.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe592ea3640 1 -- start start 2024-04-25T16:52:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe592ea3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe58c06ff60 0x7fe58c06d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe592ea3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe58c070370 0x7fe58c06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe592ea3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe58c1637c0 0x7fe58c163bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe592ea3640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe58c164110 con 0x7fe58c1637c0 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe592ea3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe58c06ebe0 con 0x7fe58c070370 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe592ea3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe58c06eec0 con 0x7fe58c06ff60 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe591440640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe58c1637c0 0x7fe58c163bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe591440640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe58c1637c0 0x7fe58c163bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59082/0 (socket says 172.21.15.94:59082) 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe591440640 1 -- 172.21.15.94:0/2683323673 learned_addr learned my addr 172.21.15.94:0/2683323673 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe58bfff640 1 --2- 172.21.15.94:0/2683323673 >> v2:172.21.15.138:3300/0 conn(0x7fe58c070370 0x7fe58c06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe590c3f640 1 --2- 172.21.15.94:0/2683323673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe58c06ff60 0x7fe58c06d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe591440640 1 -- 172.21.15.94:0/2683323673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe58c06ff60 msgr2=0x7fe58c06d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe591440640 1 --2- 172.21.15.94:0/2683323673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe58c06ff60 0x7fe58c06d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe591440640 1 -- 172.21.15.94:0/2683323673 >> v2:172.21.15.138:3300/0 conn(0x7fe58c070370 msgr2=0x7fe58c06dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe591440640 1 --2- 172.21.15.94:0/2683323673 >> v2:172.21.15.138:3300/0 conn(0x7fe58c070370 0x7fe58c06dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe591440640 1 -- 172.21.15.94:0/2683323673 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c06f1a0 con 0x7fe58c1637c0 2024-04-25T16:52:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.335+0000 7fe58bfff640 1 --2- 172.21.15.94:0/2683323673 >> v2:172.21.15.138:3300/0 conn(0x7fe58c070370 0x7fe58c06dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.339+0000 7fe591440640 1 --2- 172.21.15.94:0/2683323673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe58c1637c0 0x7fe58c163bd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe58c06e1a0 tx=0x7fe58401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bf8992bac60b74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.339+0000 7fe58b7fe640 1 -- 172.21.15.94:0/2683323673 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5840b4070 con 0x7fe58c1637c0 2024-04-25T16:52:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.339+0000 7fe58b7fe640 1 -- 172.21.15.94:0/2683323673 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5840af400 con 0x7fe58c1637c0 2024-04-25T16:52:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.339+0000 7fe58b7fe640 1 -- 172.21.15.94:0/2683323673 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5840af7d0 con 0x7fe58c1637c0 2024-04-25T16:52:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.339+0000 7fe592ea3640 1 -- 172.21.15.94:0/2683323673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe58c1637c0 msgr2=0x7fe58c163bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.339+0000 7fe592ea3640 1 --2- 172.21.15.94:0/2683323673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe58c1637c0 0x7fe58c163bd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe58c06e1a0 tx=0x7fe58401f1e0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.339+0000 7fe592ea3640 1 -- 172.21.15.94:0/2683323673 shutdown_connections 2024-04-25T16:52:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.339+0000 7fe592ea3640 1 --2- 172.21.15.94:0/2683323673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe58c1637c0 0x7fe58c163bd0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.339+0000 7fe592ea3640 1 --2- 172.21.15.94:0/2683323673 >> v2:172.21.15.138:3300/0 conn(0x7fe58c070370 0x7fe58c06dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.339+0000 7fe592ea3640 1 --2- 172.21.15.94:0/2683323673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe58c06ff60 0x7fe58c06d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.339+0000 7fe592ea3640 1 -- 172.21.15.94:0/2683323673 >> 172.21.15.94:0/2683323673 conn(0x7fe58c0b47f0 msgr2=0x7fe58c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.339+0000 7fe592ea3640 1 -- 172.21.15.94:0/2683323673 shutdown_connections 2024-04-25T16:52:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.339+0000 7fe592ea3640 1 -- 172.21.15.94:0/2683323673 wait complete. 2024-04-25T16:52:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe592ea3640 1 Processor -- start 2024-04-25T16:52:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe592ea3640 1 -- start start 2024-04-25T16:52:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe592ea3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe58c06ff60 0x7fe58c14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe592ea3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe58c070370 0x7fe58c161630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe592ea3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe58c1637c0 0x7fe58c151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe592ea3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c06fa40 con 0x7fe58c06ff60 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe592ea3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c06f760 con 0x7fe58c1637c0 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe592ea3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe58c0ae170 con 0x7fe58c070370 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe590c3f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe58c06ff60 0x7fe58c14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe590c3f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe58c06ff60 0x7fe58c14f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59088/0 (socket says 172.21.15.94:59088) 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe590c3f640 1 -- 172.21.15.94:0/2532025899 learned_addr learned my addr 172.21.15.94:0/2532025899 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe58bfff640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe58c070370 0x7fe58c161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe591440640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe58c1637c0 0x7fe58c151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe590c3f640 1 -- 172.21.15.94:0/2532025899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe58c070370 msgr2=0x7fe58c161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe590c3f640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe58c070370 0x7fe58c161630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe590c3f640 1 -- 172.21.15.94:0/2532025899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe58c1637c0 msgr2=0x7fe58c151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe590c3f640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe58c1637c0 0x7fe58c151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe590c3f640 1 -- 172.21.15.94:0/2532025899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c154340 con 0x7fe58c06ff60 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe591440640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe58c1637c0 0x7fe58c151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe590c3f640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe58c06ff60 0x7fe58c14f940 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe58000d800 tx=0x7fe58000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe5897fa640 1 -- 172.21.15.94:0/2532025899 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580018030 con 0x7fe58c06ff60 2024-04-25T16:52:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe5897fa640 1 -- 172.21.15.94:0/2532025899 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5800096a0 con 0x7fe58c06ff60 2024-04-25T16:52:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe592ea3640 1 -- 172.21.15.94:0/2532025899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58c154620 con 0x7fe58c06ff60 2024-04-25T16:52:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe5897fa640 1 -- 172.21.15.94:0/2532025899 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580009870 con 0x7fe58c06ff60 2024-04-25T16:52:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe592ea3640 1 -- 172.21.15.94:0/2532025899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe58c154be0 con 0x7fe58c06ff60 2024-04-25T16:52:32.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.343+0000 7fe592ea3640 1 -- 172.21.15.94:0/2532025899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe58c076830 con 0x7fe58c06ff60 2024-04-25T16:52:32.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.347+0000 7fe5897fa640 1 -- 172.21.15.94:0/2532025899 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe58001b070 con 0x7fe58c06ff60 2024-04-25T16:52:32.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.347+0000 7fe5897fa640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe57806f400 0x7fe578071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:32.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.347+0000 7fe5897fa640 1 -- 172.21.15.94:0/2532025899 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe58008f7c0 con 0x7fe58c06ff60 2024-04-25T16:52:32.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.347+0000 7fe58bfff640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe57806f400 0x7fe578071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:32.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.347+0000 7fe58bfff640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe57806f400 0x7fe578071910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe58c1f9df0 tx=0x7fe57c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:32.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.351+0000 7fe5897fa640 1 -- 172.21.15.94:0/2532025899 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe58005c480 con 0x7fe58c06ff60 2024-04-25T16:52:32.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.507+0000 7fe592ea3640 1 -- 172.21.15.94:0/2532025899 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe58c06e9c0 con 0x7fe57806f400 2024-04-25T16:52:32.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.515+0000 7fe5897fa640 1 -- 172.21.15.94:0/2532025899 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7fe58c06e9c0 con 0x7fe57806f400 2024-04-25T16:52:32.517 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:32.518 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:52:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.519+0000 7fe592ea3640 1 -- 172.21.15.94:0/2532025899 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe57806f400 msgr2=0x7fe578071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.519+0000 7fe592ea3640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe57806f400 0x7fe578071910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe58c1f9df0 tx=0x7fe57c008040 comp rx=0 tx=0).stop 2024-04-25T16:52:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.519+0000 7fe592ea3640 1 -- 172.21.15.94:0/2532025899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe58c06ff60 msgr2=0x7fe58c14f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.519+0000 7fe592ea3640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe58c06ff60 0x7fe58c14f940 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe58000d800 tx=0x7fe58000dcc0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.519+0000 7fe592ea3640 1 -- 172.21.15.94:0/2532025899 shutdown_connections 2024-04-25T16:52:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.519+0000 7fe592ea3640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe58c1637c0 0x7fe58c151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.519+0000 7fe592ea3640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fe57806f400 0x7fe578071910 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.519+0000 7fe592ea3640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe58c070370 0x7fe58c161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.519+0000 7fe592ea3640 1 --2- 172.21.15.94:0/2532025899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe58c06ff60 0x7fe58c14f940 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.519+0000 7fe592ea3640 1 -- 172.21.15.94:0/2532025899 >> 172.21.15.94:0/2532025899 conn(0x7fe58c0b47f0 msgr2=0x7fe58c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.519+0000 7fe592ea3640 1 -- 172.21.15.94:0/2532025899 shutdown_connections 2024-04-25T16:52:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:32.519+0000 7fe592ea3640 1 -- 172.21.15.94:0/2532025899 wait complete. 2024-04-25T16:52:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-04-25T16:52:31.694027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258276,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716444,"statfs":{"total":773094113280,"available":772829638656,"internally_reserved":0,"allocated":4759552,"data_stored":2705780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001852"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830121+0000","last_change":"2024-04-25T16:52:26.825748+0000","last_active":"2024-04-25T16:52:27.830121+0000","last_peered":"2024-04-25T16:52:27.830121+0000","last_clean":"2024-04-25T16:52:27.830121+0000","last_became_active":"2024-04-25T16:52:26.825333+0000","last_became_peered":"2024-04-25T16:52:26.825333+0000","last_unstale":"2024-04-25T16:52:27.830121+0000","last_undegraded":"2024-04-25T16:52:27.830121+0000","last_fullsized":"2024-04-25T16:52:27.830121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:19.084563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829420+0000","last_change":"2024-04-25T16:52:26.795964+0000","last_active":"2024-04-25T16:52:27.829420+0000","last_peered":"2024-04-25T16:52:27.829420+0000","last_clean":"2024-04-25T16:52:27.829420+0000","last_became_active":"2024-04-25T16:52:26.794884+0000","last_became_peered":"2024-04-25T16:52:26.794884+0000","last_unstale":"2024-04-25T16:52:27.829420+0000","last_undegraded":"2024-04-25T16:52:27.829420+0000","last_fullsized":"2024-04-25T16:52:27.829420+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832064+0000","last_change":"2024-04-25T16:52:26.802075+0000","last_active":"2024-04-25T16:52:27.832064+0000","last_peered":"2024-04-25T16:52:27.832064+0000","last_clean":"2024-04-25T16:52:27.832064+0000","last_became_active":"2024-04-25T16:52:26.801203+0000","last_became_peered":"2024-04-25T16:52:26.801203+0000","last_unstale":"2024-04-25T16:52:27.832064+0000","last_undegraded":"2024-04-25T16:52:27.832064+0000","last_fullsized":"2024-04-25T16:52:27.832064+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782213+0000","last_change":"2024-04-25T16:52:26.793541+0000","last_active":"2024-04-25T16:52:27.782213+0000","last_peered":"2024-04-25T16:52:27.782213+0000","last_clean":"2024-04-25T16:52:27.782213+0000","last_became_active":"2024-04-25T16:52:26.789101+0000","last_became_peered":"2024-04-25T16:52:26.789101+0000","last_unstale":"2024-04-25T16:52:27.782213+0000","last_undegraded":"2024-04-25T16:52:27.782213+0000","last_fullsized":"2024-04-25T16:52:27.782213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:38.013058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781321+0000","last_change":"2024-04-25T16:52:26.784467+0000","last_active":"2024-04-25T16:52:27.781321+0000","last_peered":"2024-04-25T16:52:27.781321+0000","last_clean":"2024-04-25T16:52:27.781321+0000","last_became_active":"2024-04-25T16:52:26.783673+0000","last_became_peered":"2024-04-25T16:52:26.783673+0000","last_unstale":"2024-04-25T16:52:27.781321+0000","last_undegraded":"2024-04-25T16:52:27.781321+0000","last_fullsized":"2024-04-25T16:52:27.781321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:36.583897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831274+0000","last_change":"2024-04-25T16:52:26.823413+0000","last_active":"2024-04-25T16:52:27.831274+0000","last_peered":"2024-04-25T16:52:27.831274+0000","last_clean":"2024-04-25T16:52:27.831274+0000","last_became_active":"2024-04-25T16:52:26.823059+0000","last_became_peered":"2024-04-25T16:52:26.823059+0000","last_unstale":"2024-04-25T16:52:27.831274+0000","last_undegraded":"2024-04-25T16:52:27.831274+0000","last_fullsized":"2024-04-25T16:52:27.831274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.821862+0000","last_change":"2024-04-25T16:52:26.808661+0000","last_active":"2024-04-25T16:52:27.821862+0000","last_peered":"2024-04-25T16:52:27.821862+0000","last_clean":"2024-04-25T16:52:27.821862+0000","last_became_active":"2024-04-25T16:52:26.805296+0000","last_became_peered":"2024-04-25T16:52:26.805296+0000","last_unstale":"2024-04-25T16:52:27.821862+0000","last_undegraded":"2024-04-25T16:52:27.821862+0000","last_fullsized":"2024-04-25T16:52:27.821862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829292+0000","last_change":"2024-04-25T16:52:26.820861+0000","last_active":"2024-04-25T16:52:27.829292+0000","last_peered":"2024-04-25T16:52:27.829292+0000","last_clean":"2024-04-25T16:52:27.829292+0000","last_became_active":"2024-04-25T16:52:26.820528+0000","last_became_peered":"2024-04-25T16:52:26.820528+0000","last_unstale":"2024-04-25T16:52:27.829292+0000","last_undegraded":"2024-04-25T16:52:27.829292+0000","last_fullsized":"2024-04-25T16:52:27.829292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:33.962706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832345+0000","last_change":"2024-04-25T16:52:26.803363+0000","last_active":"2024-04-25T16:52:27.832345+0000","last_peered":"2024-04-25T16:52:27.832345+0000","last_clean":"2024-04-25T16:52:27.832345+0000","last_became_active":"2024-04-25T16:52:26.802539+0000","last_became_peered":"2024-04-25T16:52:26.802539+0000","last_unstale":"2024-04-25T16:52:27.832345+0000","last_undegraded":"2024-04-25T16:52:27.832345+0000","last_fullsized":"2024-04-25T16:52:27.832345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822740+0000","last_change":"2024-04-25T16:52:26.842728+0000","last_active":"2024-04-25T16:52:27.822740+0000","last_peered":"2024-04-25T16:52:27.822740+0000","last_clean":"2024-04-25T16:52:27.822740+0000","last_became_active":"2024-04-25T16:52:26.842041+0000","last_became_peered":"2024-04-25T16:52:26.842041+0000","last_unstale":"2024-04-25T16:52:27.822740+0000","last_undegraded":"2024-04-25T16:52:27.822740+0000","last_fullsized":"2024-04-25T16:52:27.822740+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:33.962706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823850+0000","last_change":"2024-04-25T16:52:26.819329+0000","last_active":"2024-04-25T16:52:27.823850+0000","last_peered":"2024-04-25T16:52:27.823850+0000","last_clean":"2024-04-25T16:52:27.823850+0000","last_became_active":"2024-04-25T16:52:26.818528+0000","last_became_peered":"2024-04-25T16:52:26.818528+0000","last_unstale":"2024-04-25T16:52:27.823850+0000","last_undegraded":"2024-04-25T16:52:27.823850+0000","last_fullsized":"2024-04-25T16:52:27.823850+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832046+0000","last_change":"2024-04-25T16:52:26.795942+0000","last_active":"2024-04-25T16:52:27.832046+0000","last_peered":"2024-04-25T16:52:27.832046+0000","last_clean":"2024-04-25T16:52:27.832046+0000","last_became_active":"2024-04-25T16:52:26.795124+0000","last_became_peered":"2024-04-25T16:52:26.795124+0000","last_unstale":"2024-04-25T16:52:27.832046+0000","last_undegraded":"2024-04-25T16:52:27.832046+0000","last_fullsized":"2024-04-25T16:52:27.832046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822028+0000","last_change":"2024-04-25T16:52:26.814749+0000","last_active":"2024-04-25T16:52:27.822028+0000","last_peered":"2024-04-25T16:52:27.822028+0000","last_clean":"2024-04-25T16:52:27.822028+0000","last_became_active":"2024-04-25T16:52:26.814020+0000","last_became_peered":"2024-04-25T16:52:26.814020+0000","last_unstale":"2024-04-25T16:52:27.822028+0000","last_undegraded":"2024-04-25T16:52:27.822028+0000","last_fullsized":"2024-04-25T16:52:27.822028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832107+0000","last_change":"2024-04-25T16:52:26.813118+0000","last_active":"2024-04-25T16:52:27.832107+0000","last_peered":"2024-04-25T16:52:27.832107+0000","last_clean":"2024-04-25T16:52:27.832107+0000","last_became_active":"2024-04-25T16:52:26.812400+0000","last_became_peered":"2024-04-25T16:52:26.812400+0000","last_unstale":"2024-04-25T16:52:27.832107+0000","last_undegraded":"2024-04-25T16:52:27.832107+0000","last_fullsized":"2024-04-25T16:52:27.832107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:16.169654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822023+0000","last_change":"2024-04-25T16:52:26.795864+0000","last_active":"2024-04-25T16:52:27.822023+0000","last_peered":"2024-04-25T16:52:27.822023+0000","last_clean":"2024-04-25T16:52:27.822023+0000","last_became_active":"2024-04-25T16:52:26.794738+0000","last_became_peered":"2024-04-25T16:52:26.794738+0000","last_unstale":"2024-04-25T16:52:27.822023+0000","last_undegraded":"2024-04-25T16:52:27.822023+0000","last_fullsized":"2024-04-25T16:52:27.822023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781621+0000","last_change":"2024-04-25T16:52:26.836640+0000","last_active":"2024-04-25T16:52:27.781621+0000","last_peered":"2024-04-25T16:52:27.781621+0000","last_clean":"2024-04-25T16:52:27.781621+0000","last_became_active":"2024-04-25T16:52:26.833071+0000","last_became_peered":"2024-04-25T16:52:26.833071+0000","last_unstale":"2024-04-25T16:52:27.781621+0000","last_undegraded":"2024-04-25T16:52:27.781621+0000","last_fullsized":"2024-04-25T16:52:27.781621+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.783331+0000","last_change":"2024-04-25T16:52:26.841207+0000","last_active":"2024-04-25T16:52:27.783331+0000","last_peered":"2024-04-25T16:52:27.783331+0000","last_clean":"2024-04-25T16:52:27.783331+0000","last_became_active":"2024-04-25T16:52:26.838750+0000","last_became_peered":"2024-04-25T16:52:26.838750+0000","last_unstale":"2024-04-25T16:52:27.783331+0000","last_undegraded":"2024-04-25T16:52:27.783331+0000","last_fullsized":"2024-04-25T16:52:27.783331+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780940+0000","last_change":"2024-04-25T16:52:26.795939+0000","last_active":"2024-04-25T16:52:27.780940+0000","last_peered":"2024-04-25T16:52:27.780940+0000","last_clean":"2024-04-25T16:52:27.780940+0000","last_became_active":"2024-04-25T16:52:26.795084+0000","last_became_peered":"2024-04-25T16:52:26.795084+0000","last_unstale":"2024-04-25T16:52:27.780940+0000","last_undegraded":"2024-04-25T16:52:27.780940+0000","last_fullsized":"2024-04-25T16:52:27.780940+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781139+0000","last_change":"2024-04-25T16:52:26.818025+0000","last_active":"2024-04-25T16:52:27.781139+0000","last_peered":"2024-04-25T16:52:27.781139+0000","last_clean":"2024-04-25T16:52:27.781139+0000","last_became_active":"2024-04-25T16:52:26.817652+0000","last_became_peered":"2024-04-25T16:52:26.817652+0000","last_unstale":"2024-04-25T16:52:27.781139+0000","last_undegraded":"2024-04-25T16:52:27.781139+0000","last_fullsized":"2024-04-25T16:52:27.781139+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:46.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781002+0000","last_change":"2024-04-25T16:52:26.796384+0000","last_active":"2024-04-25T16:52:27.781002+0000","last_peered":"2024-04-25T16:52:27.781002+0000","last_clean":"2024-04-25T16:52:27.781002+0000","last_became_active":"2024-04-25T16:52:26.795507+0000","last_became_peered":"2024-04-25T16:52:26.795507+0000","last_unstale":"2024-04-25T16:52:27.781002+0000","last_undegraded":"2024-04-25T16:52:27.781002+0000","last_fullsized":"2024-04-25T16:52:27.781002+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822359+0000","last_change":"2024-04-25T16:52:26.809988+0000","last_active":"2024-04-25T16:52:27.822359+0000","last_peered":"2024-04-25T16:52:27.822359+0000","last_clean":"2024-04-25T16:52:27.822359+0000","last_became_active":"2024-04-25T16:52:26.809107+0000","last_became_peered":"2024-04-25T16:52:26.809107+0000","last_unstale":"2024-04-25T16:52:27.822359+0000","last_undegraded":"2024-04-25T16:52:27.822359+0000","last_fullsized":"2024-04-25T16:52:27.822359+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831788+0000","last_change":"2024-04-25T16:52:26.784466+0000","last_active":"2024-04-25T16:52:27.831788+0000","last_peered":"2024-04-25T16:52:27.831788+0000","last_clean":"2024-04-25T16:52:27.831788+0000","last_became_active":"2024-04-25T16:52:26.783550+0000","last_became_peered":"2024-04-25T16:52:26.783550+0000","last_unstale":"2024-04-25T16:52:27.831788+0000","last_undegraded":"2024-04-25T16:52:27.831788+0000","last_fullsized":"2024-04-25T16:52:27.831788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781970+0000","last_change":"2024-04-25T16:52:26.832452+0000","last_active":"2024-04-25T16:52:27.781970+0000","last_peered":"2024-04-25T16:52:27.781970+0000","last_clean":"2024-04-25T16:52:27.781970+0000","last_became_active":"2024-04-25T16:52:26.831753+0000","last_became_peered":"2024-04-25T16:52:26.831753+0000","last_unstale":"2024-04-25T16:52:27.781970+0000","last_undegraded":"2024-04-25T16:52:27.781970+0000","last_fullsized":"2024-04-25T16:52:27.781970+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780752+0000","last_change":"2024-04-25T16:52:26.805633+0000","last_active":"2024-04-25T16:52:27.780752+0000","last_peered":"2024-04-25T16:52:27.780752+0000","last_clean":"2024-04-25T16:52:27.780752+0000","last_became_active":"2024-04-25T16:52:26.804789+0000","last_became_peered":"2024-04-25T16:52:26.804789+0000","last_unstale":"2024-04-25T16:52:27.780752+0000","last_undegraded":"2024-04-25T16:52:27.780752+0000","last_fullsized":"2024-04-25T16:52:27.780752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:08.931465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780486+0000","last_change":"2024-04-25T16:52:26.798693+0000","last_active":"2024-04-25T16:52:27.780486+0000","last_peered":"2024-04-25T16:52:27.780486+0000","last_clean":"2024-04-25T16:52:27.780486+0000","last_became_active":"2024-04-25T16:52:26.797852+0000","last_became_peered":"2024-04-25T16:52:26.797852+0000","last_unstale":"2024-04-25T16:52:27.780486+0000","last_undegraded":"2024-04-25T16:52:27.780486+0000","last_fullsized":"2024-04-25T16:52:27.780486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831376+0000","last_change":"2024-04-25T16:52:26.819630+0000","last_active":"2024-04-25T16:52:27.831376+0000","last_peered":"2024-04-25T16:52:27.831376+0000","last_clean":"2024-04-25T16:52:27.831376+0000","last_became_active":"2024-04-25T16:52:26.819274+0000","last_became_peered":"2024-04-25T16:52:26.819274+0000","last_unstale":"2024-04-25T16:52:27.831376+0000","last_undegraded":"2024-04-25T16:52:27.831376+0000","last_fullsized":"2024-04-25T16:52:27.831376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781126+0000","last_change":"2024-04-25T16:52:26.791433+0000","last_active":"2024-04-25T16:52:27.781126+0000","last_peered":"2024-04-25T16:52:27.781126+0000","last_clean":"2024-04-25T16:52:27.781126+0000","last_became_active":"2024-04-25T16:52:26.790483+0000","last_became_peered":"2024-04-25T16:52:26.790483+0000","last_unstale":"2024-04-25T16:52:27.781126+0000","last_undegraded":"2024-04-25T16:52:27.781126+0000","last_fullsized":"2024-04-25T16:52:27.781126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822953+0000","last_change":"2024-04-25T16:52:26.804860+0000","last_active":"2024-04-25T16:52:27.822953+0000","last_peered":"2024-04-25T16:52:27.822953+0000","last_clean":"2024-04-25T16:52:27.822953+0000","last_became_active":"2024-04-25T16:52:26.804065+0000","last_became_peered":"2024-04-25T16:52:26.804065+0000","last_unstale":"2024-04-25T16:52:27.822953+0000","last_undegraded":"2024-04-25T16:52:27.822953+0000","last_fullsized":"2024-04-25T16:52:27.822953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780617+0000","last_change":"2024-04-25T16:52:26.806705+0000","last_active":"2024-04-25T16:52:27.780617+0000","last_peered":"2024-04-25T16:52:27.780617+0000","last_clean":"2024-04-25T16:52:27.780617+0000","last_became_active":"2024-04-25T16:52:26.805889+0000","last_became_peered":"2024-04-25T16:52:26.805889+0000","last_unstale":"2024-04-25T16:52:27.780617+0000","last_undegraded":"2024-04-25T16:52:27.780617+0000","last_fullsized":"2024-04-25T16:52:27.780617+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780149+0000","last_change":"2024-04-25T16:52:26.798903+0000","last_active":"2024-04-25T16:52:27.780149+0000","last_peered":"2024-04-25T16:52:27.780149+0000","last_clean":"2024-04-25T16:52:27.780149+0000","last_became_active":"2024-04-25T16:52:26.798011+0000","last_became_peered":"2024-04-25T16:52:26.798011+0000","last_unstale":"2024-04-25T16:52:27.780149+0000","last_undegraded":"2024-04-25T16:52:27.780149+0000","last_fullsized":"2024-04-25T16:52:27.780149+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782037+0000","last_change":"2024-04-25T16:52:26.841037+0000","last_active":"2024-04-25T16:52:27.782037+0000","last_peered":"2024-04-25T16:52:27.782037+0000","last_clean":"2024-04-25T16:52:27.782037+0000","last_became_active":"2024-04-25T16:52:26.838194+0000","last_became_peered":"2024-04-25T16:52:26.838194+0000","last_unstale":"2024-04-25T16:52:27.782037+0000","last_undegraded":"2024-04-25T16:52:27.782037+0000","last_fullsized":"2024-04-25T16:52:27.782037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830883+0000","last_change":"2024-04-25T16:52:26.813243+0000","last_active":"2024-04-25T16:52:27.830883+0000","last_peered":"2024-04-25T16:52:27.830883+0000","last_clean":"2024-04-25T16:52:27.830883+0000","last_became_active":"2024-04-25T16:52:26.811807+0000","last_became_peered":"2024-04-25T16:52:26.811807+0000","last_unstale":"2024-04-25T16:52:27.830883+0000","last_undegraded":"2024-04-25T16:52:27.830883+0000","last_fullsized":"2024-04-25T16:52:27.830883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:42.553053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782541+0000","last_change":"2024-04-25T16:52:26.841017+0000","last_active":"2024-04-25T16:52:27.782541+0000","last_peered":"2024-04-25T16:52:27.782541+0000","last_clean":"2024-04-25T16:52:27.782541+0000","last_became_active":"2024-04-25T16:52:26.837543+0000","last_became_peered":"2024-04-25T16:52:26.837543+0000","last_unstale":"2024-04-25T16:52:27.782541+0000","last_undegraded":"2024-04-25T16:52:27.782541+0000","last_fullsized":"2024-04-25T16:52:27.782541+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780599+0000","last_change":"2024-04-25T16:52:25.818069+0000","last_active":"2024-04-25T16:52:27.780599+0000","last_peered":"2024-04-25T16:52:27.780599+0000","last_clean":"2024-04-25T16:52:27.780599+0000","last_became_active":"2024-04-25T16:52:25.817362+0000","last_became_peered":"2024-04-25T16:52:25.817362+0000","last_unstale":"2024-04-25T16:52:27.780599+0000","last_undegraded":"2024-04-25T16:52:27.780599+0000","last_fullsized":"2024-04-25T16:52:27.780599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:08.488136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833310+0000","last_change":"2024-04-25T16:52:26.829008+0000","last_active":"2024-04-25T16:52:27.833310+0000","last_peered":"2024-04-25T16:52:27.833310+0000","last_clean":"2024-04-25T16:52:27.833310+0000","last_became_active":"2024-04-25T16:52:26.828706+0000","last_became_peered":"2024-04-25T16:52:26.828706+0000","last_unstale":"2024-04-25T16:52:27.833310+0000","last_undegraded":"2024-04-25T16:52:27.833310+0000","last_fullsized":"2024-04-25T16:52:27.833310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832100+0000","last_change":"2024-04-25T16:52:25.795967+0000","last_active":"2024-04-25T16:52:27.832100+0000","last_peered":"2024-04-25T16:52:27.832100+0000","last_clean":"2024-04-25T16:52:27.832100+0000","last_became_active":"2024-04-25T16:52:25.795695+0000","last_became_peered":"2024-04-25T16:52:25.795695+0000","last_unstale":"2024-04-25T16:52:27.832100+0000","last_undegraded":"2024-04-25T16:52:27.832100+0000","last_fullsized":"2024-04-25T16:52:27.832100+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831737+0000","last_change":"2024-04-25T16:52:26.789430+0000","last_active":"2024-04-25T16:52:27.831737+0000","last_peered":"2024-04-25T16:52:27.831737+0000","last_clean":"2024-04-25T16:52:27.831737+0000","last_became_active":"2024-04-25T16:52:26.782243+0000","last_became_peered":"2024-04-25T16:52:26.782243+0000","last_unstale":"2024-04-25T16:52:27.831737+0000","last_undegraded":"2024-04-25T16:52:27.831737+0000","last_fullsized":"2024-04-25T16:52:27.831737+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833428+0000","last_change":"2024-04-25T16:52:25.819162+0000","last_active":"2024-04-25T16:52:27.833428+0000","last_peered":"2024-04-25T16:52:27.833428+0000","last_clean":"2024-04-25T16:52:27.833428+0000","last_became_active":"2024-04-25T16:52:25.818859+0000","last_became_peered":"2024-04-25T16:52:25.818859+0000","last_unstale":"2024-04-25T16:52:27.833428+0000","last_undegraded":"2024-04-25T16:52:27.833428+0000","last_fullsized":"2024-04-25T16:52:27.833428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829867+0000","last_change":"2024-04-25T16:52:26.805376+0000","last_active":"2024-04-25T16:52:27.829867+0000","last_peered":"2024-04-25T16:52:27.829867+0000","last_clean":"2024-04-25T16:52:27.829867+0000","last_became_active":"2024-04-25T16:52:26.804542+0000","last_became_peered":"2024-04-25T16:52:26.804542+0000","last_unstale":"2024-04-25T16:52:27.829867+0000","last_undegraded":"2024-04-25T16:52:27.829867+0000","last_fullsized":"2024-04-25T16:52:27.829867+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.620224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831272+0000","last_change":"2024-04-25T16:52:25.815751+0000","last_active":"2024-04-25T16:52:27.831272+0000","last_peered":"2024-04-25T16:52:27.831272+0000","last_clean":"2024-04-25T16:52:27.831272+0000","last_became_active":"2024-04-25T16:52:25.814927+0000","last_became_peered":"2024-04-25T16:52:25.814927+0000","last_unstale":"2024-04-25T16:52:27.831272+0000","last_undegraded":"2024-04-25T16:52:27.831272+0000","last_fullsized":"2024-04-25T16:52:27.831272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831681+0000","last_change":"2024-04-25T16:52:26.814640+0000","last_active":"2024-04-25T16:52:27.831681+0000","last_peered":"2024-04-25T16:52:27.831681+0000","last_clean":"2024-04-25T16:52:27.831681+0000","last_became_active":"2024-04-25T16:52:26.814016+0000","last_became_peered":"2024-04-25T16:52:26.814016+0000","last_unstale":"2024-04-25T16:52:27.831681+0000","last_undegraded":"2024-04-25T16:52:27.831681+0000","last_fullsized":"2024-04-25T16:52:27.831681+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822321+0000","last_change":"2024-04-25T16:52:25.805323+0000","last_active":"2024-04-25T16:52:27.822321+0000","last_peered":"2024-04-25T16:52:27.822321+0000","last_clean":"2024-04-25T16:52:27.822321+0000","last_became_active":"2024-04-25T16:52:25.796444+0000","last_became_peered":"2024-04-25T16:52:25.796444+0000","last_unstale":"2024-04-25T16:52:27.822321+0000","last_undegraded":"2024-04-25T16:52:27.822321+0000","last_fullsized":"2024-04-25T16:52:27.822321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:28.996351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822686+0000","last_change":"2024-04-25T16:52:26.821648+0000","last_active":"2024-04-25T16:52:27.822686+0000","last_peered":"2024-04-25T16:52:27.822686+0000","last_clean":"2024-04-25T16:52:27.822686+0000","last_became_active":"2024-04-25T16:52:26.820835+0000","last_became_peered":"2024-04-25T16:52:26.820835+0000","last_unstale":"2024-04-25T16:52:27.822686+0000","last_undegraded":"2024-04-25T16:52:27.822686+0000","last_fullsized":"2024-04-25T16:52:27.822686+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781761+0000","last_change":"2024-04-25T16:52:25.816250+0000","last_active":"2024-04-25T16:52:27.781761+0000","last_peered":"2024-04-25T16:52:27.781761+0000","last_clean":"2024-04-25T16:52:27.781761+0000","last_became_active":"2024-04-25T16:52:25.815656+0000","last_became_peered":"2024-04-25T16:52:25.815656+0000","last_unstale":"2024-04-25T16:52:27.781761+0000","last_undegraded":"2024-04-25T16:52:27.781761+0000","last_fullsized":"2024-04-25T16:52:27.781761+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831178+0000","last_change":"2024-04-25T16:52:26.813425+0000","last_active":"2024-04-25T16:52:27.831178+0000","last_peered":"2024-04-25T16:52:27.831178+0000","last_clean":"2024-04-25T16:52:27.831178+0000","last_became_active":"2024-04-25T16:52:26.812713+0000","last_became_peered":"2024-04-25T16:52:26.812713+0000","last_unstale":"2024-04-25T16:52:27.831178+0000","last_undegraded":"2024-04-25T16:52:27.831178+0000","last_fullsized":"2024-04-25T16:52:27.831178+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831072+0000","last_change":"2024-04-25T16:52:25.803265+0000","last_active":"2024-04-25T16:52:27.831072+0000","last_peered":"2024-04-25T16:52:27.831072+0000","last_clean":"2024-04-25T16:52:27.831072+0000","last_became_active":"2024-04-25T16:52:25.802617+0000","last_became_peered":"2024-04-25T16:52:25.802617+0000","last_unstale":"2024-04-25T16:52:27.831072+0000","last_undegraded":"2024-04-25T16:52:27.831072+0000","last_fullsized":"2024-04-25T16:52:27.831072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829740+0000","last_change":"2024-04-25T16:52:26.819267+0000","last_active":"2024-04-25T16:52:27.829740+0000","last_peered":"2024-04-25T16:52:27.829740+0000","last_clean":"2024-04-25T16:52:27.829740+0000","last_became_active":"2024-04-25T16:52:26.818740+0000","last_became_peered":"2024-04-25T16:52:26.818740+0000","last_unstale":"2024-04-25T16:52:27.829740+0000","last_undegraded":"2024-04-25T16:52:27.829740+0000","last_fullsized":"2024-04-25T16:52:27.829740+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830189+0000","last_change":"2024-04-25T16:52:25.816663+0000","last_active":"2024-04-25T16:52:27.830189+0000","last_peered":"2024-04-25T16:52:27.830189+0000","last_clean":"2024-04-25T16:52:27.830189+0000","last_became_active":"2024-04-25T16:52:25.815420+0000","last_became_peered":"2024-04-25T16:52:25.815420+0000","last_unstale":"2024-04-25T16:52:27.830189+0000","last_undegraded":"2024-04-25T16:52:27.830189+0000","last_fullsized":"2024-04-25T16:52:27.830189+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:30.288538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782313+0000","last_change":"2024-04-25T16:52:26.789927+0000","last_active":"2024-04-25T16:52:27.782313+0000","last_peered":"2024-04-25T16:52:27.782313+0000","last_clean":"2024-04-25T16:52:27.782313+0000","last_became_active":"2024-04-25T16:52:26.788014+0000","last_became_peered":"2024-04-25T16:52:26.788014+0000","last_unstale":"2024-04-25T16:52:27.782313+0000","last_undegraded":"2024-04-25T16:52:27.782313+0000","last_fullsized":"2024-04-25T16:52:27.782313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:47.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830050+0000","last_change":"2024-04-25T16:52:25.798974+0000","last_active":"2024-04-25T16:52:27.830050+0000","last_peered":"2024-04-25T16:52:27.830050+0000","last_clean":"2024-04-25T16:52:27.830050+0000","last_became_active":"2024-04-25T16:52:25.798314+0000","last_became_peered":"2024-04-25T16:52:25.798314+0000","last_unstale":"2024-04-25T16:52:27.830050+0000","last_undegraded":"2024-04-25T16:52:27.830050+0000","last_fullsized":"2024-04-25T16:52:27.830050+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.348530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782542+0000","last_change":"2024-04-25T16:52:26.842485+0000","last_active":"2024-04-25T16:52:27.782542+0000","last_peered":"2024-04-25T16:52:27.782542+0000","last_clean":"2024-04-25T16:52:27.782542+0000","last_became_active":"2024-04-25T16:52:26.838813+0000","last_became_peered":"2024-04-25T16:52:26.838813+0000","last_unstale":"2024-04-25T16:52:27.782542+0000","last_undegraded":"2024-04-25T16:52:27.782542+0000","last_fullsized":"2024-04-25T16:52:27.782542+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823330+0000","last_change":"2024-04-25T16:52:25.803289+0000","last_active":"2024-04-25T16:52:27.823330+0000","last_peered":"2024-04-25T16:52:27.823330+0000","last_clean":"2024-04-25T16:52:27.823330+0000","last_became_active":"2024-04-25T16:52:25.801415+0000","last_became_peered":"2024-04-25T16:52:25.801415+0000","last_unstale":"2024-04-25T16:52:27.823330+0000","last_undegraded":"2024-04-25T16:52:27.823330+0000","last_fullsized":"2024-04-25T16:52:27.823330+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:45.310089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832151+0000","last_change":"2024-04-25T16:52:26.840379+0000","last_active":"2024-04-25T16:52:27.832151+0000","last_peered":"2024-04-25T16:52:27.832151+0000","last_clean":"2024-04-25T16:52:27.832151+0000","last_became_active":"2024-04-25T16:52:26.840062+0000","last_became_peered":"2024-04-25T16:52:26.840062+0000","last_unstale":"2024-04-25T16:52:27.832151+0000","last_undegraded":"2024-04-25T16:52:27.832151+0000","last_fullsized":"2024-04-25T16:52:27.832151+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823961+0000","last_change":"2024-04-25T16:52:25.809391+0000","last_active":"2024-04-25T16:52:27.823961+0000","last_peered":"2024-04-25T16:52:27.823961+0000","last_clean":"2024-04-25T16:52:27.823961+0000","last_became_active":"2024-04-25T16:52:25.808455+0000","last_became_peered":"2024-04-25T16:52:25.808455+0000","last_unstale":"2024-04-25T16:52:27.823961+0000","last_undegraded":"2024-04-25T16:52:27.823961+0000","last_fullsized":"2024-04-25T16:52:27.823961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831993+0000","last_change":"2024-04-25T16:52:26.782954+0000","last_active":"2024-04-25T16:52:27.831993+0000","last_peered":"2024-04-25T16:52:27.831993+0000","last_clean":"2024-04-25T16:52:27.831993+0000","last_became_active":"2024-04-25T16:52:26.782108+0000","last_became_peered":"2024-04-25T16:52:26.782108+0000","last_unstale":"2024-04-25T16:52:27.831993+0000","last_undegraded":"2024-04-25T16:52:27.831993+0000","last_fullsized":"2024-04-25T16:52:27.831993+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830290+0000","last_change":"2024-04-25T16:52:25.795020+0000","last_active":"2024-04-25T16:52:27.830290+0000","last_peered":"2024-04-25T16:52:27.830290+0000","last_clean":"2024-04-25T16:52:27.830290+0000","last_became_active":"2024-04-25T16:52:25.794633+0000","last_became_peered":"2024-04-25T16:52:25.794633+0000","last_unstale":"2024-04-25T16:52:27.830290+0000","last_undegraded":"2024-04-25T16:52:27.830290+0000","last_fullsized":"2024-04-25T16:52:27.830290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:38.694859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831678+0000","last_change":"2024-04-25T16:52:26.821081+0000","last_active":"2024-04-25T16:52:27.831678+0000","last_peered":"2024-04-25T16:52:27.831678+0000","last_clean":"2024-04-25T16:52:27.831678+0000","last_became_active":"2024-04-25T16:52:26.819337+0000","last_became_peered":"2024-04-25T16:52:26.819337+0000","last_unstale":"2024-04-25T16:52:27.831678+0000","last_undegraded":"2024-04-25T16:52:27.831678+0000","last_fullsized":"2024-04-25T16:52:27.831678+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830009+0000","last_change":"2024-04-25T16:52:25.793039+0000","last_active":"2024-04-25T16:52:27.830009+0000","last_peered":"2024-04-25T16:52:27.830009+0000","last_clean":"2024-04-25T16:52:27.830009+0000","last_became_active":"2024-04-25T16:52:25.792552+0000","last_became_peered":"2024-04-25T16:52:25.792552+0000","last_unstale":"2024-04-25T16:52:27.830009+0000","last_undegraded":"2024-04-25T16:52:27.830009+0000","last_fullsized":"2024-04-25T16:52:27.830009+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:30.954700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829446+0000","last_change":"2024-04-25T16:52:26.823643+0000","last_active":"2024-04-25T16:52:27.829446+0000","last_peered":"2024-04-25T16:52:27.829446+0000","last_clean":"2024-04-25T16:52:27.829446+0000","last_became_active":"2024-04-25T16:52:26.823348+0000","last_became_peered":"2024-04-25T16:52:26.823348+0000","last_unstale":"2024-04-25T16:52:27.829446+0000","last_undegraded":"2024-04-25T16:52:27.829446+0000","last_fullsized":"2024-04-25T16:52:27.829446+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830926+0000","last_change":"2024-04-25T16:52:25.813706+0000","last_active":"2024-04-25T16:52:27.830926+0000","last_peered":"2024-04-25T16:52:27.830926+0000","last_clean":"2024-04-25T16:52:27.830926+0000","last_became_active":"2024-04-25T16:52:25.813387+0000","last_became_peered":"2024-04-25T16:52:25.813387+0000","last_unstale":"2024-04-25T16:52:27.830926+0000","last_undegraded":"2024-04-25T16:52:27.830926+0000","last_fullsized":"2024-04-25T16:52:27.830926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781928+0000","last_change":"2024-04-25T16:52:26.817332+0000","last_active":"2024-04-25T16:52:27.781928+0000","last_peered":"2024-04-25T16:52:27.781928+0000","last_clean":"2024-04-25T16:52:27.781928+0000","last_became_active":"2024-04-25T16:52:26.816020+0000","last_became_peered":"2024-04-25T16:52:26.816020+0000","last_unstale":"2024-04-25T16:52:27.781928+0000","last_undegraded":"2024-04-25T16:52:27.781928+0000","last_fullsized":"2024-04-25T16:52:27.781928+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.779458+0000","last_change":"2024-04-25T16:52:25.815545+0000","last_active":"2024-04-25T16:52:27.779458+0000","last_peered":"2024-04-25T16:52:27.779458+0000","last_clean":"2024-04-25T16:52:27.779458+0000","last_became_active":"2024-04-25T16:52:25.814711+0000","last_became_peered":"2024-04-25T16:52:25.814711+0000","last_unstale":"2024-04-25T16:52:27.779458+0000","last_undegraded":"2024-04-25T16:52:27.779458+0000","last_fullsized":"2024-04-25T16:52:27.779458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829829+0000","last_change":"2024-04-25T16:52:26.826567+0000","last_active":"2024-04-25T16:52:27.829829+0000","last_peered":"2024-04-25T16:52:27.829829+0000","last_clean":"2024-04-25T16:52:27.829829+0000","last_became_active":"2024-04-25T16:52:26.826224+0000","last_became_peered":"2024-04-25T16:52:26.826224+0000","last_unstale":"2024-04-25T16:52:27.829829+0000","last_undegraded":"2024-04-25T16:52:27.829829+0000","last_fullsized":"2024-04-25T16:52:27.829829+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:06.823841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782735+0000","last_change":"2024-04-25T16:52:25.815153+0000","last_active":"2024-04-25T16:52:27.782735+0000","last_peered":"2024-04-25T16:52:27.782735+0000","last_clean":"2024-04-25T16:52:27.782735+0000","last_became_active":"2024-04-25T16:52:25.814360+0000","last_became_peered":"2024-04-25T16:52:25.814360+0000","last_unstale":"2024-04-25T16:52:27.782735+0000","last_undegraded":"2024-04-25T16:52:27.782735+0000","last_fullsized":"2024-04-25T16:52:27.782735+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830252+0000","last_change":"2024-04-25T16:52:26.804704+0000","last_active":"2024-04-25T16:52:27.830252+0000","last_peered":"2024-04-25T16:52:27.830252+0000","last_clean":"2024-04-25T16:52:27.830252+0000","last_became_active":"2024-04-25T16:52:26.803888+0000","last_became_peered":"2024-04-25T16:52:26.803888+0000","last_unstale":"2024-04-25T16:52:27.830252+0000","last_undegraded":"2024-04-25T16:52:27.830252+0000","last_fullsized":"2024-04-25T16:52:27.830252+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.640204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823002+0000","last_change":"2024-04-25T16:52:25.818880+0000","last_active":"2024-04-25T16:52:27.823002+0000","last_peered":"2024-04-25T16:52:27.823002+0000","last_clean":"2024-04-25T16:52:27.823002+0000","last_became_active":"2024-04-25T16:52:25.817841+0000","last_became_peered":"2024-04-25T16:52:25.817841+0000","last_unstale":"2024-04-25T16:52:27.823002+0000","last_undegraded":"2024-04-25T16:52:27.823002+0000","last_fullsized":"2024-04-25T16:52:27.823002+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:26.817110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823891+0000","last_change":"2024-04-25T16:52:26.809554+0000","last_active":"2024-04-25T16:52:27.823891+0000","last_peered":"2024-04-25T16:52:27.823891+0000","last_clean":"2024-04-25T16:52:27.823891+0000","last_became_active":"2024-04-25T16:52:26.808709+0000","last_became_peered":"2024-04-25T16:52:26.808709+0000","last_unstale":"2024-04-25T16:52:27.823891+0000","last_undegraded":"2024-04-25T16:52:27.823891+0000","last_fullsized":"2024-04-25T16:52:27.823891+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831719+0000","last_change":"2024-04-25T16:52:25.812269+0000","last_active":"2024-04-25T16:52:27.831719+0000","last_peered":"2024-04-25T16:52:27.831719+0000","last_clean":"2024-04-25T16:52:27.831719+0000","last_became_active":"2024-04-25T16:52:25.810831+0000","last_became_peered":"2024-04-25T16:52:25.810831+0000","last_unstale":"2024-04-25T16:52:27.831719+0000","last_undegraded":"2024-04-25T16:52:27.831719+0000","last_fullsized":"2024-04-25T16:52:27.831719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831522+0000","last_change":"2024-04-25T16:52:26.806939+0000","last_active":"2024-04-25T16:52:27.831522+0000","last_peered":"2024-04-25T16:52:27.831522+0000","last_clean":"2024-04-25T16:52:27.831522+0000","last_became_active":"2024-04-25T16:52:26.806053+0000","last_became_peered":"2024-04-25T16:52:26.806053+0000","last_unstale":"2024-04-25T16:52:27.831522+0000","last_undegraded":"2024-04-25T16:52:27.831522+0000","last_fullsized":"2024-04-25T16:52:27.831522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781168+0000","last_change":"2024-04-25T16:52:25.783759+0000","last_active":"2024-04-25T16:52:27.781168+0000","last_peered":"2024-04-25T16:52:27.781168+0000","last_clean":"2024-04-25T16:52:27.781168+0000","last_became_active":"2024-04-25T16:52:25.783353+0000","last_became_peered":"2024-04-25T16:52:25.783353+0000","last_unstale":"2024-04-25T16:52:27.781168+0000","last_undegraded":"2024-04-25T16:52:27.781168+0000","last_fullsized":"2024-04-25T16:52:27.781168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:34.281502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780945+0000","last_change":"2024-04-25T16:52:26.798434+0000","last_active":"2024-04-25T16:52:27.780945+0000","last_peered":"2024-04-25T16:52:27.780945+0000","last_clean":"2024-04-25T16:52:27.780945+0000","last_became_active":"2024-04-25T16:52:26.797631+0000","last_became_peered":"2024-04-25T16:52:26.797631+0000","last_unstale":"2024-04-25T16:52:27.780945+0000","last_undegraded":"2024-04-25T16:52:27.780945+0000","last_fullsized":"2024-04-25T16:52:27.780945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:16.248387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831612+0000","last_change":"2024-04-25T16:52:25.819136+0000","last_active":"2024-04-25T16:52:27.831612+0000","last_peered":"2024-04-25T16:52:27.831612+0000","last_clean":"2024-04-25T16:52:27.831612+0000","last_became_active":"2024-04-25T16:52:25.818809+0000","last_became_peered":"2024-04-25T16:52:25.818809+0000","last_unstale":"2024-04-25T16:52:27.831612+0000","last_undegraded":"2024-04-25T16:52:27.831612+0000","last_fullsized":"2024-04-25T16:52:27.831612+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782782+0000","last_change":"2024-04-25T16:52:26.819612+0000","last_active":"2024-04-25T16:52:27.782782+0000","last_peered":"2024-04-25T16:52:27.782782+0000","last_clean":"2024-04-25T16:52:27.782782+0000","last_became_active":"2024-04-25T16:52:26.818874+0000","last_became_peered":"2024-04-25T16:52:26.818874+0000","last_unstale":"2024-04-25T16:52:27.782782+0000","last_undegraded":"2024-04-25T16:52:27.782782+0000","last_fullsized":"2024-04-25T16:52:27.782782+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781486+0000","last_change":"2024-04-25T16:52:25.780874+0000","last_active":"2024-04-25T16:52:27.781486+0000","last_peered":"2024-04-25T16:52:27.781486+0000","last_clean":"2024-04-25T16:52:27.781486+0000","last_became_active":"2024-04-25T16:52:25.780529+0000","last_became_peered":"2024-04-25T16:52:25.780529+0000","last_unstale":"2024-04-25T16:52:27.781486+0000","last_undegraded":"2024-04-25T16:52:27.781486+0000","last_fullsized":"2024-04-25T16:52:27.781486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:57.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831873+0000","last_change":"2024-04-25T16:52:26.845027+0000","last_active":"2024-04-25T16:52:27.831873+0000","last_peered":"2024-04-25T16:52:27.831873+0000","last_clean":"2024-04-25T16:52:27.831873+0000","last_became_active":"2024-04-25T16:52:26.844620+0000","last_became_peered":"2024-04-25T16:52:26.844620+0000","last_unstale":"2024-04-25T16:52:27.831873+0000","last_undegraded":"2024-04-25T16:52:27.831873+0000","last_fullsized":"2024-04-25T16:52:27.831873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780947+0000","last_change":"2024-04-25T16:52:25.821889+0000","last_active":"2024-04-25T16:52:27.780947+0000","last_peered":"2024-04-25T16:52:27.780947+0000","last_clean":"2024-04-25T16:52:27.780947+0000","last_became_active":"2024-04-25T16:52:25.821643+0000","last_became_peered":"2024-04-25T16:52:25.821643+0000","last_unstale":"2024-04-25T16:52:27.780947+0000","last_undegraded":"2024-04-25T16:52:27.780947+0000","last_fullsized":"2024-04-25T16:52:27.780947+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:27.786669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823650+0000","last_change":"2024-04-25T16:52:26.802864+0000","last_active":"2024-04-25T16:52:27.823650+0000","last_peered":"2024-04-25T16:52:27.823650+0000","last_clean":"2024-04-25T16:52:27.823650+0000","last_became_active":"2024-04-25T16:52:26.801913+0000","last_became_peered":"2024-04-25T16:52:26.801913+0000","last_unstale":"2024-04-25T16:52:27.823650+0000","last_undegraded":"2024-04-25T16:52:27.823650+0000","last_fullsized":"2024-04-25T16:52:27.823650+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782146+0000","last_change":"2024-04-25T16:52:25.806124+0000","last_active":"2024-04-25T16:52:27.782146+0000","last_peered":"2024-04-25T16:52:27.782146+0000","last_clean":"2024-04-25T16:52:27.782146+0000","last_became_active":"2024-04-25T16:52:25.805306+0000","last_became_peered":"2024-04-25T16:52:25.805306+0000","last_unstale":"2024-04-25T16:52:27.782146+0000","last_undegraded":"2024-04-25T16:52:27.782146+0000","last_fullsized":"2024-04-25T16:52:27.782146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829948+0000","last_change":"2024-04-25T16:52:26.819372+0000","last_active":"2024-04-25T16:52:27.829948+0000","last_peered":"2024-04-25T16:52:27.829948+0000","last_clean":"2024-04-25T16:52:27.829948+0000","last_became_active":"2024-04-25T16:52:26.807182+0000","last_became_peered":"2024-04-25T16:52:26.807182+0000","last_unstale":"2024-04-25T16:52:27.829948+0000","last_undegraded":"2024-04-25T16:52:27.829948+0000","last_fullsized":"2024-04-25T16:52:27.829948+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:14.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780505+0000","last_change":"2024-04-25T16:52:25.788959+0000","last_active":"2024-04-25T16:52:27.780505+0000","last_peered":"2024-04-25T16:52:27.780505+0000","last_clean":"2024-04-25T16:52:27.780505+0000","last_became_active":"2024-04-25T16:52:25.788261+0000","last_became_peered":"2024-04-25T16:52:25.788261+0000","last_unstale":"2024-04-25T16:52:27.780505+0000","last_undegraded":"2024-04-25T16:52:27.780505+0000","last_fullsized":"2024-04-25T16:52:27.780505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:13.943131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831765+0000","last_change":"2024-04-25T16:52:26.819641+0000","last_active":"2024-04-25T16:52:27.831765+0000","last_peered":"2024-04-25T16:52:27.831765+0000","last_clean":"2024-04-25T16:52:27.831765+0000","last_became_active":"2024-04-25T16:52:26.819314+0000","last_became_peered":"2024-04-25T16:52:26.819314+0000","last_unstale":"2024-04-25T16:52:27.831765+0000","last_undegraded":"2024-04-25T16:52:27.831765+0000","last_fullsized":"2024-04-25T16:52:27.831765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.823426+0000","last_change":"2024-04-25T16:52:25.806269+0000","last_active":"2024-04-25T16:52:27.823426+0000","last_peered":"2024-04-25T16:52:27.823426+0000","last_clean":"2024-04-25T16:52:27.823426+0000","last_became_active":"2024-04-25T16:52:25.804183+0000","last_became_peered":"2024-04-25T16:52:25.804183+0000","last_unstale":"2024-04-25T16:52:27.823426+0000","last_undegraded":"2024-04-25T16:52:27.823426+0000","last_fullsized":"2024-04-25T16:52:27.823426+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:07.902516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.783237+0000","last_change":"2024-04-25T16:52:26.841753+0000","last_active":"2024-04-25T16:52:27.783237+0000","last_peered":"2024-04-25T16:52:27.783237+0000","last_clean":"2024-04-25T16:52:27.783237+0000","last_became_active":"2024-04-25T16:52:26.841112+0000","last_became_peered":"2024-04-25T16:52:26.841112+0000","last_unstale":"2024-04-25T16:52:27.783237+0000","last_undegraded":"2024-04-25T16:52:27.783237+0000","last_fullsized":"2024-04-25T16:52:27.783237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781212+0000","last_change":"2024-04-25T16:52:25.803168+0000","last_active":"2024-04-25T16:52:27.781212+0000","last_peered":"2024-04-25T16:52:27.781212+0000","last_clean":"2024-04-25T16:52:27.781212+0000","last_became_active":"2024-04-25T16:52:25.802880+0000","last_became_peered":"2024-04-25T16:52:25.802880+0000","last_unstale":"2024-04-25T16:52:27.781212+0000","last_undegraded":"2024-04-25T16:52:27.781212+0000","last_fullsized":"2024-04-25T16:52:27.781212+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.210482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829616+0000","last_change":"2024-04-25T16:52:26.819781+0000","last_active":"2024-04-25T16:52:27.829616+0000","last_peered":"2024-04-25T16:52:27.829616+0000","last_clean":"2024-04-25T16:52:27.829616+0000","last_became_active":"2024-04-25T16:52:26.819467+0000","last_became_peered":"2024-04-25T16:52:26.819467+0000","last_unstale":"2024-04-25T16:52:27.829616+0000","last_undegraded":"2024-04-25T16:52:27.829616+0000","last_fullsized":"2024-04-25T16:52:27.829616+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832139+0000","last_change":"2024-04-25T16:52:25.799953+0000","last_active":"2024-04-25T16:52:27.832139+0000","last_peered":"2024-04-25T16:52:27.832139+0000","last_clean":"2024-04-25T16:52:27.832139+0000","last_became_active":"2024-04-25T16:52:25.799689+0000","last_became_peered":"2024-04-25T16:52:25.799689+0000","last_unstale":"2024-04-25T16:52:27.832139+0000","last_undegraded":"2024-04-25T16:52:27.832139+0000","last_fullsized":"2024-04-25T16:52:27.832139+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831852+0000","last_change":"2024-04-25T16:52:26.799367+0000","last_active":"2024-04-25T16:52:27.831852+0000","last_peered":"2024-04-25T16:52:27.831852+0000","last_clean":"2024-04-25T16:52:27.831852+0000","last_became_active":"2024-04-25T16:52:26.797337+0000","last_became_peered":"2024-04-25T16:52:26.797337+0000","last_unstale":"2024-04-25T16:52:27.831852+0000","last_undegraded":"2024-04-25T16:52:27.831852+0000","last_fullsized":"2024-04-25T16:52:27.831852+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.832932+0000","last_change":"2024-04-25T16:52:25.806956+0000","last_active":"2024-04-25T16:52:27.832932+0000","last_peered":"2024-04-25T16:52:27.832932+0000","last_clean":"2024-04-25T16:52:27.832932+0000","last_became_active":"2024-04-25T16:52:25.806604+0000","last_became_peered":"2024-04-25T16:52:25.806604+0000","last_unstale":"2024-04-25T16:52:27.832932+0000","last_undegraded":"2024-04-25T16:52:27.832932+0000","last_fullsized":"2024-04-25T16:52:27.832932+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831008+0000","last_change":"2024-04-25T16:52:26.820845+0000","last_active":"2024-04-25T16:52:27.831008+0000","last_peered":"2024-04-25T16:52:27.831008+0000","last_clean":"2024-04-25T16:52:27.831008+0000","last_became_active":"2024-04-25T16:52:26.820484+0000","last_became_peered":"2024-04-25T16:52:26.820484+0000","last_unstale":"2024-04-25T16:52:27.831008+0000","last_undegraded":"2024-04-25T16:52:27.831008+0000","last_fullsized":"2024-04-25T16:52:27.831008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:44.359144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.779900+0000","last_change":"2024-04-25T16:52:25.785723+0000","last_active":"2024-04-25T16:52:27.779900+0000","last_peered":"2024-04-25T16:52:27.779900+0000","last_clean":"2024-04-25T16:52:27.779900+0000","last_became_active":"2024-04-25T16:52:25.784930+0000","last_became_peered":"2024-04-25T16:52:25.784930+0000","last_unstale":"2024-04-25T16:52:27.779900+0000","last_undegraded":"2024-04-25T16:52:27.779900+0000","last_fullsized":"2024-04-25T16:52:27.779900+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830715+0000","last_change":"2024-04-25T16:52:26.799277+0000","last_active":"2024-04-25T16:52:27.830715+0000","last_peered":"2024-04-25T16:52:27.830715+0000","last_clean":"2024-04-25T16:52:27.830715+0000","last_became_active":"2024-04-25T16:52:26.797817+0000","last_became_peered":"2024-04-25T16:52:26.797817+0000","last_unstale":"2024-04-25T16:52:27.830715+0000","last_undegraded":"2024-04-25T16:52:27.830715+0000","last_fullsized":"2024-04-25T16:52:27.830715+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830490+0000","last_change":"2024-04-25T16:52:25.810628+0000","last_active":"2024-04-25T16:52:27.830490+0000","last_peered":"2024-04-25T16:52:27.830490+0000","last_clean":"2024-04-25T16:52:27.830490+0000","last_became_active":"2024-04-25T16:52:25.809669+0000","last_became_peered":"2024-04-25T16:52:25.809669+0000","last_unstale":"2024-04-25T16:52:27.830490+0000","last_undegraded":"2024-04-25T16:52:27.830490+0000","last_fullsized":"2024-04-25T16:52:27.830490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:36.638682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782480+0000","last_change":"2024-04-25T16:52:26.832124+0000","last_active":"2024-04-25T16:52:27.782480+0000","last_peered":"2024-04-25T16:52:27.782480+0000","last_clean":"2024-04-25T16:52:27.782480+0000","last_became_active":"2024-04-25T16:52:26.831437+0000","last_became_peered":"2024-04-25T16:52:26.831437+0000","last_unstale":"2024-04-25T16:52:27.782480+0000","last_undegraded":"2024-04-25T16:52:27.782480+0000","last_fullsized":"2024-04-25T16:52:27.782480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.829356+0000","last_change":"2024-04-25T16:52:25.793944+0000","last_active":"2024-04-25T16:52:27.829356+0000","last_peered":"2024-04-25T16:52:27.829356+0000","last_clean":"2024-04-25T16:52:27.829356+0000","last_became_active":"2024-04-25T16:52:25.793522+0000","last_became_peered":"2024-04-25T16:52:25.793522+0000","last_unstale":"2024-04-25T16:52:27.829356+0000","last_undegraded":"2024-04-25T16:52:27.829356+0000","last_fullsized":"2024-04-25T16:52:27.829356+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:41.542439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.822709+0000","last_change":"2024-04-25T16:52:26.804643+0000","last_active":"2024-04-25T16:52:27.822709+0000","last_peered":"2024-04-25T16:52:27.822709+0000","last_clean":"2024-04-25T16:52:27.822709+0000","last_became_active":"2024-04-25T16:52:26.802297+0000","last_became_peered":"2024-04-25T16:52:26.802297+0000","last_unstale":"2024-04-25T16:52:27.822709+0000","last_undegraded":"2024-04-25T16:52:27.822709+0000","last_fullsized":"2024-04-25T16:52:27.822709+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782714+0000","last_change":"2024-04-25T16:52:25.796075+0000","last_active":"2024-04-25T16:52:27.782714+0000","last_peered":"2024-04-25T16:52:27.782714+0000","last_clean":"2024-04-25T16:52:27.782714+0000","last_became_active":"2024-04-25T16:52:25.795226+0000","last_became_peered":"2024-04-25T16:52:25.795226+0000","last_unstale":"2024-04-25T16:52:27.782714+0000","last_undegraded":"2024-04-25T16:52:27.782714+0000","last_fullsized":"2024-04-25T16:52:27.782714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":143,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782982+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:52:27.782982+0000","last_peered":"2024-04-25T16:52:27.782982+0000","last_clean":"2024-04-25T16:52:27.782982+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:52:27.782982+0000","last_undegraded":"2024-04-25T16:52:27.782982+0000","last_fullsized":"2024-04-25T16:52:27.782982+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":190610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":446464,"data_stored":190632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35788,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336052,"statfs":{"total":96636764160,"available":96600117248,"internally_reserved":0,"allocated":454656,"data_stored":191674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":190098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35776,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336064,"statfs":{"total":96636764160,"available":96600129536,"internally_reserved":0,"allocated":442368,"data_stored":190118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335484,"statfs":{"total":96636764160,"available":96599535616,"internally_reserved":0,"allocated":1036288,"data_stored":780430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:52:35.543 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:52:35.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.703+0000 7f39a5525640 1 Processor -- start 2024-04-25T16:52:35.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.703+0000 7f39a5525640 1 -- start start 2024-04-25T16:52:35.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.707+0000 7f39a5525640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39a015e990 0x7f39a0165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:35.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.707+0000 7f39a5525640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f39a0165b90 0x7f39a0167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:35.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.707+0000 7f39a5525640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a006d610 con 0x7f39a0070080 2024-04-25T16:52:35.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.707+0000 7f39a5525640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f39a0158550 con 0x7f39a0165b90 2024-04-25T16:52:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.707+0000 7f39a5525640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f39a006f7d0 con 0x7f39a015e990 2024-04-25T16:52:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.707+0000 7f399effd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f39a0070080 0x7f39a015e280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56842/0 (socket says 172.21.15.94:56842) 2024-04-25T16:52:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.707+0000 7f399f7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f39a0165b90 0x7f39a0167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.707+0000 7f399effd640 1 -- 172.21.15.94:0/689378411 learned_addr learned my addr 172.21.15.94:0/689378411 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.707+0000 7f399e7fc640 1 --2- 172.21.15.94:0/689378411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39a015e990 0x7f39a0165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.707+0000 7f399f7fe640 1 -- 172.21.15.94:0/689378411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39a015e990 msgr2=0x7f39a0165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.707+0000 7f399f7fe640 1 --2- 172.21.15.94:0/689378411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39a015e990 0x7f39a0165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.707+0000 7f399f7fe640 1 -- 172.21.15.94:0/689378411 >> v1:172.21.15.94:6789/0 conn(0x7f39a0070080 legacy=0x7f39a015e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.707+0000 7f399f7fe640 1 -- 172.21.15.94:0/689378411 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a006fc60 con 0x7f39a0165b90 2024-04-25T16:52:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f399f7fe640 1 --2- 172.21.15.94:0/689378411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0165b90 0x7f39a0167fb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f39a01586d0 tx=0x7f399801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee020f27da5d9fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f399dffb640 1 -- 172.21.15.94:0/689378411 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980b2070 con 0x7f39a0165b90 2024-04-25T16:52:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f399dffb640 1 -- 172.21.15.94:0/689378411 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39980a59f0 con 0x7f39a0165b90 2024-04-25T16:52:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f399dffb640 1 -- 172.21.15.94:0/689378411 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980a5dc0 con 0x7f39a0165b90 2024-04-25T16:52:35.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f39a5525640 1 -- 172.21.15.94:0/689378411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0165b90 msgr2=0x7f39a0167fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:35.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f39a5525640 1 --2- 172.21.15.94:0/689378411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0165b90 0x7f39a0167fb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f39a01586d0 tx=0x7f399801c930 comp rx=0 tx=0).stop 2024-04-25T16:52:35.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f39a5525640 1 -- 172.21.15.94:0/689378411 shutdown_connections 2024-04-25T16:52:35.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f39a5525640 1 --2- 172.21.15.94:0/689378411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0165b90 0x7f39a0167fb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:35.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f39a5525640 1 --2- 172.21.15.94:0/689378411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39a015e990 0x7f39a0165650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:35.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f39a5525640 1 -- 172.21.15.94:0/689378411 >> 172.21.15.94:0/689378411 conn(0x7f39a00b47f0 msgr2=0x7f39a00b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:35.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f39a5525640 1 -- 172.21.15.94:0/689378411 shutdown_connections 2024-04-25T16:52:35.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f39a5525640 1 -- 172.21.15.94:0/689378411 wait complete. 2024-04-25T16:52:35.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f39a5525640 1 Processor -- start 2024-04-25T16:52:35.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.711+0000 7f39a5525640 1 -- start start 2024-04-25T16:52:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f39a5525640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a0070080 0x7f39a009aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f39a5525640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39a015e990 0x7f39a009c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f39a5525640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0165b90 0x7f39a01538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:35.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f39a5525640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39a00ae6c0 con 0x7f39a0165b90 2024-04-25T16:52:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f39a5525640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39a006f460 con 0x7f39a0070080 2024-04-25T16:52:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f39a5525640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f39a0070490 con 0x7f39a015e990 2024-04-25T16:52:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f399f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0165b90 0x7f39a01538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f399f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0165b90 0x7f39a01538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59096/0 (socket says 172.21.15.94:59096) 2024-04-25T16:52:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f399f7fe640 1 -- 172.21.15.94:0/3344721118 learned_addr learned my addr 172.21.15.94:0/3344721118 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f399effd640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a0070080 0x7f39a009aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f399e7fc640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39a015e990 0x7f39a009c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f399f7fe640 1 -- 172.21.15.94:0/3344721118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39a015e990 msgr2=0x7f39a009c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f399f7fe640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39a015e990 0x7f39a009c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f399f7fe640 1 -- 172.21.15.94:0/3344721118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a0070080 msgr2=0x7f39a009aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f399f7fe640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a0070080 0x7f39a009aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f399f7fe640 1 -- 172.21.15.94:0/3344721118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a009c880 con 0x7f39a0165b90 2024-04-25T16:52:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f399f7fe640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0165b90 0x7f39a01538a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f39980b0040 tx=0x7f39980196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f39877fe640 1 -- 172.21.15.94:0/3344721118 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980b2050 con 0x7f39a0165b90 2024-04-25T16:52:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f39877fe640 1 -- 172.21.15.94:0/3344721118 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39980be070 con 0x7f39a0165b90 2024-04-25T16:52:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f39877fe640 1 -- 172.21.15.94:0/3344721118 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980adab0 con 0x7f39a0165b90 2024-04-25T16:52:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f39a5525640 1 -- 172.21.15.94:0/3344721118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a0167ca0 con 0x7f39a0165b90 2024-04-25T16:52:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f39a5525640 1 -- 172.21.15.94:0/3344721118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a009d4b0 con 0x7f39a0165b90 2024-04-25T16:52:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f39a5525640 1 -- 172.21.15.94:0/3344721118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a0076830 con 0x7f39a0165b90 2024-04-25T16:52:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.715+0000 7f39877fe640 1 -- 172.21.15.94:0/3344721118 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f39980adc50 con 0x7f39a0165b90 2024-04-25T16:52:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.719+0000 7f39877fe640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f398c06f400 0x7f398c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.719+0000 7f399effd640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f398c06f400 0x7f398c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.719+0000 7f39877fe640 1 -- 172.21.15.94:0/3344721118 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6130+0+0 (secure 0 0 0) 0x7f399812a860 con 0x7f39a0165b90 2024-04-25T16:52:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.719+0000 7f399effd640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f398c06f400 0x7f398c071910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3994000f70 tx=0x7f3994009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.723+0000 7f39877fe640 1 -- 172.21.15.94:0/3344721118 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3998016520 con 0x7f39a0165b90 2024-04-25T16:52:35.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.883+0000 7f39a5525640 1 -- 172.21.15.94:0/3344721118 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f39a00b31e0 con 0x7f398c06f400 2024-04-25T16:52:35.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.899+0000 7f39877fe640 1 -- 172.21.15.94:0/3344721118 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251881 (secure 0 0 0) 0x7f39a00b31e0 con 0x7f398c06f400 2024-04-25T16:52:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:35.903 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:52:35.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.903+0000 7f39a5525640 1 -- 172.21.15.94:0/3344721118 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f398c06f400 msgr2=0x7f398c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:35.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.903+0000 7f39a5525640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f398c06f400 0x7f398c071910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3994000f70 tx=0x7f3994009040 comp rx=0 tx=0).stop 2024-04-25T16:52:35.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.903+0000 7f39a5525640 1 -- 172.21.15.94:0/3344721118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0165b90 msgr2=0x7f39a01538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:35.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.903+0000 7f39a5525640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0165b90 0x7f39a01538a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f39980b0040 tx=0x7f39980196d0 comp rx=0 tx=0).stop 2024-04-25T16:52:35.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.903+0000 7f39a5525640 1 -- 172.21.15.94:0/3344721118 shutdown_connections 2024-04-25T16:52:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.903+0000 7f39a5525640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39a0165b90 0x7f39a01538a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.903+0000 7f39a5525640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f398c06f400 0x7f398c071910 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.903+0000 7f39a5525640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39a015e990 0x7f39a009c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.903+0000 7f39a5525640 1 --2- 172.21.15.94:0/3344721118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a0070080 0x7f39a009aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.903+0000 7f39a5525640 1 -- 172.21.15.94:0/3344721118 >> 172.21.15.94:0/3344721118 conn(0x7f39a00b47f0 msgr2=0x7f39a00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.903+0000 7f39a5525640 1 -- 172.21.15.94:0/3344721118 shutdown_connections 2024-04-25T16:52:35.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:35.903+0000 7f39a5525640 1 -- 172.21.15.94:0/3344721118 wait complete. 2024-04-25T16:52:35.920 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-25T16:52:35.877728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258320,"kb_used_data":4708,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716400,"statfs":{"total":773094113280,"available":772829593600,"internally_reserved":0,"allocated":4820992,"data_stored":2737575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.100997"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873724+0000","last_change":"2024-04-25T16:52:26.825748+0000","last_active":"2024-04-25T16:52:34.873724+0000","last_peered":"2024-04-25T16:52:34.873724+0000","last_clean":"2024-04-25T16:52:34.873724+0000","last_became_active":"2024-04-25T16:52:26.825333+0000","last_became_peered":"2024-04-25T16:52:26.825333+0000","last_unstale":"2024-04-25T16:52:34.873724+0000","last_undegraded":"2024-04-25T16:52:34.873724+0000","last_fullsized":"2024-04-25T16:52:34.873724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:19.084563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873506+0000","last_change":"2024-04-25T16:52:26.795964+0000","last_active":"2024-04-25T16:52:34.873506+0000","last_peered":"2024-04-25T16:52:34.873506+0000","last_clean":"2024-04-25T16:52:34.873506+0000","last_became_active":"2024-04-25T16:52:26.794884+0000","last_became_peered":"2024-04-25T16:52:26.794884+0000","last_unstale":"2024-04-25T16:52:34.873506+0000","last_undegraded":"2024-04-25T16:52:34.873506+0000","last_fullsized":"2024-04-25T16:52:34.873506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878233+0000","last_change":"2024-04-25T16:52:26.802075+0000","last_active":"2024-04-25T16:52:34.878233+0000","last_peered":"2024-04-25T16:52:34.878233+0000","last_clean":"2024-04-25T16:52:34.878233+0000","last_became_active":"2024-04-25T16:52:26.801203+0000","last_became_peered":"2024-04-25T16:52:26.801203+0000","last_unstale":"2024-04-25T16:52:34.878233+0000","last_undegraded":"2024-04-25T16:52:34.878233+0000","last_fullsized":"2024-04-25T16:52:34.878233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782213+0000","last_change":"2024-04-25T16:52:26.793541+0000","last_active":"2024-04-25T16:52:27.782213+0000","last_peered":"2024-04-25T16:52:27.782213+0000","last_clean":"2024-04-25T16:52:27.782213+0000","last_became_active":"2024-04-25T16:52:26.789101+0000","last_became_peered":"2024-04-25T16:52:26.789101+0000","last_unstale":"2024-04-25T16:52:27.782213+0000","last_undegraded":"2024-04-25T16:52:27.782213+0000","last_fullsized":"2024-04-25T16:52:27.782213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:38.013058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781321+0000","last_change":"2024-04-25T16:52:26.784467+0000","last_active":"2024-04-25T16:52:27.781321+0000","last_peered":"2024-04-25T16:52:27.781321+0000","last_clean":"2024-04-25T16:52:27.781321+0000","last_became_active":"2024-04-25T16:52:26.783673+0000","last_became_peered":"2024-04-25T16:52:26.783673+0000","last_unstale":"2024-04-25T16:52:27.781321+0000","last_undegraded":"2024-04-25T16:52:27.781321+0000","last_fullsized":"2024-04-25T16:52:27.781321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:36.583897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831274+0000","last_change":"2024-04-25T16:52:26.823413+0000","last_active":"2024-04-25T16:52:27.831274+0000","last_peered":"2024-04-25T16:52:27.831274+0000","last_clean":"2024-04-25T16:52:27.831274+0000","last_became_active":"2024-04-25T16:52:26.823059+0000","last_became_peered":"2024-04-25T16:52:26.823059+0000","last_unstale":"2024-04-25T16:52:27.831274+0000","last_undegraded":"2024-04-25T16:52:27.831274+0000","last_fullsized":"2024-04-25T16:52:27.831274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.876836+0000","last_change":"2024-04-25T16:52:26.808661+0000","last_active":"2024-04-25T16:52:34.876836+0000","last_peered":"2024-04-25T16:52:34.876836+0000","last_clean":"2024-04-25T16:52:34.876836+0000","last_became_active":"2024-04-25T16:52:26.805296+0000","last_became_peered":"2024-04-25T16:52:26.805296+0000","last_unstale":"2024-04-25T16:52:34.876836+0000","last_undegraded":"2024-04-25T16:52:34.876836+0000","last_fullsized":"2024-04-25T16:52:34.876836+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"peering","last_fresh":"2024-04-25T16:52:30.709501+0000","last_change":"2024-04-25T16:52:34.873590+0000","last_active":"2024-04-25T16:52:34.873590+0000","last_peered":"2024-04-25T16:52:30.709501+0000","last_clean":"2024-04-25T16:52:30.709501+0000","last_became_active":"2024-04-25T16:52:26.820528+0000","last_became_peered":"2024-04-25T16:52:26.820528+0000","last_unstale":"2024-04-25T16:52:30.709501+0000","last_undegraded":"2024-04-25T16:52:30.709501+0000","last_fullsized":"2024-04-25T16:52:30.709501+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.879080+0000","last_change":"2024-04-25T16:52:26.803363+0000","last_active":"2024-04-25T16:52:34.879080+0000","last_peered":"2024-04-25T16:52:34.879080+0000","last_clean":"2024-04-25T16:52:34.879080+0000","last_became_active":"2024-04-25T16:52:26.802539+0000","last_became_peered":"2024-04-25T16:52:26.802539+0000","last_unstale":"2024-04-25T16:52:34.879080+0000","last_undegraded":"2024-04-25T16:52:34.879080+0000","last_fullsized":"2024-04-25T16:52:34.879080+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877890+0000","last_change":"2024-04-25T16:52:26.842728+0000","last_active":"2024-04-25T16:52:34.877890+0000","last_peered":"2024-04-25T16:52:34.877890+0000","last_clean":"2024-04-25T16:52:34.877890+0000","last_became_active":"2024-04-25T16:52:26.842041+0000","last_became_peered":"2024-04-25T16:52:26.842041+0000","last_unstale":"2024-04-25T16:52:34.877890+0000","last_undegraded":"2024-04-25T16:52:34.877890+0000","last_fullsized":"2024-04-25T16:52:34.877890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:33.962706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877607+0000","last_change":"2024-04-25T16:52:26.819329+0000","last_active":"2024-04-25T16:52:34.877607+0000","last_peered":"2024-04-25T16:52:34.877607+0000","last_clean":"2024-04-25T16:52:34.877607+0000","last_became_active":"2024-04-25T16:52:26.818528+0000","last_became_peered":"2024-04-25T16:52:26.818528+0000","last_unstale":"2024-04-25T16:52:34.877607+0000","last_undegraded":"2024-04-25T16:52:34.877607+0000","last_fullsized":"2024-04-25T16:52:34.877607+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878111+0000","last_change":"2024-04-25T16:52:26.795942+0000","last_active":"2024-04-25T16:52:34.878111+0000","last_peered":"2024-04-25T16:52:34.878111+0000","last_clean":"2024-04-25T16:52:34.878111+0000","last_became_active":"2024-04-25T16:52:26.795124+0000","last_became_peered":"2024-04-25T16:52:26.795124+0000","last_unstale":"2024-04-25T16:52:34.878111+0000","last_undegraded":"2024-04-25T16:52:34.878111+0000","last_fullsized":"2024-04-25T16:52:34.878111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877026+0000","last_change":"2024-04-25T16:52:26.814749+0000","last_active":"2024-04-25T16:52:34.877026+0000","last_peered":"2024-04-25T16:52:34.877026+0000","last_clean":"2024-04-25T16:52:34.877026+0000","last_became_active":"2024-04-25T16:52:26.814020+0000","last_became_peered":"2024-04-25T16:52:26.814020+0000","last_unstale":"2024-04-25T16:52:34.877026+0000","last_undegraded":"2024-04-25T16:52:34.877026+0000","last_fullsized":"2024-04-25T16:52:34.877026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871890+0000","last_change":"2024-04-25T16:52:26.813118+0000","last_active":"2024-04-25T16:52:34.871890+0000","last_peered":"2024-04-25T16:52:34.871890+0000","last_clean":"2024-04-25T16:52:34.871890+0000","last_became_active":"2024-04-25T16:52:26.812400+0000","last_became_peered":"2024-04-25T16:52:26.812400+0000","last_unstale":"2024-04-25T16:52:34.871890+0000","last_undegraded":"2024-04-25T16:52:34.871890+0000","last_fullsized":"2024-04-25T16:52:34.871890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:16.169654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877027+0000","last_change":"2024-04-25T16:52:26.795864+0000","last_active":"2024-04-25T16:52:34.877027+0000","last_peered":"2024-04-25T16:52:34.877027+0000","last_clean":"2024-04-25T16:52:34.877027+0000","last_became_active":"2024-04-25T16:52:26.794738+0000","last_became_peered":"2024-04-25T16:52:26.794738+0000","last_unstale":"2024-04-25T16:52:34.877027+0000","last_undegraded":"2024-04-25T16:52:34.877027+0000","last_fullsized":"2024-04-25T16:52:34.877027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340722+0000","last_change":"2024-04-25T16:52:26.836640+0000","last_active":"2024-04-25T16:52:35.340722+0000","last_peered":"2024-04-25T16:52:35.340722+0000","last_clean":"2024-04-25T16:52:35.340722+0000","last_became_active":"2024-04-25T16:52:26.833071+0000","last_became_peered":"2024-04-25T16:52:26.833071+0000","last_unstale":"2024-04-25T16:52:35.340722+0000","last_undegraded":"2024-04-25T16:52:35.340722+0000","last_fullsized":"2024-04-25T16:52:35.340722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341191+0000","last_change":"2024-04-25T16:52:26.841207+0000","last_active":"2024-04-25T16:52:35.341191+0000","last_peered":"2024-04-25T16:52:35.341191+0000","last_clean":"2024-04-25T16:52:35.341191+0000","last_became_active":"2024-04-25T16:52:26.838750+0000","last_became_peered":"2024-04-25T16:52:26.838750+0000","last_unstale":"2024-04-25T16:52:35.341191+0000","last_undegraded":"2024-04-25T16:52:35.341191+0000","last_fullsized":"2024-04-25T16:52:35.341191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875046+0000","last_change":"2024-04-25T16:52:26.795939+0000","last_active":"2024-04-25T16:52:34.875046+0000","last_peered":"2024-04-25T16:52:34.875046+0000","last_clean":"2024-04-25T16:52:34.875046+0000","last_became_active":"2024-04-25T16:52:26.795084+0000","last_became_peered":"2024-04-25T16:52:26.795084+0000","last_unstale":"2024-04-25T16:52:34.875046+0000","last_undegraded":"2024-04-25T16:52:34.875046+0000","last_fullsized":"2024-04-25T16:52:34.875046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781139+0000","last_change":"2024-04-25T16:52:26.818025+0000","last_active":"2024-04-25T16:52:27.781139+0000","last_peered":"2024-04-25T16:52:27.781139+0000","last_clean":"2024-04-25T16:52:27.781139+0000","last_became_active":"2024-04-25T16:52:26.817652+0000","last_became_peered":"2024-04-25T16:52:26.817652+0000","last_unstale":"2024-04-25T16:52:27.781139+0000","last_undegraded":"2024-04-25T16:52:27.781139+0000","last_fullsized":"2024-04-25T16:52:27.781139+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:46.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872811+0000","last_change":"2024-04-25T16:52:26.796384+0000","last_active":"2024-04-25T16:52:34.872811+0000","last_peered":"2024-04-25T16:52:34.872811+0000","last_clean":"2024-04-25T16:52:34.872811+0000","last_became_active":"2024-04-25T16:52:26.795507+0000","last_became_peered":"2024-04-25T16:52:26.795507+0000","last_unstale":"2024-04-25T16:52:34.872811+0000","last_undegraded":"2024-04-25T16:52:34.872811+0000","last_fullsized":"2024-04-25T16:52:34.872811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877104+0000","last_change":"2024-04-25T16:52:26.809988+0000","last_active":"2024-04-25T16:52:34.877104+0000","last_peered":"2024-04-25T16:52:34.877104+0000","last_clean":"2024-04-25T16:52:34.877104+0000","last_became_active":"2024-04-25T16:52:26.809107+0000","last_became_peered":"2024-04-25T16:52:26.809107+0000","last_unstale":"2024-04-25T16:52:34.877104+0000","last_undegraded":"2024-04-25T16:52:34.877104+0000","last_fullsized":"2024-04-25T16:52:34.877104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872865+0000","last_change":"2024-04-25T16:52:26.784466+0000","last_active":"2024-04-25T16:52:34.872865+0000","last_peered":"2024-04-25T16:52:34.872865+0000","last_clean":"2024-04-25T16:52:34.872865+0000","last_became_active":"2024-04-25T16:52:26.783550+0000","last_became_peered":"2024-04-25T16:52:26.783550+0000","last_unstale":"2024-04-25T16:52:34.872865+0000","last_undegraded":"2024-04-25T16:52:34.872865+0000","last_fullsized":"2024-04-25T16:52:34.872865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340668+0000","last_change":"2024-04-25T16:52:26.832452+0000","last_active":"2024-04-25T16:52:35.340668+0000","last_peered":"2024-04-25T16:52:35.340668+0000","last_clean":"2024-04-25T16:52:35.340668+0000","last_became_active":"2024-04-25T16:52:26.831753+0000","last_became_peered":"2024-04-25T16:52:26.831753+0000","last_unstale":"2024-04-25T16:52:35.340668+0000","last_undegraded":"2024-04-25T16:52:35.340668+0000","last_fullsized":"2024-04-25T16:52:35.340668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780752+0000","last_change":"2024-04-25T16:52:26.805633+0000","last_active":"2024-04-25T16:52:27.780752+0000","last_peered":"2024-04-25T16:52:27.780752+0000","last_clean":"2024-04-25T16:52:27.780752+0000","last_became_active":"2024-04-25T16:52:26.804789+0000","last_became_peered":"2024-04-25T16:52:26.804789+0000","last_unstale":"2024-04-25T16:52:27.780752+0000","last_undegraded":"2024-04-25T16:52:27.780752+0000","last_fullsized":"2024-04-25T16:52:27.780752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:08.931465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875174+0000","last_change":"2024-04-25T16:52:26.798693+0000","last_active":"2024-04-25T16:52:34.875174+0000","last_peered":"2024-04-25T16:52:34.875174+0000","last_clean":"2024-04-25T16:52:34.875174+0000","last_became_active":"2024-04-25T16:52:26.797852+0000","last_became_peered":"2024-04-25T16:52:26.797852+0000","last_unstale":"2024-04-25T16:52:34.875174+0000","last_undegraded":"2024-04-25T16:52:34.875174+0000","last_fullsized":"2024-04-25T16:52:34.875174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831376+0000","last_change":"2024-04-25T16:52:26.819630+0000","last_active":"2024-04-25T16:52:27.831376+0000","last_peered":"2024-04-25T16:52:27.831376+0000","last_clean":"2024-04-25T16:52:27.831376+0000","last_became_active":"2024-04-25T16:52:26.819274+0000","last_became_peered":"2024-04-25T16:52:26.819274+0000","last_unstale":"2024-04-25T16:52:27.831376+0000","last_undegraded":"2024-04-25T16:52:27.831376+0000","last_fullsized":"2024-04-25T16:52:27.831376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873718+0000","last_change":"2024-04-25T16:52:26.791433+0000","last_active":"2024-04-25T16:52:34.873718+0000","last_peered":"2024-04-25T16:52:34.873718+0000","last_clean":"2024-04-25T16:52:34.873718+0000","last_became_active":"2024-04-25T16:52:26.790483+0000","last_became_peered":"2024-04-25T16:52:26.790483+0000","last_unstale":"2024-04-25T16:52:34.873718+0000","last_undegraded":"2024-04-25T16:52:34.873718+0000","last_fullsized":"2024-04-25T16:52:34.873718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877762+0000","last_change":"2024-04-25T16:52:26.804860+0000","last_active":"2024-04-25T16:52:34.877762+0000","last_peered":"2024-04-25T16:52:34.877762+0000","last_clean":"2024-04-25T16:52:34.877762+0000","last_became_active":"2024-04-25T16:52:26.804065+0000","last_became_peered":"2024-04-25T16:52:26.804065+0000","last_unstale":"2024-04-25T16:52:34.877762+0000","last_undegraded":"2024-04-25T16:52:34.877762+0000","last_fullsized":"2024-04-25T16:52:34.877762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873009+0000","last_change":"2024-04-25T16:52:26.806705+0000","last_active":"2024-04-25T16:52:34.873009+0000","last_peered":"2024-04-25T16:52:34.873009+0000","last_clean":"2024-04-25T16:52:34.873009+0000","last_became_active":"2024-04-25T16:52:26.805889+0000","last_became_peered":"2024-04-25T16:52:26.805889+0000","last_unstale":"2024-04-25T16:52:34.873009+0000","last_undegraded":"2024-04-25T16:52:34.873009+0000","last_fullsized":"2024-04-25T16:52:34.873009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874258+0000","last_change":"2024-04-25T16:52:26.798903+0000","last_active":"2024-04-25T16:52:34.874258+0000","last_peered":"2024-04-25T16:52:34.874258+0000","last_clean":"2024-04-25T16:52:34.874258+0000","last_became_active":"2024-04-25T16:52:26.798011+0000","last_became_peered":"2024-04-25T16:52:26.798011+0000","last_unstale":"2024-04-25T16:52:34.874258+0000","last_undegraded":"2024-04-25T16:52:34.874258+0000","last_fullsized":"2024-04-25T16:52:34.874258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340837+0000","last_change":"2024-04-25T16:52:26.841037+0000","last_active":"2024-04-25T16:52:35.340837+0000","last_peered":"2024-04-25T16:52:35.340837+0000","last_clean":"2024-04-25T16:52:35.340837+0000","last_became_active":"2024-04-25T16:52:26.838194+0000","last_became_peered":"2024-04-25T16:52:26.838194+0000","last_unstale":"2024-04-25T16:52:35.340837+0000","last_undegraded":"2024-04-25T16:52:35.340837+0000","last_fullsized":"2024-04-25T16:52:35.340837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830883+0000","last_change":"2024-04-25T16:52:26.813243+0000","last_active":"2024-04-25T16:52:27.830883+0000","last_peered":"2024-04-25T16:52:27.830883+0000","last_clean":"2024-04-25T16:52:27.830883+0000","last_became_active":"2024-04-25T16:52:26.811807+0000","last_became_peered":"2024-04-25T16:52:26.811807+0000","last_unstale":"2024-04-25T16:52:27.830883+0000","last_undegraded":"2024-04-25T16:52:27.830883+0000","last_fullsized":"2024-04-25T16:52:27.830883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:42.553053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341094+0000","last_change":"2024-04-25T16:52:26.841017+0000","last_active":"2024-04-25T16:52:35.341094+0000","last_peered":"2024-04-25T16:52:35.341094+0000","last_clean":"2024-04-25T16:52:35.341094+0000","last_became_active":"2024-04-25T16:52:26.837543+0000","last_became_peered":"2024-04-25T16:52:26.837543+0000","last_unstale":"2024-04-25T16:52:35.341094+0000","last_undegraded":"2024-04-25T16:52:35.341094+0000","last_fullsized":"2024-04-25T16:52:35.341094+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875394+0000","last_change":"2024-04-25T16:52:25.818069+0000","last_active":"2024-04-25T16:52:34.875394+0000","last_peered":"2024-04-25T16:52:34.875394+0000","last_clean":"2024-04-25T16:52:34.875394+0000","last_became_active":"2024-04-25T16:52:25.817362+0000","last_became_peered":"2024-04-25T16:52:25.817362+0000","last_unstale":"2024-04-25T16:52:34.875394+0000","last_undegraded":"2024-04-25T16:52:34.875394+0000","last_fullsized":"2024-04-25T16:52:34.875394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:08.488136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833310+0000","last_change":"2024-04-25T16:52:26.829008+0000","last_active":"2024-04-25T16:52:27.833310+0000","last_peered":"2024-04-25T16:52:27.833310+0000","last_clean":"2024-04-25T16:52:27.833310+0000","last_became_active":"2024-04-25T16:52:26.828706+0000","last_became_peered":"2024-04-25T16:52:26.828706+0000","last_unstale":"2024-04-25T16:52:27.833310+0000","last_undegraded":"2024-04-25T16:52:27.833310+0000","last_fullsized":"2024-04-25T16:52:27.833310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878876+0000","last_change":"2024-04-25T16:52:25.795967+0000","last_active":"2024-04-25T16:52:34.878876+0000","last_peered":"2024-04-25T16:52:34.878876+0000","last_clean":"2024-04-25T16:52:34.878876+0000","last_became_active":"2024-04-25T16:52:25.795695+0000","last_became_peered":"2024-04-25T16:52:25.795695+0000","last_unstale":"2024-04-25T16:52:34.878876+0000","last_undegraded":"2024-04-25T16:52:34.878876+0000","last_fullsized":"2024-04-25T16:52:34.878876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875243+0000","last_change":"2024-04-25T16:52:26.789430+0000","last_active":"2024-04-25T16:52:34.875243+0000","last_peered":"2024-04-25T16:52:34.875243+0000","last_clean":"2024-04-25T16:52:34.875243+0000","last_became_active":"2024-04-25T16:52:26.782243+0000","last_became_peered":"2024-04-25T16:52:26.782243+0000","last_unstale":"2024-04-25T16:52:34.875243+0000","last_undegraded":"2024-04-25T16:52:34.875243+0000","last_fullsized":"2024-04-25T16:52:34.875243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833428+0000","last_change":"2024-04-25T16:52:25.819162+0000","last_active":"2024-04-25T16:52:27.833428+0000","last_peered":"2024-04-25T16:52:27.833428+0000","last_clean":"2024-04-25T16:52:27.833428+0000","last_became_active":"2024-04-25T16:52:25.818859+0000","last_became_peered":"2024-04-25T16:52:25.818859+0000","last_unstale":"2024-04-25T16:52:27.833428+0000","last_undegraded":"2024-04-25T16:52:27.833428+0000","last_fullsized":"2024-04-25T16:52:27.833428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874082+0000","last_change":"2024-04-25T16:52:26.805376+0000","last_active":"2024-04-25T16:52:34.874082+0000","last_peered":"2024-04-25T16:52:34.874082+0000","last_clean":"2024-04-25T16:52:34.874082+0000","last_became_active":"2024-04-25T16:52:26.804542+0000","last_became_peered":"2024-04-25T16:52:26.804542+0000","last_unstale":"2024-04-25T16:52:34.874082+0000","last_undegraded":"2024-04-25T16:52:34.874082+0000","last_fullsized":"2024-04-25T16:52:34.874082+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.620224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874004+0000","last_change":"2024-04-25T16:52:25.815751+0000","last_active":"2024-04-25T16:52:34.874004+0000","last_peered":"2024-04-25T16:52:34.874004+0000","last_clean":"2024-04-25T16:52:34.874004+0000","last_became_active":"2024-04-25T16:52:25.814927+0000","last_became_peered":"2024-04-25T16:52:25.814927+0000","last_unstale":"2024-04-25T16:52:34.874004+0000","last_undegraded":"2024-04-25T16:52:34.874004+0000","last_fullsized":"2024-04-25T16:52:34.874004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877461+0000","last_change":"2024-04-25T16:52:26.814640+0000","last_active":"2024-04-25T16:52:34.877461+0000","last_peered":"2024-04-25T16:52:34.877461+0000","last_clean":"2024-04-25T16:52:34.877461+0000","last_became_active":"2024-04-25T16:52:26.814016+0000","last_became_peered":"2024-04-25T16:52:26.814016+0000","last_unstale":"2024-04-25T16:52:34.877461+0000","last_undegraded":"2024-04-25T16:52:34.877461+0000","last_fullsized":"2024-04-25T16:52:34.877461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877431+0000","last_change":"2024-04-25T16:52:25.805323+0000","last_active":"2024-04-25T16:52:34.877431+0000","last_peered":"2024-04-25T16:52:34.877431+0000","last_clean":"2024-04-25T16:52:34.877431+0000","last_became_active":"2024-04-25T16:52:25.796444+0000","last_became_peered":"2024-04-25T16:52:25.796444+0000","last_unstale":"2024-04-25T16:52:34.877431+0000","last_undegraded":"2024-04-25T16:52:34.877431+0000","last_fullsized":"2024-04-25T16:52:34.877431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:28.996351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877181+0000","last_change":"2024-04-25T16:52:26.821648+0000","last_active":"2024-04-25T16:52:34.877181+0000","last_peered":"2024-04-25T16:52:34.877181+0000","last_clean":"2024-04-25T16:52:34.877181+0000","last_became_active":"2024-04-25T16:52:26.820835+0000","last_became_peered":"2024-04-25T16:52:26.820835+0000","last_unstale":"2024-04-25T16:52:34.877181+0000","last_undegraded":"2024-04-25T16:52:34.877181+0000","last_fullsized":"2024-04-25T16:52:34.877181+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340828+0000","last_change":"2024-04-25T16:52:25.816250+0000","last_active":"2024-04-25T16:52:35.340828+0000","last_peered":"2024-04-25T16:52:35.340828+0000","last_clean":"2024-04-25T16:52:35.340828+0000","last_became_active":"2024-04-25T16:52:25.815656+0000","last_became_peered":"2024-04-25T16:52:25.815656+0000","last_unstale":"2024-04-25T16:52:35.340828+0000","last_undegraded":"2024-04-25T16:52:35.340828+0000","last_fullsized":"2024-04-25T16:52:35.340828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872461+0000","last_change":"2024-04-25T16:52:26.813425+0000","last_active":"2024-04-25T16:52:34.872461+0000","last_peered":"2024-04-25T16:52:34.872461+0000","last_clean":"2024-04-25T16:52:34.872461+0000","last_became_active":"2024-04-25T16:52:26.812713+0000","last_became_peered":"2024-04-25T16:52:26.812713+0000","last_unstale":"2024-04-25T16:52:34.872461+0000","last_undegraded":"2024-04-25T16:52:34.872461+0000","last_fullsized":"2024-04-25T16:52:34.872461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831072+0000","last_change":"2024-04-25T16:52:25.803265+0000","last_active":"2024-04-25T16:52:27.831072+0000","last_peered":"2024-04-25T16:52:27.831072+0000","last_clean":"2024-04-25T16:52:27.831072+0000","last_became_active":"2024-04-25T16:52:25.802617+0000","last_became_peered":"2024-04-25T16:52:25.802617+0000","last_unstale":"2024-04-25T16:52:27.831072+0000","last_undegraded":"2024-04-25T16:52:27.831072+0000","last_fullsized":"2024-04-25T16:52:27.831072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872313+0000","last_change":"2024-04-25T16:52:26.819267+0000","last_active":"2024-04-25T16:52:34.872313+0000","last_peered":"2024-04-25T16:52:34.872313+0000","last_clean":"2024-04-25T16:52:34.872313+0000","last_became_active":"2024-04-25T16:52:26.818740+0000","last_became_peered":"2024-04-25T16:52:26.818740+0000","last_unstale":"2024-04-25T16:52:34.872313+0000","last_undegraded":"2024-04-25T16:52:34.872313+0000","last_fullsized":"2024-04-25T16:52:34.872313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871955+0000","last_change":"2024-04-25T16:52:25.816663+0000","last_active":"2024-04-25T16:52:34.871955+0000","last_peered":"2024-04-25T16:52:34.871955+0000","last_clean":"2024-04-25T16:52:34.871955+0000","last_became_active":"2024-04-25T16:52:25.815420+0000","last_became_peered":"2024-04-25T16:52:25.815420+0000","last_unstale":"2024-04-25T16:52:34.871955+0000","last_undegraded":"2024-04-25T16:52:34.871955+0000","last_fullsized":"2024-04-25T16:52:34.871955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:30.288538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782313+0000","last_change":"2024-04-25T16:52:26.789927+0000","last_active":"2024-04-25T16:52:27.782313+0000","last_peered":"2024-04-25T16:52:27.782313+0000","last_clean":"2024-04-25T16:52:27.782313+0000","last_became_active":"2024-04-25T16:52:26.788014+0000","last_became_peered":"2024-04-25T16:52:26.788014+0000","last_unstale":"2024-04-25T16:52:27.782313+0000","last_undegraded":"2024-04-25T16:52:27.782313+0000","last_fullsized":"2024-04-25T16:52:27.782313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:47.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873885+0000","last_change":"2024-04-25T16:52:25.798974+0000","last_active":"2024-04-25T16:52:34.873885+0000","last_peered":"2024-04-25T16:52:34.873885+0000","last_clean":"2024-04-25T16:52:34.873885+0000","last_became_active":"2024-04-25T16:52:25.798314+0000","last_became_peered":"2024-04-25T16:52:25.798314+0000","last_unstale":"2024-04-25T16:52:34.873885+0000","last_undegraded":"2024-04-25T16:52:34.873885+0000","last_fullsized":"2024-04-25T16:52:34.873885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.348530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341450+0000","last_change":"2024-04-25T16:52:26.842485+0000","last_active":"2024-04-25T16:52:35.341450+0000","last_peered":"2024-04-25T16:52:35.341450+0000","last_clean":"2024-04-25T16:52:35.341450+0000","last_became_active":"2024-04-25T16:52:26.838813+0000","last_became_peered":"2024-04-25T16:52:26.838813+0000","last_unstale":"2024-04-25T16:52:35.341450+0000","last_undegraded":"2024-04-25T16:52:35.341450+0000","last_fullsized":"2024-04-25T16:52:35.341450+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878521+0000","last_change":"2024-04-25T16:52:25.803289+0000","last_active":"2024-04-25T16:52:34.878521+0000","last_peered":"2024-04-25T16:52:34.878521+0000","last_clean":"2024-04-25T16:52:34.878521+0000","last_became_active":"2024-04-25T16:52:25.801415+0000","last_became_peered":"2024-04-25T16:52:25.801415+0000","last_unstale":"2024-04-25T16:52:34.878521+0000","last_undegraded":"2024-04-25T16:52:34.878521+0000","last_fullsized":"2024-04-25T16:52:34.878521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:45.310089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878182+0000","last_change":"2024-04-25T16:52:26.840379+0000","last_active":"2024-04-25T16:52:34.878182+0000","last_peered":"2024-04-25T16:52:34.878182+0000","last_clean":"2024-04-25T16:52:34.878182+0000","last_became_active":"2024-04-25T16:52:26.840062+0000","last_became_peered":"2024-04-25T16:52:26.840062+0000","last_unstale":"2024-04-25T16:52:34.878182+0000","last_undegraded":"2024-04-25T16:52:34.878182+0000","last_fullsized":"2024-04-25T16:52:34.878182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878024+0000","last_change":"2024-04-25T16:52:25.809391+0000","last_active":"2024-04-25T16:52:34.878024+0000","last_peered":"2024-04-25T16:52:34.878024+0000","last_clean":"2024-04-25T16:52:34.878024+0000","last_became_active":"2024-04-25T16:52:25.808455+0000","last_became_peered":"2024-04-25T16:52:25.808455+0000","last_unstale":"2024-04-25T16:52:34.878024+0000","last_undegraded":"2024-04-25T16:52:34.878024+0000","last_fullsized":"2024-04-25T16:52:34.878024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872072+0000","last_change":"2024-04-25T16:52:26.782954+0000","last_active":"2024-04-25T16:52:34.872072+0000","last_peered":"2024-04-25T16:52:34.872072+0000","last_clean":"2024-04-25T16:52:34.872072+0000","last_became_active":"2024-04-25T16:52:26.782108+0000","last_became_peered":"2024-04-25T16:52:26.782108+0000","last_unstale":"2024-04-25T16:52:34.872072+0000","last_undegraded":"2024-04-25T16:52:34.872072+0000","last_fullsized":"2024-04-25T16:52:34.872072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873363+0000","last_change":"2024-04-25T16:52:25.795020+0000","last_active":"2024-04-25T16:52:34.873363+0000","last_peered":"2024-04-25T16:52:34.873363+0000","last_clean":"2024-04-25T16:52:34.873363+0000","last_became_active":"2024-04-25T16:52:25.794633+0000","last_became_peered":"2024-04-25T16:52:25.794633+0000","last_unstale":"2024-04-25T16:52:34.873363+0000","last_undegraded":"2024-04-25T16:52:34.873363+0000","last_fullsized":"2024-04-25T16:52:34.873363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:38.694859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877510+0000","last_change":"2024-04-25T16:52:26.821081+0000","last_active":"2024-04-25T16:52:34.877510+0000","last_peered":"2024-04-25T16:52:34.877510+0000","last_clean":"2024-04-25T16:52:34.877510+0000","last_became_active":"2024-04-25T16:52:26.819337+0000","last_became_peered":"2024-04-25T16:52:26.819337+0000","last_unstale":"2024-04-25T16:52:34.877510+0000","last_undegraded":"2024-04-25T16:52:34.877510+0000","last_fullsized":"2024-04-25T16:52:34.877510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872999+0000","last_change":"2024-04-25T16:52:25.793039+0000","last_active":"2024-04-25T16:52:34.872999+0000","last_peered":"2024-04-25T16:52:34.872999+0000","last_clean":"2024-04-25T16:52:34.872999+0000","last_became_active":"2024-04-25T16:52:25.792552+0000","last_became_peered":"2024-04-25T16:52:25.792552+0000","last_unstale":"2024-04-25T16:52:34.872999+0000","last_undegraded":"2024-04-25T16:52:34.872999+0000","last_fullsized":"2024-04-25T16:52:34.872999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:30.954700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872507+0000","last_change":"2024-04-25T16:52:26.823643+0000","last_active":"2024-04-25T16:52:34.872507+0000","last_peered":"2024-04-25T16:52:34.872507+0000","last_clean":"2024-04-25T16:52:34.872507+0000","last_became_active":"2024-04-25T16:52:26.823348+0000","last_became_peered":"2024-04-25T16:52:26.823348+0000","last_unstale":"2024-04-25T16:52:34.872507+0000","last_undegraded":"2024-04-25T16:52:34.872507+0000","last_fullsized":"2024-04-25T16:52:34.872507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830926+0000","last_change":"2024-04-25T16:52:25.813706+0000","last_active":"2024-04-25T16:52:27.830926+0000","last_peered":"2024-04-25T16:52:27.830926+0000","last_clean":"2024-04-25T16:52:27.830926+0000","last_became_active":"2024-04-25T16:52:25.813387+0000","last_became_peered":"2024-04-25T16:52:25.813387+0000","last_unstale":"2024-04-25T16:52:27.830926+0000","last_undegraded":"2024-04-25T16:52:27.830926+0000","last_fullsized":"2024-04-25T16:52:27.830926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340755+0000","last_change":"2024-04-25T16:52:26.817332+0000","last_active":"2024-04-25T16:52:35.340755+0000","last_peered":"2024-04-25T16:52:35.340755+0000","last_clean":"2024-04-25T16:52:35.340755+0000","last_became_active":"2024-04-25T16:52:26.816020+0000","last_became_peered":"2024-04-25T16:52:26.816020+0000","last_unstale":"2024-04-25T16:52:35.340755+0000","last_undegraded":"2024-04-25T16:52:35.340755+0000","last_fullsized":"2024-04-25T16:52:35.340755+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872306+0000","last_change":"2024-04-25T16:52:25.815545+0000","last_active":"2024-04-25T16:52:34.872306+0000","last_peered":"2024-04-25T16:52:34.872306+0000","last_clean":"2024-04-25T16:52:34.872306+0000","last_became_active":"2024-04-25T16:52:25.814711+0000","last_became_peered":"2024-04-25T16:52:25.814711+0000","last_unstale":"2024-04-25T16:52:34.872306+0000","last_undegraded":"2024-04-25T16:52:34.872306+0000","last_fullsized":"2024-04-25T16:52:34.872306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872682+0000","last_change":"2024-04-25T16:52:26.826567+0000","last_active":"2024-04-25T16:52:34.872682+0000","last_peered":"2024-04-25T16:52:34.872682+0000","last_clean":"2024-04-25T16:52:34.872682+0000","last_became_active":"2024-04-25T16:52:26.826224+0000","last_became_peered":"2024-04-25T16:52:26.826224+0000","last_unstale":"2024-04-25T16:52:34.872682+0000","last_undegraded":"2024-04-25T16:52:34.872682+0000","last_fullsized":"2024-04-25T16:52:34.872682+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:06.823841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":24,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340795+0000","last_change":"2024-04-25T16:52:25.815153+0000","last_active":"2024-04-25T16:52:35.340795+0000","last_peered":"2024-04-25T16:52:35.340795+0000","last_clean":"2024-04-25T16:52:35.340795+0000","last_became_active":"2024-04-25T16:52:25.814360+0000","last_became_peered":"2024-04-25T16:52:25.814360+0000","last_unstale":"2024-04-25T16:52:35.340795+0000","last_undegraded":"2024-04-25T16:52:35.340795+0000","last_fullsized":"2024-04-25T16:52:35.340795+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873169+0000","last_change":"2024-04-25T16:52:26.804704+0000","last_active":"2024-04-25T16:52:34.873169+0000","last_peered":"2024-04-25T16:52:34.873169+0000","last_clean":"2024-04-25T16:52:34.873169+0000","last_became_active":"2024-04-25T16:52:26.803888+0000","last_became_peered":"2024-04-25T16:52:26.803888+0000","last_unstale":"2024-04-25T16:52:34.873169+0000","last_undegraded":"2024-04-25T16:52:34.873169+0000","last_fullsized":"2024-04-25T16:52:34.873169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.640204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878107+0000","last_change":"2024-04-25T16:52:25.818880+0000","last_active":"2024-04-25T16:52:34.878107+0000","last_peered":"2024-04-25T16:52:34.878107+0000","last_clean":"2024-04-25T16:52:34.878107+0000","last_became_active":"2024-04-25T16:52:25.817841+0000","last_became_peered":"2024-04-25T16:52:25.817841+0000","last_unstale":"2024-04-25T16:52:34.878107+0000","last_undegraded":"2024-04-25T16:52:34.878107+0000","last_fullsized":"2024-04-25T16:52:34.878107+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:26.817110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877704+0000","last_change":"2024-04-25T16:52:26.809554+0000","last_active":"2024-04-25T16:52:34.877704+0000","last_peered":"2024-04-25T16:52:34.877704+0000","last_clean":"2024-04-25T16:52:34.877704+0000","last_became_active":"2024-04-25T16:52:26.808709+0000","last_became_peered":"2024-04-25T16:52:26.808709+0000","last_unstale":"2024-04-25T16:52:34.877704+0000","last_undegraded":"2024-04-25T16:52:34.877704+0000","last_fullsized":"2024-04-25T16:52:34.877704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875584+0000","last_change":"2024-04-25T16:52:25.812269+0000","last_active":"2024-04-25T16:52:34.875584+0000","last_peered":"2024-04-25T16:52:34.875584+0000","last_clean":"2024-04-25T16:52:34.875584+0000","last_became_active":"2024-04-25T16:52:25.810831+0000","last_became_peered":"2024-04-25T16:52:25.810831+0000","last_unstale":"2024-04-25T16:52:34.875584+0000","last_undegraded":"2024-04-25T16:52:34.875584+0000","last_fullsized":"2024-04-25T16:52:34.875584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874980+0000","last_change":"2024-04-25T16:52:26.806939+0000","last_active":"2024-04-25T16:52:34.874980+0000","last_peered":"2024-04-25T16:52:34.874980+0000","last_clean":"2024-04-25T16:52:34.874980+0000","last_became_active":"2024-04-25T16:52:26.806053+0000","last_became_peered":"2024-04-25T16:52:26.806053+0000","last_unstale":"2024-04-25T16:52:34.874980+0000","last_undegraded":"2024-04-25T16:52:34.874980+0000","last_fullsized":"2024-04-25T16:52:34.874980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781168+0000","last_change":"2024-04-25T16:52:25.783759+0000","last_active":"2024-04-25T16:52:27.781168+0000","last_peered":"2024-04-25T16:52:27.781168+0000","last_clean":"2024-04-25T16:52:27.781168+0000","last_became_active":"2024-04-25T16:52:25.783353+0000","last_became_peered":"2024-04-25T16:52:25.783353+0000","last_unstale":"2024-04-25T16:52:27.781168+0000","last_undegraded":"2024-04-25T16:52:27.781168+0000","last_fullsized":"2024-04-25T16:52:27.781168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:34.281502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780945+0000","last_change":"2024-04-25T16:52:26.798434+0000","last_active":"2024-04-25T16:52:27.780945+0000","last_peered":"2024-04-25T16:52:27.780945+0000","last_clean":"2024-04-25T16:52:27.780945+0000","last_became_active":"2024-04-25T16:52:26.797631+0000","last_became_peered":"2024-04-25T16:52:26.797631+0000","last_unstale":"2024-04-25T16:52:27.780945+0000","last_undegraded":"2024-04-25T16:52:27.780945+0000","last_fullsized":"2024-04-25T16:52:27.780945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:16.248387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831612+0000","last_change":"2024-04-25T16:52:25.819136+0000","last_active":"2024-04-25T16:52:27.831612+0000","last_peered":"2024-04-25T16:52:27.831612+0000","last_clean":"2024-04-25T16:52:27.831612+0000","last_became_active":"2024-04-25T16:52:25.818809+0000","last_became_peered":"2024-04-25T16:52:25.818809+0000","last_unstale":"2024-04-25T16:52:27.831612+0000","last_undegraded":"2024-04-25T16:52:27.831612+0000","last_fullsized":"2024-04-25T16:52:27.831612+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341292+0000","last_change":"2024-04-25T16:52:26.819612+0000","last_active":"2024-04-25T16:52:35.341292+0000","last_peered":"2024-04-25T16:52:35.341292+0000","last_clean":"2024-04-25T16:52:35.341292+0000","last_became_active":"2024-04-25T16:52:26.818874+0000","last_became_peered":"2024-04-25T16:52:26.818874+0000","last_unstale":"2024-04-25T16:52:35.341292+0000","last_undegraded":"2024-04-25T16:52:35.341292+0000","last_fullsized":"2024-04-25T16:52:35.341292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781486+0000","last_change":"2024-04-25T16:52:25.780874+0000","last_active":"2024-04-25T16:52:27.781486+0000","last_peered":"2024-04-25T16:52:27.781486+0000","last_clean":"2024-04-25T16:52:27.781486+0000","last_became_active":"2024-04-25T16:52:25.780529+0000","last_became_peered":"2024-04-25T16:52:25.780529+0000","last_unstale":"2024-04-25T16:52:27.781486+0000","last_undegraded":"2024-04-25T16:52:27.781486+0000","last_fullsized":"2024-04-25T16:52:27.781486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:57.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878077+0000","last_change":"2024-04-25T16:52:26.845027+0000","last_active":"2024-04-25T16:52:34.878077+0000","last_peered":"2024-04-25T16:52:34.878077+0000","last_clean":"2024-04-25T16:52:34.878077+0000","last_became_active":"2024-04-25T16:52:26.844620+0000","last_became_peered":"2024-04-25T16:52:26.844620+0000","last_unstale":"2024-04-25T16:52:34.878077+0000","last_undegraded":"2024-04-25T16:52:34.878077+0000","last_fullsized":"2024-04-25T16:52:34.878077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780947+0000","last_change":"2024-04-25T16:52:25.821889+0000","last_active":"2024-04-25T16:52:27.780947+0000","last_peered":"2024-04-25T16:52:27.780947+0000","last_clean":"2024-04-25T16:52:27.780947+0000","last_became_active":"2024-04-25T16:52:25.821643+0000","last_became_peered":"2024-04-25T16:52:25.821643+0000","last_unstale":"2024-04-25T16:52:27.780947+0000","last_undegraded":"2024-04-25T16:52:27.780947+0000","last_fullsized":"2024-04-25T16:52:27.780947+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:27.786669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877414+0000","last_change":"2024-04-25T16:52:26.802864+0000","last_active":"2024-04-25T16:52:34.877414+0000","last_peered":"2024-04-25T16:52:34.877414+0000","last_clean":"2024-04-25T16:52:34.877414+0000","last_became_active":"2024-04-25T16:52:26.801913+0000","last_became_peered":"2024-04-25T16:52:26.801913+0000","last_unstale":"2024-04-25T16:52:34.877414+0000","last_undegraded":"2024-04-25T16:52:34.877414+0000","last_fullsized":"2024-04-25T16:52:34.877414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341042+0000","last_change":"2024-04-25T16:52:25.806124+0000","last_active":"2024-04-25T16:52:35.341042+0000","last_peered":"2024-04-25T16:52:35.341042+0000","last_clean":"2024-04-25T16:52:35.341042+0000","last_became_active":"2024-04-25T16:52:25.805306+0000","last_became_peered":"2024-04-25T16:52:25.805306+0000","last_unstale":"2024-04-25T16:52:35.341042+0000","last_undegraded":"2024-04-25T16:52:35.341042+0000","last_fullsized":"2024-04-25T16:52:35.341042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872688+0000","last_change":"2024-04-25T16:52:26.819372+0000","last_active":"2024-04-25T16:52:34.872688+0000","last_peered":"2024-04-25T16:52:34.872688+0000","last_clean":"2024-04-25T16:52:34.872688+0000","last_became_active":"2024-04-25T16:52:26.807182+0000","last_became_peered":"2024-04-25T16:52:26.807182+0000","last_unstale":"2024-04-25T16:52:34.872688+0000","last_undegraded":"2024-04-25T16:52:34.872688+0000","last_fullsized":"2024-04-25T16:52:34.872688+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:14.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780505+0000","last_change":"2024-04-25T16:52:25.788959+0000","last_active":"2024-04-25T16:52:27.780505+0000","last_peered":"2024-04-25T16:52:27.780505+0000","last_clean":"2024-04-25T16:52:27.780505+0000","last_became_active":"2024-04-25T16:52:25.788261+0000","last_became_peered":"2024-04-25T16:52:25.788261+0000","last_unstale":"2024-04-25T16:52:27.780505+0000","last_undegraded":"2024-04-25T16:52:27.780505+0000","last_fullsized":"2024-04-25T16:52:27.780505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:13.943131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831765+0000","last_change":"2024-04-25T16:52:26.819641+0000","last_active":"2024-04-25T16:52:27.831765+0000","last_peered":"2024-04-25T16:52:27.831765+0000","last_clean":"2024-04-25T16:52:27.831765+0000","last_became_active":"2024-04-25T16:52:26.819314+0000","last_became_peered":"2024-04-25T16:52:26.819314+0000","last_unstale":"2024-04-25T16:52:27.831765+0000","last_undegraded":"2024-04-25T16:52:27.831765+0000","last_fullsized":"2024-04-25T16:52:27.831765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878486+0000","last_change":"2024-04-25T16:52:25.806269+0000","last_active":"2024-04-25T16:52:34.878486+0000","last_peered":"2024-04-25T16:52:34.878486+0000","last_clean":"2024-04-25T16:52:34.878486+0000","last_became_active":"2024-04-25T16:52:25.804183+0000","last_became_peered":"2024-04-25T16:52:25.804183+0000","last_unstale":"2024-04-25T16:52:34.878486+0000","last_undegraded":"2024-04-25T16:52:34.878486+0000","last_fullsized":"2024-04-25T16:52:34.878486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:07.902516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341557+0000","last_change":"2024-04-25T16:52:26.841753+0000","last_active":"2024-04-25T16:52:35.341557+0000","last_peered":"2024-04-25T16:52:35.341557+0000","last_clean":"2024-04-25T16:52:35.341557+0000","last_became_active":"2024-04-25T16:52:26.841112+0000","last_became_peered":"2024-04-25T16:52:26.841112+0000","last_unstale":"2024-04-25T16:52:35.341557+0000","last_undegraded":"2024-04-25T16:52:35.341557+0000","last_fullsized":"2024-04-25T16:52:35.341557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781212+0000","last_change":"2024-04-25T16:52:25.803168+0000","last_active":"2024-04-25T16:52:27.781212+0000","last_peered":"2024-04-25T16:52:27.781212+0000","last_clean":"2024-04-25T16:52:27.781212+0000","last_became_active":"2024-04-25T16:52:25.802880+0000","last_became_peered":"2024-04-25T16:52:25.802880+0000","last_unstale":"2024-04-25T16:52:27.781212+0000","last_undegraded":"2024-04-25T16:52:27.781212+0000","last_fullsized":"2024-04-25T16:52:27.781212+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.210482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873797+0000","last_change":"2024-04-25T16:52:26.819781+0000","last_active":"2024-04-25T16:52:34.873797+0000","last_peered":"2024-04-25T16:52:34.873797+0000","last_clean":"2024-04-25T16:52:34.873797+0000","last_became_active":"2024-04-25T16:52:26.819467+0000","last_became_peered":"2024-04-25T16:52:26.819467+0000","last_unstale":"2024-04-25T16:52:34.873797+0000","last_undegraded":"2024-04-25T16:52:34.873797+0000","last_fullsized":"2024-04-25T16:52:34.873797+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878534+0000","last_change":"2024-04-25T16:52:25.799953+0000","last_active":"2024-04-25T16:52:34.878534+0000","last_peered":"2024-04-25T16:52:34.878534+0000","last_clean":"2024-04-25T16:52:34.878534+0000","last_became_active":"2024-04-25T16:52:25.799689+0000","last_became_peered":"2024-04-25T16:52:25.799689+0000","last_unstale":"2024-04-25T16:52:34.878534+0000","last_undegraded":"2024-04-25T16:52:34.878534+0000","last_fullsized":"2024-04-25T16:52:34.878534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877833+0000","last_change":"2024-04-25T16:52:26.799367+0000","last_active":"2024-04-25T16:52:34.877833+0000","last_peered":"2024-04-25T16:52:34.877833+0000","last_clean":"2024-04-25T16:52:34.877833+0000","last_became_active":"2024-04-25T16:52:26.797337+0000","last_became_peered":"2024-04-25T16:52:26.797337+0000","last_unstale":"2024-04-25T16:52:34.877833+0000","last_undegraded":"2024-04-25T16:52:34.877833+0000","last_fullsized":"2024-04-25T16:52:34.877833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878430+0000","last_change":"2024-04-25T16:52:25.806956+0000","last_active":"2024-04-25T16:52:34.878430+0000","last_peered":"2024-04-25T16:52:34.878430+0000","last_clean":"2024-04-25T16:52:34.878430+0000","last_became_active":"2024-04-25T16:52:25.806604+0000","last_became_peered":"2024-04-25T16:52:25.806604+0000","last_unstale":"2024-04-25T16:52:34.878430+0000","last_undegraded":"2024-04-25T16:52:34.878430+0000","last_fullsized":"2024-04-25T16:52:34.878430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831008+0000","last_change":"2024-04-25T16:52:26.820845+0000","last_active":"2024-04-25T16:52:27.831008+0000","last_peered":"2024-04-25T16:52:27.831008+0000","last_clean":"2024-04-25T16:52:27.831008+0000","last_became_active":"2024-04-25T16:52:26.820484+0000","last_became_peered":"2024-04-25T16:52:26.820484+0000","last_unstale":"2024-04-25T16:52:27.831008+0000","last_undegraded":"2024-04-25T16:52:27.831008+0000","last_fullsized":"2024-04-25T16:52:27.831008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:44.359144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874630+0000","last_change":"2024-04-25T16:52:25.785723+0000","last_active":"2024-04-25T16:52:34.874630+0000","last_peered":"2024-04-25T16:52:34.874630+0000","last_clean":"2024-04-25T16:52:34.874630+0000","last_became_active":"2024-04-25T16:52:25.784930+0000","last_became_peered":"2024-04-25T16:52:25.784930+0000","last_unstale":"2024-04-25T16:52:34.874630+0000","last_undegraded":"2024-04-25T16:52:34.874630+0000","last_fullsized":"2024-04-25T16:52:34.874630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872198+0000","last_change":"2024-04-25T16:52:26.799277+0000","last_active":"2024-04-25T16:52:34.872198+0000","last_peered":"2024-04-25T16:52:34.872198+0000","last_clean":"2024-04-25T16:52:34.872198+0000","last_became_active":"2024-04-25T16:52:26.797817+0000","last_became_peered":"2024-04-25T16:52:26.797817+0000","last_unstale":"2024-04-25T16:52:34.872198+0000","last_undegraded":"2024-04-25T16:52:34.872198+0000","last_fullsized":"2024-04-25T16:52:34.872198+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871616+0000","last_change":"2024-04-25T16:52:25.810628+0000","last_active":"2024-04-25T16:52:34.871616+0000","last_peered":"2024-04-25T16:52:34.871616+0000","last_clean":"2024-04-25T16:52:34.871616+0000","last_became_active":"2024-04-25T16:52:25.809669+0000","last_became_peered":"2024-04-25T16:52:25.809669+0000","last_unstale":"2024-04-25T16:52:34.871616+0000","last_undegraded":"2024-04-25T16:52:34.871616+0000","last_fullsized":"2024-04-25T16:52:34.871616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:36.638682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341549+0000","last_change":"2024-04-25T16:52:26.832124+0000","last_active":"2024-04-25T16:52:35.341549+0000","last_peered":"2024-04-25T16:52:35.341549+0000","last_clean":"2024-04-25T16:52:35.341549+0000","last_became_active":"2024-04-25T16:52:26.831437+0000","last_became_peered":"2024-04-25T16:52:26.831437+0000","last_unstale":"2024-04-25T16:52:35.341549+0000","last_undegraded":"2024-04-25T16:52:35.341549+0000","last_fullsized":"2024-04-25T16:52:35.341549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872502+0000","last_change":"2024-04-25T16:52:25.793944+0000","last_active":"2024-04-25T16:52:34.872502+0000","last_peered":"2024-04-25T16:52:34.872502+0000","last_clean":"2024-04-25T16:52:34.872502+0000","last_became_active":"2024-04-25T16:52:25.793522+0000","last_became_peered":"2024-04-25T16:52:25.793522+0000","last_unstale":"2024-04-25T16:52:34.872502+0000","last_undegraded":"2024-04-25T16:52:34.872502+0000","last_fullsized":"2024-04-25T16:52:34.872502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:41.542439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877722+0000","last_change":"2024-04-25T16:52:26.804643+0000","last_active":"2024-04-25T16:52:34.877722+0000","last_peered":"2024-04-25T16:52:34.877722+0000","last_clean":"2024-04-25T16:52:34.877722+0000","last_became_active":"2024-04-25T16:52:26.802297+0000","last_became_peered":"2024-04-25T16:52:26.802297+0000","last_unstale":"2024-04-25T16:52:34.877722+0000","last_undegraded":"2024-04-25T16:52:34.877722+0000","last_fullsized":"2024-04-25T16:52:34.877722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341255+0000","last_change":"2024-04-25T16:52:25.796075+0000","last_active":"2024-04-25T16:52:35.341255+0000","last_peered":"2024-04-25T16:52:35.341255+0000","last_clean":"2024-04-25T16:52:35.341255+0000","last_became_active":"2024-04-25T16:52:25.795226+0000","last_became_peered":"2024-04-25T16:52:25.795226+0000","last_unstale":"2024-04-25T16:52:35.341255+0000","last_undegraded":"2024-04-25T16:52:35.341255+0000","last_fullsized":"2024-04-25T16:52:35.341255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":145,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340402+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:52:35.340402+0000","last_peered":"2024-04-25T16:52:35.340402+0000","last_clean":"2024-04-25T16:52:35.340402+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:52:35.340402+0000","last_undegraded":"2024-04-25T16:52:35.340402+0000","last_fullsized":"2024-04-25T16:52:35.340402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":196969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":196499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":458752,"data_stored":196991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35796,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336044,"statfs":{"total":96636764160,"available":96600109056,"internally_reserved":0,"allocated":466944,"data_stored":198033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":190098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35776,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336064,"statfs":{"total":96636764160,"available":96600129536,"internally_reserved":0,"allocated":442368,"data_stored":190118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335476,"statfs":{"total":96636764160,"available":96599527424,"internally_reserved":0,"allocated":1048576,"data_stored":786789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:52:35.924 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:52:36.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342e98a640 1 Processor -- start 2024-04-25T16:52:36.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342e98a640 1 -- start start 2024-04-25T16:52:36.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342e98a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3428072d80 0x7f342815fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342e98a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f34280cf350 0x7f3428160010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342e98a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34280d3230 con 0x7f342806e960 2024-04-25T16:52:36.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342e98a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f34280d2990 con 0x7f34280cf350 2024-04-25T16:52:36.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342e98a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f34280d2c70 con 0x7f3428072d80 2024-04-25T16:52:36.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342cf27640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f34280cf350 0x7f3428160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342cf27640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f34280cf350 0x7f3428160010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59100/0 (socket says 172.21.15.94:59100) 2024-04-25T16:52:36.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342cf27640 1 -- 172.21.15.94:0/3160287739 learned_addr learned my addr 172.21.15.94:0/3160287739 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:36.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f34277fe640 1 --2- 172.21.15.94:0/3160287739 >> v2:172.21.15.138:3300/0 conn(0x7f3428072d80 0x7f342815fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342cf27640 1 -- 172.21.15.94:0/3160287739 >> v2:172.21.15.138:3300/0 conn(0x7f3428072d80 msgr2=0x7f342815fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:36.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342cf27640 1 --2- 172.21.15.94:0/3160287739 >> v2:172.21.15.138:3300/0 conn(0x7f3428072d80 0x7f342815fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342cf27640 1 -- 172.21.15.94:0/3160287739 >> v1:172.21.15.138:6789/0 conn(0x7f342806e960 legacy=0x7f3428072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:36.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.115+0000 7f342cf27640 1 -- 172.21.15.94:0/3160287739 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34280d2f50 con 0x7f34280cf350 2024-04-25T16:52:36.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.119+0000 7f342cf27640 1 --2- 172.21.15.94:0/3160287739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34280cf350 0x7f3428160010 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3418009870 tx=0x7f341801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9d5f4d174294ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:36.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.119+0000 7f3426ffd640 1 -- 172.21.15.94:0/3160287739 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34180b2070 con 0x7f34280cf350 2024-04-25T16:52:36.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.119+0000 7f3426ffd640 1 -- 172.21.15.94:0/3160287739 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34180a9070 con 0x7f34280cf350 2024-04-25T16:52:36.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.119+0000 7f3426ffd640 1 -- 172.21.15.94:0/3160287739 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34180ad880 con 0x7f34280cf350 2024-04-25T16:52:36.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.119+0000 7f342e98a640 1 -- 172.21.15.94:0/3160287739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34280cf350 msgr2=0x7f3428160010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.119+0000 7f342e98a640 1 --2- 172.21.15.94:0/3160287739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34280cf350 0x7f3428160010 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3418009870 tx=0x7f341801c450 comp rx=0 tx=0).stop 2024-04-25T16:52:36.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.119+0000 7f342e98a640 1 -- 172.21.15.94:0/3160287739 shutdown_connections 2024-04-25T16:52:36.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.119+0000 7f342e98a640 1 --2- 172.21.15.94:0/3160287739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34280cf350 0x7f3428160010 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.119+0000 7f342e98a640 1 --2- 172.21.15.94:0/3160287739 >> v2:172.21.15.138:3300/0 conn(0x7f3428072d80 0x7f342815fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.119+0000 7f342e98a640 1 -- 172.21.15.94:0/3160287739 >> 172.21.15.94:0/3160287739 conn(0x7f34280d3850 msgr2=0x7f34280d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:36.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.119+0000 7f342e98a640 1 -- 172.21.15.94:0/3160287739 shutdown_connections 2024-04-25T16:52:36.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.119+0000 7f342e98a640 1 -- 172.21.15.94:0/3160287739 wait complete. 2024-04-25T16:52:36.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f342e98a640 1 Processor -- start 2024-04-25T16:52:36.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f342e98a640 1 -- start start 2024-04-25T16:52:36.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f342e98a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f342806e960 0x7f3428158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f342e98a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34280cf350 0x7f34280b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f342e98a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34280b2e70 0x7f3428161d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f342e98a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f342816c7d0 con 0x7f342806e960 2024-04-25T16:52:36.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f342e98a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3428160f70 con 0x7f34280cf350 2024-04-25T16:52:36.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f342e98a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f342816c950 con 0x7f34280b2e70 2024-04-25T16:52:36.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f3427fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f342806e960 0x7f3428158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f3427fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f342806e960 0x7f3428158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59112/0 (socket says 172.21.15.94:59112) 2024-04-25T16:52:36.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f3427fff640 1 -- 172.21.15.94:0/937237154 learned_addr learned my addr 172.21.15.94:0/937237154 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:36.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f34277fe640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34280cf350 0x7f34280b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f342cf27640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34280b2e70 0x7f3428161d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f3427fff640 1 -- 172.21.15.94:0/937237154 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34280b2e70 msgr2=0x7f3428161d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f3427fff640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34280b2e70 0x7f3428161d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f3427fff640 1 -- 172.21.15.94:0/937237154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34280cf350 msgr2=0x7f34280b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f3427fff640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34280cf350 0x7f34280b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f3427fff640 1 -- 172.21.15.94:0/937237154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34280b02f0 con 0x7f342806e960 2024-04-25T16:52:36.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f34277fe640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34280cf350 0x7f34280b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:36.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f3427fff640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f342806e960 0x7f3428158670 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f342000b3e0 tx=0x7f342000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:36.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f3424ff9640 1 -- 172.21.15.94:0/937237154 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420015020 con 0x7f342806e960 2024-04-25T16:52:36.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f3424ff9640 1 -- 172.21.15.94:0/937237154 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3420011830 con 0x7f342806e960 2024-04-25T16:52:36.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f342e98a640 1 -- 172.21.15.94:0/937237154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34280b05d0 con 0x7f342806e960 2024-04-25T16:52:36.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f3424ff9640 1 -- 172.21.15.94:0/937237154 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420011c60 con 0x7f342806e960 2024-04-25T16:52:36.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.123+0000 7f342e98a640 1 -- 172.21.15.94:0/937237154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34280b0b90 con 0x7f342806e960 2024-04-25T16:52:36.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.127+0000 7f342e98a640 1 -- 172.21.15.94:0/937237154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342806a730 con 0x7f342806e960 2024-04-25T16:52:36.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.127+0000 7f3424ff9640 1 -- 172.21.15.94:0/937237154 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3420018020 con 0x7f342806e960 2024-04-25T16:52:36.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.127+0000 7f3424ff9640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f340806f400 0x7f3408071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.127+0000 7f3424ff9640 1 -- 172.21.15.94:0/937237154 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f342008f490 con 0x7f342806e960 2024-04-25T16:52:36.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.127+0000 7f34277fe640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f340806f400 0x7f3408071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.127+0000 7f34277fe640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f340806f400 0x7f3408071910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f34280cc470 tx=0x7f341000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:36.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.131+0000 7f3424ff9640 1 -- 172.21.15.94:0/937237154 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f342001f030 con 0x7f342806e960 2024-04-25T16:52:36.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.291+0000 7f342e98a640 1 -- 172.21.15.94:0/937237154 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f342815baf0 con 0x7f340806f400 2024-04-25T16:52:36.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.299+0000 7f3424ff9640 1 -- 172.21.15.94:0/937237154 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251881 (secure 0 0 0) 0x7f3410058800 con 0x7f340806f400 2024-04-25T16:52:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:36.301 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:52:36.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.303+0000 7f342e98a640 1 -- 172.21.15.94:0/937237154 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f340806f400 msgr2=0x7f3408071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.303+0000 7f342e98a640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f340806f400 0x7f3408071910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f34280cc470 tx=0x7f341000b040 comp rx=0 tx=0).stop 2024-04-25T16:52:36.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.303+0000 7f342e98a640 1 -- 172.21.15.94:0/937237154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f342806e960 msgr2=0x7f3428158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.303+0000 7f342e98a640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f342806e960 0x7f3428158670 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f342000b3e0 tx=0x7f342000b8a0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.303+0000 7f342e98a640 1 -- 172.21.15.94:0/937237154 shutdown_connections 2024-04-25T16:52:36.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.303+0000 7f342e98a640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f340806f400 0x7f3408071910 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.303+0000 7f342e98a640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34280b2e70 0x7f3428161d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.303+0000 7f342e98a640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34280cf350 0x7f34280b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.303+0000 7f342e98a640 1 --2- 172.21.15.94:0/937237154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f342806e960 0x7f3428158670 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.303+0000 7f342e98a640 1 -- 172.21.15.94:0/937237154 >> 172.21.15.94:0/937237154 conn(0x7f34280d3850 msgr2=0x7f342815e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:36.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.303+0000 7f342e98a640 1 -- 172.21.15.94:0/937237154 shutdown_connections 2024-04-25T16:52:36.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.303+0000 7f342e98a640 1 -- 172.21.15.94:0/937237154 wait complete. 2024-04-25T16:52:36.322 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-25T16:52:35.877728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258320,"kb_used_data":4708,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716400,"statfs":{"total":773094113280,"available":772829593600,"internally_reserved":0,"allocated":4820992,"data_stored":2737575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.100997"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873724+0000","last_change":"2024-04-25T16:52:26.825748+0000","last_active":"2024-04-25T16:52:34.873724+0000","last_peered":"2024-04-25T16:52:34.873724+0000","last_clean":"2024-04-25T16:52:34.873724+0000","last_became_active":"2024-04-25T16:52:26.825333+0000","last_became_peered":"2024-04-25T16:52:26.825333+0000","last_unstale":"2024-04-25T16:52:34.873724+0000","last_undegraded":"2024-04-25T16:52:34.873724+0000","last_fullsized":"2024-04-25T16:52:34.873724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:19.084563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873506+0000","last_change":"2024-04-25T16:52:26.795964+0000","last_active":"2024-04-25T16:52:34.873506+0000","last_peered":"2024-04-25T16:52:34.873506+0000","last_clean":"2024-04-25T16:52:34.873506+0000","last_became_active":"2024-04-25T16:52:26.794884+0000","last_became_peered":"2024-04-25T16:52:26.794884+0000","last_unstale":"2024-04-25T16:52:34.873506+0000","last_undegraded":"2024-04-25T16:52:34.873506+0000","last_fullsized":"2024-04-25T16:52:34.873506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878233+0000","last_change":"2024-04-25T16:52:26.802075+0000","last_active":"2024-04-25T16:52:34.878233+0000","last_peered":"2024-04-25T16:52:34.878233+0000","last_clean":"2024-04-25T16:52:34.878233+0000","last_became_active":"2024-04-25T16:52:26.801203+0000","last_became_peered":"2024-04-25T16:52:26.801203+0000","last_unstale":"2024-04-25T16:52:34.878233+0000","last_undegraded":"2024-04-25T16:52:34.878233+0000","last_fullsized":"2024-04-25T16:52:34.878233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782213+0000","last_change":"2024-04-25T16:52:26.793541+0000","last_active":"2024-04-25T16:52:27.782213+0000","last_peered":"2024-04-25T16:52:27.782213+0000","last_clean":"2024-04-25T16:52:27.782213+0000","last_became_active":"2024-04-25T16:52:26.789101+0000","last_became_peered":"2024-04-25T16:52:26.789101+0000","last_unstale":"2024-04-25T16:52:27.782213+0000","last_undegraded":"2024-04-25T16:52:27.782213+0000","last_fullsized":"2024-04-25T16:52:27.782213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:38.013058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781321+0000","last_change":"2024-04-25T16:52:26.784467+0000","last_active":"2024-04-25T16:52:27.781321+0000","last_peered":"2024-04-25T16:52:27.781321+0000","last_clean":"2024-04-25T16:52:27.781321+0000","last_became_active":"2024-04-25T16:52:26.783673+0000","last_became_peered":"2024-04-25T16:52:26.783673+0000","last_unstale":"2024-04-25T16:52:27.781321+0000","last_undegraded":"2024-04-25T16:52:27.781321+0000","last_fullsized":"2024-04-25T16:52:27.781321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:36.583897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831274+0000","last_change":"2024-04-25T16:52:26.823413+0000","last_active":"2024-04-25T16:52:27.831274+0000","last_peered":"2024-04-25T16:52:27.831274+0000","last_clean":"2024-04-25T16:52:27.831274+0000","last_became_active":"2024-04-25T16:52:26.823059+0000","last_became_peered":"2024-04-25T16:52:26.823059+0000","last_unstale":"2024-04-25T16:52:27.831274+0000","last_undegraded":"2024-04-25T16:52:27.831274+0000","last_fullsized":"2024-04-25T16:52:27.831274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.876836+0000","last_change":"2024-04-25T16:52:26.808661+0000","last_active":"2024-04-25T16:52:34.876836+0000","last_peered":"2024-04-25T16:52:34.876836+0000","last_clean":"2024-04-25T16:52:34.876836+0000","last_became_active":"2024-04-25T16:52:26.805296+0000","last_became_peered":"2024-04-25T16:52:26.805296+0000","last_unstale":"2024-04-25T16:52:34.876836+0000","last_undegraded":"2024-04-25T16:52:34.876836+0000","last_fullsized":"2024-04-25T16:52:34.876836+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"peering","last_fresh":"2024-04-25T16:52:30.709501+0000","last_change":"2024-04-25T16:52:34.873590+0000","last_active":"2024-04-25T16:52:34.873590+0000","last_peered":"2024-04-25T16:52:30.709501+0000","last_clean":"2024-04-25T16:52:30.709501+0000","last_became_active":"2024-04-25T16:52:26.820528+0000","last_became_peered":"2024-04-25T16:52:26.820528+0000","last_unstale":"2024-04-25T16:52:30.709501+0000","last_undegraded":"2024-04-25T16:52:30.709501+0000","last_fullsized":"2024-04-25T16:52:30.709501+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.879080+0000","last_change":"2024-04-25T16:52:26.803363+0000","last_active":"2024-04-25T16:52:34.879080+0000","last_peered":"2024-04-25T16:52:34.879080+0000","last_clean":"2024-04-25T16:52:34.879080+0000","last_became_active":"2024-04-25T16:52:26.802539+0000","last_became_peered":"2024-04-25T16:52:26.802539+0000","last_unstale":"2024-04-25T16:52:34.879080+0000","last_undegraded":"2024-04-25T16:52:34.879080+0000","last_fullsized":"2024-04-25T16:52:34.879080+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877890+0000","last_change":"2024-04-25T16:52:26.842728+0000","last_active":"2024-04-25T16:52:34.877890+0000","last_peered":"2024-04-25T16:52:34.877890+0000","last_clean":"2024-04-25T16:52:34.877890+0000","last_became_active":"2024-04-25T16:52:26.842041+0000","last_became_peered":"2024-04-25T16:52:26.842041+0000","last_unstale":"2024-04-25T16:52:34.877890+0000","last_undegraded":"2024-04-25T16:52:34.877890+0000","last_fullsized":"2024-04-25T16:52:34.877890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:33.962706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877607+0000","last_change":"2024-04-25T16:52:26.819329+0000","last_active":"2024-04-25T16:52:34.877607+0000","last_peered":"2024-04-25T16:52:34.877607+0000","last_clean":"2024-04-25T16:52:34.877607+0000","last_became_active":"2024-04-25T16:52:26.818528+0000","last_became_peered":"2024-04-25T16:52:26.818528+0000","last_unstale":"2024-04-25T16:52:34.877607+0000","last_undegraded":"2024-04-25T16:52:34.877607+0000","last_fullsized":"2024-04-25T16:52:34.877607+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878111+0000","last_change":"2024-04-25T16:52:26.795942+0000","last_active":"2024-04-25T16:52:34.878111+0000","last_peered":"2024-04-25T16:52:34.878111+0000","last_clean":"2024-04-25T16:52:34.878111+0000","last_became_active":"2024-04-25T16:52:26.795124+0000","last_became_peered":"2024-04-25T16:52:26.795124+0000","last_unstale":"2024-04-25T16:52:34.878111+0000","last_undegraded":"2024-04-25T16:52:34.878111+0000","last_fullsized":"2024-04-25T16:52:34.878111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877026+0000","last_change":"2024-04-25T16:52:26.814749+0000","last_active":"2024-04-25T16:52:34.877026+0000","last_peered":"2024-04-25T16:52:34.877026+0000","last_clean":"2024-04-25T16:52:34.877026+0000","last_became_active":"2024-04-25T16:52:26.814020+0000","last_became_peered":"2024-04-25T16:52:26.814020+0000","last_unstale":"2024-04-25T16:52:34.877026+0000","last_undegraded":"2024-04-25T16:52:34.877026+0000","last_fullsized":"2024-04-25T16:52:34.877026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871890+0000","last_change":"2024-04-25T16:52:26.813118+0000","last_active":"2024-04-25T16:52:34.871890+0000","last_peered":"2024-04-25T16:52:34.871890+0000","last_clean":"2024-04-25T16:52:34.871890+0000","last_became_active":"2024-04-25T16:52:26.812400+0000","last_became_peered":"2024-04-25T16:52:26.812400+0000","last_unstale":"2024-04-25T16:52:34.871890+0000","last_undegraded":"2024-04-25T16:52:34.871890+0000","last_fullsized":"2024-04-25T16:52:34.871890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:16.169654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877027+0000","last_change":"2024-04-25T16:52:26.795864+0000","last_active":"2024-04-25T16:52:34.877027+0000","last_peered":"2024-04-25T16:52:34.877027+0000","last_clean":"2024-04-25T16:52:34.877027+0000","last_became_active":"2024-04-25T16:52:26.794738+0000","last_became_peered":"2024-04-25T16:52:26.794738+0000","last_unstale":"2024-04-25T16:52:34.877027+0000","last_undegraded":"2024-04-25T16:52:34.877027+0000","last_fullsized":"2024-04-25T16:52:34.877027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340722+0000","last_change":"2024-04-25T16:52:26.836640+0000","last_active":"2024-04-25T16:52:35.340722+0000","last_peered":"2024-04-25T16:52:35.340722+0000","last_clean":"2024-04-25T16:52:35.340722+0000","last_became_active":"2024-04-25T16:52:26.833071+0000","last_became_peered":"2024-04-25T16:52:26.833071+0000","last_unstale":"2024-04-25T16:52:35.340722+0000","last_undegraded":"2024-04-25T16:52:35.340722+0000","last_fullsized":"2024-04-25T16:52:35.340722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341191+0000","last_change":"2024-04-25T16:52:26.841207+0000","last_active":"2024-04-25T16:52:35.341191+0000","last_peered":"2024-04-25T16:52:35.341191+0000","last_clean":"2024-04-25T16:52:35.341191+0000","last_became_active":"2024-04-25T16:52:26.838750+0000","last_became_peered":"2024-04-25T16:52:26.838750+0000","last_unstale":"2024-04-25T16:52:35.341191+0000","last_undegraded":"2024-04-25T16:52:35.341191+0000","last_fullsized":"2024-04-25T16:52:35.341191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875046+0000","last_change":"2024-04-25T16:52:26.795939+0000","last_active":"2024-04-25T16:52:34.875046+0000","last_peered":"2024-04-25T16:52:34.875046+0000","last_clean":"2024-04-25T16:52:34.875046+0000","last_became_active":"2024-04-25T16:52:26.795084+0000","last_became_peered":"2024-04-25T16:52:26.795084+0000","last_unstale":"2024-04-25T16:52:34.875046+0000","last_undegraded":"2024-04-25T16:52:34.875046+0000","last_fullsized":"2024-04-25T16:52:34.875046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781139+0000","last_change":"2024-04-25T16:52:26.818025+0000","last_active":"2024-04-25T16:52:27.781139+0000","last_peered":"2024-04-25T16:52:27.781139+0000","last_clean":"2024-04-25T16:52:27.781139+0000","last_became_active":"2024-04-25T16:52:26.817652+0000","last_became_peered":"2024-04-25T16:52:26.817652+0000","last_unstale":"2024-04-25T16:52:27.781139+0000","last_undegraded":"2024-04-25T16:52:27.781139+0000","last_fullsized":"2024-04-25T16:52:27.781139+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:46.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872811+0000","last_change":"2024-04-25T16:52:26.796384+0000","last_active":"2024-04-25T16:52:34.872811+0000","last_peered":"2024-04-25T16:52:34.872811+0000","last_clean":"2024-04-25T16:52:34.872811+0000","last_became_active":"2024-04-25T16:52:26.795507+0000","last_became_peered":"2024-04-25T16:52:26.795507+0000","last_unstale":"2024-04-25T16:52:34.872811+0000","last_undegraded":"2024-04-25T16:52:34.872811+0000","last_fullsized":"2024-04-25T16:52:34.872811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877104+0000","last_change":"2024-04-25T16:52:26.809988+0000","last_active":"2024-04-25T16:52:34.877104+0000","last_peered":"2024-04-25T16:52:34.877104+0000","last_clean":"2024-04-25T16:52:34.877104+0000","last_became_active":"2024-04-25T16:52:26.809107+0000","last_became_peered":"2024-04-25T16:52:26.809107+0000","last_unstale":"2024-04-25T16:52:34.877104+0000","last_undegraded":"2024-04-25T16:52:34.877104+0000","last_fullsized":"2024-04-25T16:52:34.877104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872865+0000","last_change":"2024-04-25T16:52:26.784466+0000","last_active":"2024-04-25T16:52:34.872865+0000","last_peered":"2024-04-25T16:52:34.872865+0000","last_clean":"2024-04-25T16:52:34.872865+0000","last_became_active":"2024-04-25T16:52:26.783550+0000","last_became_peered":"2024-04-25T16:52:26.783550+0000","last_unstale":"2024-04-25T16:52:34.872865+0000","last_undegraded":"2024-04-25T16:52:34.872865+0000","last_fullsized":"2024-04-25T16:52:34.872865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340668+0000","last_change":"2024-04-25T16:52:26.832452+0000","last_active":"2024-04-25T16:52:35.340668+0000","last_peered":"2024-04-25T16:52:35.340668+0000","last_clean":"2024-04-25T16:52:35.340668+0000","last_became_active":"2024-04-25T16:52:26.831753+0000","last_became_peered":"2024-04-25T16:52:26.831753+0000","last_unstale":"2024-04-25T16:52:35.340668+0000","last_undegraded":"2024-04-25T16:52:35.340668+0000","last_fullsized":"2024-04-25T16:52:35.340668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780752+0000","last_change":"2024-04-25T16:52:26.805633+0000","last_active":"2024-04-25T16:52:27.780752+0000","last_peered":"2024-04-25T16:52:27.780752+0000","last_clean":"2024-04-25T16:52:27.780752+0000","last_became_active":"2024-04-25T16:52:26.804789+0000","last_became_peered":"2024-04-25T16:52:26.804789+0000","last_unstale":"2024-04-25T16:52:27.780752+0000","last_undegraded":"2024-04-25T16:52:27.780752+0000","last_fullsized":"2024-04-25T16:52:27.780752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:08.931465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875174+0000","last_change":"2024-04-25T16:52:26.798693+0000","last_active":"2024-04-25T16:52:34.875174+0000","last_peered":"2024-04-25T16:52:34.875174+0000","last_clean":"2024-04-25T16:52:34.875174+0000","last_became_active":"2024-04-25T16:52:26.797852+0000","last_became_peered":"2024-04-25T16:52:26.797852+0000","last_unstale":"2024-04-25T16:52:34.875174+0000","last_undegraded":"2024-04-25T16:52:34.875174+0000","last_fullsized":"2024-04-25T16:52:34.875174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831376+0000","last_change":"2024-04-25T16:52:26.819630+0000","last_active":"2024-04-25T16:52:27.831376+0000","last_peered":"2024-04-25T16:52:27.831376+0000","last_clean":"2024-04-25T16:52:27.831376+0000","last_became_active":"2024-04-25T16:52:26.819274+0000","last_became_peered":"2024-04-25T16:52:26.819274+0000","last_unstale":"2024-04-25T16:52:27.831376+0000","last_undegraded":"2024-04-25T16:52:27.831376+0000","last_fullsized":"2024-04-25T16:52:27.831376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873718+0000","last_change":"2024-04-25T16:52:26.791433+0000","last_active":"2024-04-25T16:52:34.873718+0000","last_peered":"2024-04-25T16:52:34.873718+0000","last_clean":"2024-04-25T16:52:34.873718+0000","last_became_active":"2024-04-25T16:52:26.790483+0000","last_became_peered":"2024-04-25T16:52:26.790483+0000","last_unstale":"2024-04-25T16:52:34.873718+0000","last_undegraded":"2024-04-25T16:52:34.873718+0000","last_fullsized":"2024-04-25T16:52:34.873718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877762+0000","last_change":"2024-04-25T16:52:26.804860+0000","last_active":"2024-04-25T16:52:34.877762+0000","last_peered":"2024-04-25T16:52:34.877762+0000","last_clean":"2024-04-25T16:52:34.877762+0000","last_became_active":"2024-04-25T16:52:26.804065+0000","last_became_peered":"2024-04-25T16:52:26.804065+0000","last_unstale":"2024-04-25T16:52:34.877762+0000","last_undegraded":"2024-04-25T16:52:34.877762+0000","last_fullsized":"2024-04-25T16:52:34.877762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873009+0000","last_change":"2024-04-25T16:52:26.806705+0000","last_active":"2024-04-25T16:52:34.873009+0000","last_peered":"2024-04-25T16:52:34.873009+0000","last_clean":"2024-04-25T16:52:34.873009+0000","last_became_active":"2024-04-25T16:52:26.805889+0000","last_became_peered":"2024-04-25T16:52:26.805889+0000","last_unstale":"2024-04-25T16:52:34.873009+0000","last_undegraded":"2024-04-25T16:52:34.873009+0000","last_fullsized":"2024-04-25T16:52:34.873009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874258+0000","last_change":"2024-04-25T16:52:26.798903+0000","last_active":"2024-04-25T16:52:34.874258+0000","last_peered":"2024-04-25T16:52:34.874258+0000","last_clean":"2024-04-25T16:52:34.874258+0000","last_became_active":"2024-04-25T16:52:26.798011+0000","last_became_peered":"2024-04-25T16:52:26.798011+0000","last_unstale":"2024-04-25T16:52:34.874258+0000","last_undegraded":"2024-04-25T16:52:34.874258+0000","last_fullsized":"2024-04-25T16:52:34.874258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340837+0000","last_change":"2024-04-25T16:52:26.841037+0000","last_active":"2024-04-25T16:52:35.340837+0000","last_peered":"2024-04-25T16:52:35.340837+0000","last_clean":"2024-04-25T16:52:35.340837+0000","last_became_active":"2024-04-25T16:52:26.838194+0000","last_became_peered":"2024-04-25T16:52:26.838194+0000","last_unstale":"2024-04-25T16:52:35.340837+0000","last_undegraded":"2024-04-25T16:52:35.340837+0000","last_fullsized":"2024-04-25T16:52:35.340837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830883+0000","last_change":"2024-04-25T16:52:26.813243+0000","last_active":"2024-04-25T16:52:27.830883+0000","last_peered":"2024-04-25T16:52:27.830883+0000","last_clean":"2024-04-25T16:52:27.830883+0000","last_became_active":"2024-04-25T16:52:26.811807+0000","last_became_peered":"2024-04-25T16:52:26.811807+0000","last_unstale":"2024-04-25T16:52:27.830883+0000","last_undegraded":"2024-04-25T16:52:27.830883+0000","last_fullsized":"2024-04-25T16:52:27.830883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:42.553053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341094+0000","last_change":"2024-04-25T16:52:26.841017+0000","last_active":"2024-04-25T16:52:35.341094+0000","last_peered":"2024-04-25T16:52:35.341094+0000","last_clean":"2024-04-25T16:52:35.341094+0000","last_became_active":"2024-04-25T16:52:26.837543+0000","last_became_peered":"2024-04-25T16:52:26.837543+0000","last_unstale":"2024-04-25T16:52:35.341094+0000","last_undegraded":"2024-04-25T16:52:35.341094+0000","last_fullsized":"2024-04-25T16:52:35.341094+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875394+0000","last_change":"2024-04-25T16:52:25.818069+0000","last_active":"2024-04-25T16:52:34.875394+0000","last_peered":"2024-04-25T16:52:34.875394+0000","last_clean":"2024-04-25T16:52:34.875394+0000","last_became_active":"2024-04-25T16:52:25.817362+0000","last_became_peered":"2024-04-25T16:52:25.817362+0000","last_unstale":"2024-04-25T16:52:34.875394+0000","last_undegraded":"2024-04-25T16:52:34.875394+0000","last_fullsized":"2024-04-25T16:52:34.875394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:08.488136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833310+0000","last_change":"2024-04-25T16:52:26.829008+0000","last_active":"2024-04-25T16:52:27.833310+0000","last_peered":"2024-04-25T16:52:27.833310+0000","last_clean":"2024-04-25T16:52:27.833310+0000","last_became_active":"2024-04-25T16:52:26.828706+0000","last_became_peered":"2024-04-25T16:52:26.828706+0000","last_unstale":"2024-04-25T16:52:27.833310+0000","last_undegraded":"2024-04-25T16:52:27.833310+0000","last_fullsized":"2024-04-25T16:52:27.833310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878876+0000","last_change":"2024-04-25T16:52:25.795967+0000","last_active":"2024-04-25T16:52:34.878876+0000","last_peered":"2024-04-25T16:52:34.878876+0000","last_clean":"2024-04-25T16:52:34.878876+0000","last_became_active":"2024-04-25T16:52:25.795695+0000","last_became_peered":"2024-04-25T16:52:25.795695+0000","last_unstale":"2024-04-25T16:52:34.878876+0000","last_undegraded":"2024-04-25T16:52:34.878876+0000","last_fullsized":"2024-04-25T16:52:34.878876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875243+0000","last_change":"2024-04-25T16:52:26.789430+0000","last_active":"2024-04-25T16:52:34.875243+0000","last_peered":"2024-04-25T16:52:34.875243+0000","last_clean":"2024-04-25T16:52:34.875243+0000","last_became_active":"2024-04-25T16:52:26.782243+0000","last_became_peered":"2024-04-25T16:52:26.782243+0000","last_unstale":"2024-04-25T16:52:34.875243+0000","last_undegraded":"2024-04-25T16:52:34.875243+0000","last_fullsized":"2024-04-25T16:52:34.875243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833428+0000","last_change":"2024-04-25T16:52:25.819162+0000","last_active":"2024-04-25T16:52:27.833428+0000","last_peered":"2024-04-25T16:52:27.833428+0000","last_clean":"2024-04-25T16:52:27.833428+0000","last_became_active":"2024-04-25T16:52:25.818859+0000","last_became_peered":"2024-04-25T16:52:25.818859+0000","last_unstale":"2024-04-25T16:52:27.833428+0000","last_undegraded":"2024-04-25T16:52:27.833428+0000","last_fullsized":"2024-04-25T16:52:27.833428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874082+0000","last_change":"2024-04-25T16:52:26.805376+0000","last_active":"2024-04-25T16:52:34.874082+0000","last_peered":"2024-04-25T16:52:34.874082+0000","last_clean":"2024-04-25T16:52:34.874082+0000","last_became_active":"2024-04-25T16:52:26.804542+0000","last_became_peered":"2024-04-25T16:52:26.804542+0000","last_unstale":"2024-04-25T16:52:34.874082+0000","last_undegraded":"2024-04-25T16:52:34.874082+0000","last_fullsized":"2024-04-25T16:52:34.874082+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.620224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874004+0000","last_change":"2024-04-25T16:52:25.815751+0000","last_active":"2024-04-25T16:52:34.874004+0000","last_peered":"2024-04-25T16:52:34.874004+0000","last_clean":"2024-04-25T16:52:34.874004+0000","last_became_active":"2024-04-25T16:52:25.814927+0000","last_became_peered":"2024-04-25T16:52:25.814927+0000","last_unstale":"2024-04-25T16:52:34.874004+0000","last_undegraded":"2024-04-25T16:52:34.874004+0000","last_fullsized":"2024-04-25T16:52:34.874004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877461+0000","last_change":"2024-04-25T16:52:26.814640+0000","last_active":"2024-04-25T16:52:34.877461+0000","last_peered":"2024-04-25T16:52:34.877461+0000","last_clean":"2024-04-25T16:52:34.877461+0000","last_became_active":"2024-04-25T16:52:26.814016+0000","last_became_peered":"2024-04-25T16:52:26.814016+0000","last_unstale":"2024-04-25T16:52:34.877461+0000","last_undegraded":"2024-04-25T16:52:34.877461+0000","last_fullsized":"2024-04-25T16:52:34.877461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877431+0000","last_change":"2024-04-25T16:52:25.805323+0000","last_active":"2024-04-25T16:52:34.877431+0000","last_peered":"2024-04-25T16:52:34.877431+0000","last_clean":"2024-04-25T16:52:34.877431+0000","last_became_active":"2024-04-25T16:52:25.796444+0000","last_became_peered":"2024-04-25T16:52:25.796444+0000","last_unstale":"2024-04-25T16:52:34.877431+0000","last_undegraded":"2024-04-25T16:52:34.877431+0000","last_fullsized":"2024-04-25T16:52:34.877431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:28.996351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877181+0000","last_change":"2024-04-25T16:52:26.821648+0000","last_active":"2024-04-25T16:52:34.877181+0000","last_peered":"2024-04-25T16:52:34.877181+0000","last_clean":"2024-04-25T16:52:34.877181+0000","last_became_active":"2024-04-25T16:52:26.820835+0000","last_became_peered":"2024-04-25T16:52:26.820835+0000","last_unstale":"2024-04-25T16:52:34.877181+0000","last_undegraded":"2024-04-25T16:52:34.877181+0000","last_fullsized":"2024-04-25T16:52:34.877181+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340828+0000","last_change":"2024-04-25T16:52:25.816250+0000","last_active":"2024-04-25T16:52:35.340828+0000","last_peered":"2024-04-25T16:52:35.340828+0000","last_clean":"2024-04-25T16:52:35.340828+0000","last_became_active":"2024-04-25T16:52:25.815656+0000","last_became_peered":"2024-04-25T16:52:25.815656+0000","last_unstale":"2024-04-25T16:52:35.340828+0000","last_undegraded":"2024-04-25T16:52:35.340828+0000","last_fullsized":"2024-04-25T16:52:35.340828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872461+0000","last_change":"2024-04-25T16:52:26.813425+0000","last_active":"2024-04-25T16:52:34.872461+0000","last_peered":"2024-04-25T16:52:34.872461+0000","last_clean":"2024-04-25T16:52:34.872461+0000","last_became_active":"2024-04-25T16:52:26.812713+0000","last_became_peered":"2024-04-25T16:52:26.812713+0000","last_unstale":"2024-04-25T16:52:34.872461+0000","last_undegraded":"2024-04-25T16:52:34.872461+0000","last_fullsized":"2024-04-25T16:52:34.872461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831072+0000","last_change":"2024-04-25T16:52:25.803265+0000","last_active":"2024-04-25T16:52:27.831072+0000","last_peered":"2024-04-25T16:52:27.831072+0000","last_clean":"2024-04-25T16:52:27.831072+0000","last_became_active":"2024-04-25T16:52:25.802617+0000","last_became_peered":"2024-04-25T16:52:25.802617+0000","last_unstale":"2024-04-25T16:52:27.831072+0000","last_undegraded":"2024-04-25T16:52:27.831072+0000","last_fullsized":"2024-04-25T16:52:27.831072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872313+0000","last_change":"2024-04-25T16:52:26.819267+0000","last_active":"2024-04-25T16:52:34.872313+0000","last_peered":"2024-04-25T16:52:34.872313+0000","last_clean":"2024-04-25T16:52:34.872313+0000","last_became_active":"2024-04-25T16:52:26.818740+0000","last_became_peered":"2024-04-25T16:52:26.818740+0000","last_unstale":"2024-04-25T16:52:34.872313+0000","last_undegraded":"2024-04-25T16:52:34.872313+0000","last_fullsized":"2024-04-25T16:52:34.872313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871955+0000","last_change":"2024-04-25T16:52:25.816663+0000","last_active":"2024-04-25T16:52:34.871955+0000","last_peered":"2024-04-25T16:52:34.871955+0000","last_clean":"2024-04-25T16:52:34.871955+0000","last_became_active":"2024-04-25T16:52:25.815420+0000","last_became_peered":"2024-04-25T16:52:25.815420+0000","last_unstale":"2024-04-25T16:52:34.871955+0000","last_undegraded":"2024-04-25T16:52:34.871955+0000","last_fullsized":"2024-04-25T16:52:34.871955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:30.288538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782313+0000","last_change":"2024-04-25T16:52:26.789927+0000","last_active":"2024-04-25T16:52:27.782313+0000","last_peered":"2024-04-25T16:52:27.782313+0000","last_clean":"2024-04-25T16:52:27.782313+0000","last_became_active":"2024-04-25T16:52:26.788014+0000","last_became_peered":"2024-04-25T16:52:26.788014+0000","last_unstale":"2024-04-25T16:52:27.782313+0000","last_undegraded":"2024-04-25T16:52:27.782313+0000","last_fullsized":"2024-04-25T16:52:27.782313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:47.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873885+0000","last_change":"2024-04-25T16:52:25.798974+0000","last_active":"2024-04-25T16:52:34.873885+0000","last_peered":"2024-04-25T16:52:34.873885+0000","last_clean":"2024-04-25T16:52:34.873885+0000","last_became_active":"2024-04-25T16:52:25.798314+0000","last_became_peered":"2024-04-25T16:52:25.798314+0000","last_unstale":"2024-04-25T16:52:34.873885+0000","last_undegraded":"2024-04-25T16:52:34.873885+0000","last_fullsized":"2024-04-25T16:52:34.873885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.348530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341450+0000","last_change":"2024-04-25T16:52:26.842485+0000","last_active":"2024-04-25T16:52:35.341450+0000","last_peered":"2024-04-25T16:52:35.341450+0000","last_clean":"2024-04-25T16:52:35.341450+0000","last_became_active":"2024-04-25T16:52:26.838813+0000","last_became_peered":"2024-04-25T16:52:26.838813+0000","last_unstale":"2024-04-25T16:52:35.341450+0000","last_undegraded":"2024-04-25T16:52:35.341450+0000","last_fullsized":"2024-04-25T16:52:35.341450+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878521+0000","last_change":"2024-04-25T16:52:25.803289+0000","last_active":"2024-04-25T16:52:34.878521+0000","last_peered":"2024-04-25T16:52:34.878521+0000","last_clean":"2024-04-25T16:52:34.878521+0000","last_became_active":"2024-04-25T16:52:25.801415+0000","last_became_peered":"2024-04-25T16:52:25.801415+0000","last_unstale":"2024-04-25T16:52:34.878521+0000","last_undegraded":"2024-04-25T16:52:34.878521+0000","last_fullsized":"2024-04-25T16:52:34.878521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:45.310089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878182+0000","last_change":"2024-04-25T16:52:26.840379+0000","last_active":"2024-04-25T16:52:34.878182+0000","last_peered":"2024-04-25T16:52:34.878182+0000","last_clean":"2024-04-25T16:52:34.878182+0000","last_became_active":"2024-04-25T16:52:26.840062+0000","last_became_peered":"2024-04-25T16:52:26.840062+0000","last_unstale":"2024-04-25T16:52:34.878182+0000","last_undegraded":"2024-04-25T16:52:34.878182+0000","last_fullsized":"2024-04-25T16:52:34.878182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878024+0000","last_change":"2024-04-25T16:52:25.809391+0000","last_active":"2024-04-25T16:52:34.878024+0000","last_peered":"2024-04-25T16:52:34.878024+0000","last_clean":"2024-04-25T16:52:34.878024+0000","last_became_active":"2024-04-25T16:52:25.808455+0000","last_became_peered":"2024-04-25T16:52:25.808455+0000","last_unstale":"2024-04-25T16:52:34.878024+0000","last_undegraded":"2024-04-25T16:52:34.878024+0000","last_fullsized":"2024-04-25T16:52:34.878024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872072+0000","last_change":"2024-04-25T16:52:26.782954+0000","last_active":"2024-04-25T16:52:34.872072+0000","last_peered":"2024-04-25T16:52:34.872072+0000","last_clean":"2024-04-25T16:52:34.872072+0000","last_became_active":"2024-04-25T16:52:26.782108+0000","last_became_peered":"2024-04-25T16:52:26.782108+0000","last_unstale":"2024-04-25T16:52:34.872072+0000","last_undegraded":"2024-04-25T16:52:34.872072+0000","last_fullsized":"2024-04-25T16:52:34.872072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873363+0000","last_change":"2024-04-25T16:52:25.795020+0000","last_active":"2024-04-25T16:52:34.873363+0000","last_peered":"2024-04-25T16:52:34.873363+0000","last_clean":"2024-04-25T16:52:34.873363+0000","last_became_active":"2024-04-25T16:52:25.794633+0000","last_became_peered":"2024-04-25T16:52:25.794633+0000","last_unstale":"2024-04-25T16:52:34.873363+0000","last_undegraded":"2024-04-25T16:52:34.873363+0000","last_fullsized":"2024-04-25T16:52:34.873363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:38.694859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877510+0000","last_change":"2024-04-25T16:52:26.821081+0000","last_active":"2024-04-25T16:52:34.877510+0000","last_peered":"2024-04-25T16:52:34.877510+0000","last_clean":"2024-04-25T16:52:34.877510+0000","last_became_active":"2024-04-25T16:52:26.819337+0000","last_became_peered":"2024-04-25T16:52:26.819337+0000","last_unstale":"2024-04-25T16:52:34.877510+0000","last_undegraded":"2024-04-25T16:52:34.877510+0000","last_fullsized":"2024-04-25T16:52:34.877510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872999+0000","last_change":"2024-04-25T16:52:25.793039+0000","last_active":"2024-04-25T16:52:34.872999+0000","last_peered":"2024-04-25T16:52:34.872999+0000","last_clean":"2024-04-25T16:52:34.872999+0000","last_became_active":"2024-04-25T16:52:25.792552+0000","last_became_peered":"2024-04-25T16:52:25.792552+0000","last_unstale":"2024-04-25T16:52:34.872999+0000","last_undegraded":"2024-04-25T16:52:34.872999+0000","last_fullsized":"2024-04-25T16:52:34.872999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:30.954700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872507+0000","last_change":"2024-04-25T16:52:26.823643+0000","last_active":"2024-04-25T16:52:34.872507+0000","last_peered":"2024-04-25T16:52:34.872507+0000","last_clean":"2024-04-25T16:52:34.872507+0000","last_became_active":"2024-04-25T16:52:26.823348+0000","last_became_peered":"2024-04-25T16:52:26.823348+0000","last_unstale":"2024-04-25T16:52:34.872507+0000","last_undegraded":"2024-04-25T16:52:34.872507+0000","last_fullsized":"2024-04-25T16:52:34.872507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830926+0000","last_change":"2024-04-25T16:52:25.813706+0000","last_active":"2024-04-25T16:52:27.830926+0000","last_peered":"2024-04-25T16:52:27.830926+0000","last_clean":"2024-04-25T16:52:27.830926+0000","last_became_active":"2024-04-25T16:52:25.813387+0000","last_became_peered":"2024-04-25T16:52:25.813387+0000","last_unstale":"2024-04-25T16:52:27.830926+0000","last_undegraded":"2024-04-25T16:52:27.830926+0000","last_fullsized":"2024-04-25T16:52:27.830926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340755+0000","last_change":"2024-04-25T16:52:26.817332+0000","last_active":"2024-04-25T16:52:35.340755+0000","last_peered":"2024-04-25T16:52:35.340755+0000","last_clean":"2024-04-25T16:52:35.340755+0000","last_became_active":"2024-04-25T16:52:26.816020+0000","last_became_peered":"2024-04-25T16:52:26.816020+0000","last_unstale":"2024-04-25T16:52:35.340755+0000","last_undegraded":"2024-04-25T16:52:35.340755+0000","last_fullsized":"2024-04-25T16:52:35.340755+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872306+0000","last_change":"2024-04-25T16:52:25.815545+0000","last_active":"2024-04-25T16:52:34.872306+0000","last_peered":"2024-04-25T16:52:34.872306+0000","last_clean":"2024-04-25T16:52:34.872306+0000","last_became_active":"2024-04-25T16:52:25.814711+0000","last_became_peered":"2024-04-25T16:52:25.814711+0000","last_unstale":"2024-04-25T16:52:34.872306+0000","last_undegraded":"2024-04-25T16:52:34.872306+0000","last_fullsized":"2024-04-25T16:52:34.872306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872682+0000","last_change":"2024-04-25T16:52:26.826567+0000","last_active":"2024-04-25T16:52:34.872682+0000","last_peered":"2024-04-25T16:52:34.872682+0000","last_clean":"2024-04-25T16:52:34.872682+0000","last_became_active":"2024-04-25T16:52:26.826224+0000","last_became_peered":"2024-04-25T16:52:26.826224+0000","last_unstale":"2024-04-25T16:52:34.872682+0000","last_undegraded":"2024-04-25T16:52:34.872682+0000","last_fullsized":"2024-04-25T16:52:34.872682+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:06.823841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":24,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340795+0000","last_change":"2024-04-25T16:52:25.815153+0000","last_active":"2024-04-25T16:52:35.340795+0000","last_peered":"2024-04-25T16:52:35.340795+0000","last_clean":"2024-04-25T16:52:35.340795+0000","last_became_active":"2024-04-25T16:52:25.814360+0000","last_became_peered":"2024-04-25T16:52:25.814360+0000","last_unstale":"2024-04-25T16:52:35.340795+0000","last_undegraded":"2024-04-25T16:52:35.340795+0000","last_fullsized":"2024-04-25T16:52:35.340795+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873169+0000","last_change":"2024-04-25T16:52:26.804704+0000","last_active":"2024-04-25T16:52:34.873169+0000","last_peered":"2024-04-25T16:52:34.873169+0000","last_clean":"2024-04-25T16:52:34.873169+0000","last_became_active":"2024-04-25T16:52:26.803888+0000","last_became_peered":"2024-04-25T16:52:26.803888+0000","last_unstale":"2024-04-25T16:52:34.873169+0000","last_undegraded":"2024-04-25T16:52:34.873169+0000","last_fullsized":"2024-04-25T16:52:34.873169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.640204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878107+0000","last_change":"2024-04-25T16:52:25.818880+0000","last_active":"2024-04-25T16:52:34.878107+0000","last_peered":"2024-04-25T16:52:34.878107+0000","last_clean":"2024-04-25T16:52:34.878107+0000","last_became_active":"2024-04-25T16:52:25.817841+0000","last_became_peered":"2024-04-25T16:52:25.817841+0000","last_unstale":"2024-04-25T16:52:34.878107+0000","last_undegraded":"2024-04-25T16:52:34.878107+0000","last_fullsized":"2024-04-25T16:52:34.878107+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:26.817110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877704+0000","last_change":"2024-04-25T16:52:26.809554+0000","last_active":"2024-04-25T16:52:34.877704+0000","last_peered":"2024-04-25T16:52:34.877704+0000","last_clean":"2024-04-25T16:52:34.877704+0000","last_became_active":"2024-04-25T16:52:26.808709+0000","last_became_peered":"2024-04-25T16:52:26.808709+0000","last_unstale":"2024-04-25T16:52:34.877704+0000","last_undegraded":"2024-04-25T16:52:34.877704+0000","last_fullsized":"2024-04-25T16:52:34.877704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875584+0000","last_change":"2024-04-25T16:52:25.812269+0000","last_active":"2024-04-25T16:52:34.875584+0000","last_peered":"2024-04-25T16:52:34.875584+0000","last_clean":"2024-04-25T16:52:34.875584+0000","last_became_active":"2024-04-25T16:52:25.810831+0000","last_became_peered":"2024-04-25T16:52:25.810831+0000","last_unstale":"2024-04-25T16:52:34.875584+0000","last_undegraded":"2024-04-25T16:52:34.875584+0000","last_fullsized":"2024-04-25T16:52:34.875584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874980+0000","last_change":"2024-04-25T16:52:26.806939+0000","last_active":"2024-04-25T16:52:34.874980+0000","last_peered":"2024-04-25T16:52:34.874980+0000","last_clean":"2024-04-25T16:52:34.874980+0000","last_became_active":"2024-04-25T16:52:26.806053+0000","last_became_peered":"2024-04-25T16:52:26.806053+0000","last_unstale":"2024-04-25T16:52:34.874980+0000","last_undegraded":"2024-04-25T16:52:34.874980+0000","last_fullsized":"2024-04-25T16:52:34.874980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781168+0000","last_change":"2024-04-25T16:52:25.783759+0000","last_active":"2024-04-25T16:52:27.781168+0000","last_peered":"2024-04-25T16:52:27.781168+0000","last_clean":"2024-04-25T16:52:27.781168+0000","last_became_active":"2024-04-25T16:52:25.783353+0000","last_became_peered":"2024-04-25T16:52:25.783353+0000","last_unstale":"2024-04-25T16:52:27.781168+0000","last_undegraded":"2024-04-25T16:52:27.781168+0000","last_fullsized":"2024-04-25T16:52:27.781168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:34.281502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780945+0000","last_change":"2024-04-25T16:52:26.798434+0000","last_active":"2024-04-25T16:52:27.780945+0000","last_peered":"2024-04-25T16:52:27.780945+0000","last_clean":"2024-04-25T16:52:27.780945+0000","last_became_active":"2024-04-25T16:52:26.797631+0000","last_became_peered":"2024-04-25T16:52:26.797631+0000","last_unstale":"2024-04-25T16:52:27.780945+0000","last_undegraded":"2024-04-25T16:52:27.780945+0000","last_fullsized":"2024-04-25T16:52:27.780945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:16.248387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831612+0000","last_change":"2024-04-25T16:52:25.819136+0000","last_active":"2024-04-25T16:52:27.831612+0000","last_peered":"2024-04-25T16:52:27.831612+0000","last_clean":"2024-04-25T16:52:27.831612+0000","last_became_active":"2024-04-25T16:52:25.818809+0000","last_became_peered":"2024-04-25T16:52:25.818809+0000","last_unstale":"2024-04-25T16:52:27.831612+0000","last_undegraded":"2024-04-25T16:52:27.831612+0000","last_fullsized":"2024-04-25T16:52:27.831612+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341292+0000","last_change":"2024-04-25T16:52:26.819612+0000","last_active":"2024-04-25T16:52:35.341292+0000","last_peered":"2024-04-25T16:52:35.341292+0000","last_clean":"2024-04-25T16:52:35.341292+0000","last_became_active":"2024-04-25T16:52:26.818874+0000","last_became_peered":"2024-04-25T16:52:26.818874+0000","last_unstale":"2024-04-25T16:52:35.341292+0000","last_undegraded":"2024-04-25T16:52:35.341292+0000","last_fullsized":"2024-04-25T16:52:35.341292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781486+0000","last_change":"2024-04-25T16:52:25.780874+0000","last_active":"2024-04-25T16:52:27.781486+0000","last_peered":"2024-04-25T16:52:27.781486+0000","last_clean":"2024-04-25T16:52:27.781486+0000","last_became_active":"2024-04-25T16:52:25.780529+0000","last_became_peered":"2024-04-25T16:52:25.780529+0000","last_unstale":"2024-04-25T16:52:27.781486+0000","last_undegraded":"2024-04-25T16:52:27.781486+0000","last_fullsized":"2024-04-25T16:52:27.781486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:57.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878077+0000","last_change":"2024-04-25T16:52:26.845027+0000","last_active":"2024-04-25T16:52:34.878077+0000","last_peered":"2024-04-25T16:52:34.878077+0000","last_clean":"2024-04-25T16:52:34.878077+0000","last_became_active":"2024-04-25T16:52:26.844620+0000","last_became_peered":"2024-04-25T16:52:26.844620+0000","last_unstale":"2024-04-25T16:52:34.878077+0000","last_undegraded":"2024-04-25T16:52:34.878077+0000","last_fullsized":"2024-04-25T16:52:34.878077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780947+0000","last_change":"2024-04-25T16:52:25.821889+0000","last_active":"2024-04-25T16:52:27.780947+0000","last_peered":"2024-04-25T16:52:27.780947+0000","last_clean":"2024-04-25T16:52:27.780947+0000","last_became_active":"2024-04-25T16:52:25.821643+0000","last_became_peered":"2024-04-25T16:52:25.821643+0000","last_unstale":"2024-04-25T16:52:27.780947+0000","last_undegraded":"2024-04-25T16:52:27.780947+0000","last_fullsized":"2024-04-25T16:52:27.780947+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:27.786669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877414+0000","last_change":"2024-04-25T16:52:26.802864+0000","last_active":"2024-04-25T16:52:34.877414+0000","last_peered":"2024-04-25T16:52:34.877414+0000","last_clean":"2024-04-25T16:52:34.877414+0000","last_became_active":"2024-04-25T16:52:26.801913+0000","last_became_peered":"2024-04-25T16:52:26.801913+0000","last_unstale":"2024-04-25T16:52:34.877414+0000","last_undegraded":"2024-04-25T16:52:34.877414+0000","last_fullsized":"2024-04-25T16:52:34.877414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341042+0000","last_change":"2024-04-25T16:52:25.806124+0000","last_active":"2024-04-25T16:52:35.341042+0000","last_peered":"2024-04-25T16:52:35.341042+0000","last_clean":"2024-04-25T16:52:35.341042+0000","last_became_active":"2024-04-25T16:52:25.805306+0000","last_became_peered":"2024-04-25T16:52:25.805306+0000","last_unstale":"2024-04-25T16:52:35.341042+0000","last_undegraded":"2024-04-25T16:52:35.341042+0000","last_fullsized":"2024-04-25T16:52:35.341042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872688+0000","last_change":"2024-04-25T16:52:26.819372+0000","last_active":"2024-04-25T16:52:34.872688+0000","last_peered":"2024-04-25T16:52:34.872688+0000","last_clean":"2024-04-25T16:52:34.872688+0000","last_became_active":"2024-04-25T16:52:26.807182+0000","last_became_peered":"2024-04-25T16:52:26.807182+0000","last_unstale":"2024-04-25T16:52:34.872688+0000","last_undegraded":"2024-04-25T16:52:34.872688+0000","last_fullsized":"2024-04-25T16:52:34.872688+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:14.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780505+0000","last_change":"2024-04-25T16:52:25.788959+0000","last_active":"2024-04-25T16:52:27.780505+0000","last_peered":"2024-04-25T16:52:27.780505+0000","last_clean":"2024-04-25T16:52:27.780505+0000","last_became_active":"2024-04-25T16:52:25.788261+0000","last_became_peered":"2024-04-25T16:52:25.788261+0000","last_unstale":"2024-04-25T16:52:27.780505+0000","last_undegraded":"2024-04-25T16:52:27.780505+0000","last_fullsized":"2024-04-25T16:52:27.780505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:13.943131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831765+0000","last_change":"2024-04-25T16:52:26.819641+0000","last_active":"2024-04-25T16:52:27.831765+0000","last_peered":"2024-04-25T16:52:27.831765+0000","last_clean":"2024-04-25T16:52:27.831765+0000","last_became_active":"2024-04-25T16:52:26.819314+0000","last_became_peered":"2024-04-25T16:52:26.819314+0000","last_unstale":"2024-04-25T16:52:27.831765+0000","last_undegraded":"2024-04-25T16:52:27.831765+0000","last_fullsized":"2024-04-25T16:52:27.831765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878486+0000","last_change":"2024-04-25T16:52:25.806269+0000","last_active":"2024-04-25T16:52:34.878486+0000","last_peered":"2024-04-25T16:52:34.878486+0000","last_clean":"2024-04-25T16:52:34.878486+0000","last_became_active":"2024-04-25T16:52:25.804183+0000","last_became_peered":"2024-04-25T16:52:25.804183+0000","last_unstale":"2024-04-25T16:52:34.878486+0000","last_undegraded":"2024-04-25T16:52:34.878486+0000","last_fullsized":"2024-04-25T16:52:34.878486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:07.902516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341557+0000","last_change":"2024-04-25T16:52:26.841753+0000","last_active":"2024-04-25T16:52:35.341557+0000","last_peered":"2024-04-25T16:52:35.341557+0000","last_clean":"2024-04-25T16:52:35.341557+0000","last_became_active":"2024-04-25T16:52:26.841112+0000","last_became_peered":"2024-04-25T16:52:26.841112+0000","last_unstale":"2024-04-25T16:52:35.341557+0000","last_undegraded":"2024-04-25T16:52:35.341557+0000","last_fullsized":"2024-04-25T16:52:35.341557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781212+0000","last_change":"2024-04-25T16:52:25.803168+0000","last_active":"2024-04-25T16:52:27.781212+0000","last_peered":"2024-04-25T16:52:27.781212+0000","last_clean":"2024-04-25T16:52:27.781212+0000","last_became_active":"2024-04-25T16:52:25.802880+0000","last_became_peered":"2024-04-25T16:52:25.802880+0000","last_unstale":"2024-04-25T16:52:27.781212+0000","last_undegraded":"2024-04-25T16:52:27.781212+0000","last_fullsized":"2024-04-25T16:52:27.781212+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.210482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873797+0000","last_change":"2024-04-25T16:52:26.819781+0000","last_active":"2024-04-25T16:52:34.873797+0000","last_peered":"2024-04-25T16:52:34.873797+0000","last_clean":"2024-04-25T16:52:34.873797+0000","last_became_active":"2024-04-25T16:52:26.819467+0000","last_became_peered":"2024-04-25T16:52:26.819467+0000","last_unstale":"2024-04-25T16:52:34.873797+0000","last_undegraded":"2024-04-25T16:52:34.873797+0000","last_fullsized":"2024-04-25T16:52:34.873797+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878534+0000","last_change":"2024-04-25T16:52:25.799953+0000","last_active":"2024-04-25T16:52:34.878534+0000","last_peered":"2024-04-25T16:52:34.878534+0000","last_clean":"2024-04-25T16:52:34.878534+0000","last_became_active":"2024-04-25T16:52:25.799689+0000","last_became_peered":"2024-04-25T16:52:25.799689+0000","last_unstale":"2024-04-25T16:52:34.878534+0000","last_undegraded":"2024-04-25T16:52:34.878534+0000","last_fullsized":"2024-04-25T16:52:34.878534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877833+0000","last_change":"2024-04-25T16:52:26.799367+0000","last_active":"2024-04-25T16:52:34.877833+0000","last_peered":"2024-04-25T16:52:34.877833+0000","last_clean":"2024-04-25T16:52:34.877833+0000","last_became_active":"2024-04-25T16:52:26.797337+0000","last_became_peered":"2024-04-25T16:52:26.797337+0000","last_unstale":"2024-04-25T16:52:34.877833+0000","last_undegraded":"2024-04-25T16:52:34.877833+0000","last_fullsized":"2024-04-25T16:52:34.877833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878430+0000","last_change":"2024-04-25T16:52:25.806956+0000","last_active":"2024-04-25T16:52:34.878430+0000","last_peered":"2024-04-25T16:52:34.878430+0000","last_clean":"2024-04-25T16:52:34.878430+0000","last_became_active":"2024-04-25T16:52:25.806604+0000","last_became_peered":"2024-04-25T16:52:25.806604+0000","last_unstale":"2024-04-25T16:52:34.878430+0000","last_undegraded":"2024-04-25T16:52:34.878430+0000","last_fullsized":"2024-04-25T16:52:34.878430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831008+0000","last_change":"2024-04-25T16:52:26.820845+0000","last_active":"2024-04-25T16:52:27.831008+0000","last_peered":"2024-04-25T16:52:27.831008+0000","last_clean":"2024-04-25T16:52:27.831008+0000","last_became_active":"2024-04-25T16:52:26.820484+0000","last_became_peered":"2024-04-25T16:52:26.820484+0000","last_unstale":"2024-04-25T16:52:27.831008+0000","last_undegraded":"2024-04-25T16:52:27.831008+0000","last_fullsized":"2024-04-25T16:52:27.831008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:44.359144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874630+0000","last_change":"2024-04-25T16:52:25.785723+0000","last_active":"2024-04-25T16:52:34.874630+0000","last_peered":"2024-04-25T16:52:34.874630+0000","last_clean":"2024-04-25T16:52:34.874630+0000","last_became_active":"2024-04-25T16:52:25.784930+0000","last_became_peered":"2024-04-25T16:52:25.784930+0000","last_unstale":"2024-04-25T16:52:34.874630+0000","last_undegraded":"2024-04-25T16:52:34.874630+0000","last_fullsized":"2024-04-25T16:52:34.874630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872198+0000","last_change":"2024-04-25T16:52:26.799277+0000","last_active":"2024-04-25T16:52:34.872198+0000","last_peered":"2024-04-25T16:52:34.872198+0000","last_clean":"2024-04-25T16:52:34.872198+0000","last_became_active":"2024-04-25T16:52:26.797817+0000","last_became_peered":"2024-04-25T16:52:26.797817+0000","last_unstale":"2024-04-25T16:52:34.872198+0000","last_undegraded":"2024-04-25T16:52:34.872198+0000","last_fullsized":"2024-04-25T16:52:34.872198+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871616+0000","last_change":"2024-04-25T16:52:25.810628+0000","last_active":"2024-04-25T16:52:34.871616+0000","last_peered":"2024-04-25T16:52:34.871616+0000","last_clean":"2024-04-25T16:52:34.871616+0000","last_became_active":"2024-04-25T16:52:25.809669+0000","last_became_peered":"2024-04-25T16:52:25.809669+0000","last_unstale":"2024-04-25T16:52:34.871616+0000","last_undegraded":"2024-04-25T16:52:34.871616+0000","last_fullsized":"2024-04-25T16:52:34.871616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:36.638682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341549+0000","last_change":"2024-04-25T16:52:26.832124+0000","last_active":"2024-04-25T16:52:35.341549+0000","last_peered":"2024-04-25T16:52:35.341549+0000","last_clean":"2024-04-25T16:52:35.341549+0000","last_became_active":"2024-04-25T16:52:26.831437+0000","last_became_peered":"2024-04-25T16:52:26.831437+0000","last_unstale":"2024-04-25T16:52:35.341549+0000","last_undegraded":"2024-04-25T16:52:35.341549+0000","last_fullsized":"2024-04-25T16:52:35.341549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872502+0000","last_change":"2024-04-25T16:52:25.793944+0000","last_active":"2024-04-25T16:52:34.872502+0000","last_peered":"2024-04-25T16:52:34.872502+0000","last_clean":"2024-04-25T16:52:34.872502+0000","last_became_active":"2024-04-25T16:52:25.793522+0000","last_became_peered":"2024-04-25T16:52:25.793522+0000","last_unstale":"2024-04-25T16:52:34.872502+0000","last_undegraded":"2024-04-25T16:52:34.872502+0000","last_fullsized":"2024-04-25T16:52:34.872502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:41.542439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877722+0000","last_change":"2024-04-25T16:52:26.804643+0000","last_active":"2024-04-25T16:52:34.877722+0000","last_peered":"2024-04-25T16:52:34.877722+0000","last_clean":"2024-04-25T16:52:34.877722+0000","last_became_active":"2024-04-25T16:52:26.802297+0000","last_became_peered":"2024-04-25T16:52:26.802297+0000","last_unstale":"2024-04-25T16:52:34.877722+0000","last_undegraded":"2024-04-25T16:52:34.877722+0000","last_fullsized":"2024-04-25T16:52:34.877722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341255+0000","last_change":"2024-04-25T16:52:25.796075+0000","last_active":"2024-04-25T16:52:35.341255+0000","last_peered":"2024-04-25T16:52:35.341255+0000","last_clean":"2024-04-25T16:52:35.341255+0000","last_became_active":"2024-04-25T16:52:25.795226+0000","last_became_peered":"2024-04-25T16:52:25.795226+0000","last_unstale":"2024-04-25T16:52:35.341255+0000","last_undegraded":"2024-04-25T16:52:35.341255+0000","last_fullsized":"2024-04-25T16:52:35.341255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":145,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340402+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:52:35.340402+0000","last_peered":"2024-04-25T16:52:35.340402+0000","last_clean":"2024-04-25T16:52:35.340402+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:52:35.340402+0000","last_undegraded":"2024-04-25T16:52:35.340402+0000","last_fullsized":"2024-04-25T16:52:35.340402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":196969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":196499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":458752,"data_stored":196991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35796,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336044,"statfs":{"total":96636764160,"available":96600109056,"internally_reserved":0,"allocated":466944,"data_stored":198033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":190098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35776,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336064,"statfs":{"total":96636764160,"available":96600129536,"internally_reserved":0,"allocated":442368,"data_stored":190118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335476,"statfs":{"total":96636764160,"available":96599527424,"internally_reserved":0,"allocated":1048576,"data_stored":786789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:52:36.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-04-25T16:52:36.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 13, 'reported_epoch': 38, 'state': 'peering', 'last_fresh': '2024-04-25T16:52:30.709501+0000', 'last_change': '2024-04-25T16:52:34.873590+0000', 'last_active': '2024-04-25T16:52:34.873590+0000', 'last_peered': '2024-04-25T16:52:30.709501+0000', 'last_clean': '2024-04-25T16:52:30.709501+0000', 'last_became_active': '2024-04-25T16:52:26.820528+0000', 'last_became_peered': '2024-04-25T16:52:26.820528+0000', 'last_unstale': '2024-04-25T16:52:30.709501+0000', 'last_undegraded': '2024-04-25T16:52:30.709501+0000', 'last_fullsized': '2024-04-25T16:52:30.709501+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T16:52:25.752315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T16:52:25.752315+0000', 'last_clean_scrub_stamp': '2024-04-25T16:52:25.752315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T16:52:36.326 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T16:52:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0abc954640 1 Processor -- start 2024-04-25T16:52:36.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0abc954640 1 -- start start 2024-04-25T16:52:36.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0abc954640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0ab80cdaa0 0x7f0ab80d2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0abc954640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ab80d2570 0x7f0ab80b7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0abc954640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ab80d2980 con 0x7f0ab80cdeb0 2024-04-25T16:52:36.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0abc954640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0ab80cec90 con 0x7f0ab80cdaa0 2024-04-25T16:52:36.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0abc954640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab80ce3f0 con 0x7f0ab80d2570 2024-04-25T16:52:36.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0aadd9b640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f0ab80cdeb0 0x7f0ab80b6910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56854/0 (socket says 172.21.15.94:56854) 2024-04-25T16:52:36.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0aadd9b640 1 -- 172.21.15.94:0/1490477916 learned_addr learned my addr 172.21.15.94:0/1490477916 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:36.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab659c640 1 --2- 172.21.15.94:0/1490477916 >> v2:172.21.15.138:3300/0 conn(0x7f0ab80cdaa0 0x7f0ab80d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab6d9d640 1 --2- 172.21.15.94:0/1490477916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ab80d2570 0x7f0ab80b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1230418739 0 0) 0x7f0aa0001120 con 0x7f0ab80cdeb0 2024-04-25T16:52:36.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0aa4003530 con 0x7f0ab80cdeb0 2024-04-25T16:52:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 975840255 0 0) 0x7f0aa4003530 con 0x7f0ab80cdeb0 2024-04-25T16:52:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ab80d2570 msgr2=0x7f0ab80b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 --2- 172.21.15.94:0/1490477916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ab80d2570 0x7f0ab80b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 >> v2:172.21.15.138:3300/0 conn(0x7f0ab80cdaa0 msgr2=0x7f0ab80d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 --2- 172.21.15.94:0/1490477916 >> v2:172.21.15.138:3300/0 conn(0x7f0ab80cdaa0 0x7f0ab80d2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab80ce6d0 con 0x7f0ab80cdeb0 2024-04-25T16:52:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f0aa0001850 con 0x7f0ab80cdeb0 2024-04-25T16:52:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 >> v1:172.21.15.94:6789/0 conn(0x7f0ab80cdeb0 legacy=0x7f0ab80b6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 --2- 172.21.15.94:0/1490477916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ab80d2570 0x7f0aa40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 --2- 172.21.15.94:0/1490477916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4003f40 0x7f0aa4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 --2- 172.21.15.94:0/1490477916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0aa4004d50 0x7f0aa4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab80cec90 con 0x7f0aa4003f40 2024-04-25T16:52:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab80ce3f0 con 0x7f0aa4004d50 2024-04-25T16:52:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa400bad0 con 0x7f0ab80d2570 2024-04-25T16:52:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.507+0000 7f0aadd9b640 1 --2- 172.21.15.94:0/1490477916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4003f40 0x7f0aa4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0ab659c640 1 --2- 172.21.15.94:0/1490477916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ab80d2570 0x7f0aa40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0aadd9b640 1 -- 172.21.15.94:0/1490477916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ab80d2570 msgr2=0x7f0aa40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0aadd9b640 1 --2- 172.21.15.94:0/1490477916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ab80d2570 0x7f0aa40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0ab6d9d640 1 --2- 172.21.15.94:0/1490477916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0aa4004d50 0x7f0aa4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0aadd9b640 1 -- 172.21.15.94:0/1490477916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0aa4004d50 msgr2=0x7f0aa4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0aadd9b640 1 --2- 172.21.15.94:0/1490477916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0aa4004d50 0x7f0aa4005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0aadd9b640 1 -- 172.21.15.94:0/1490477916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa4003530 con 0x7f0aa4003f40 2024-04-25T16:52:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0ab659c640 1 --2- 172.21.15.94:0/1490477916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ab80d2570 0x7f0aa40039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:36.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0aadd9b640 1 --2- 172.21.15.94:0/1490477916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4003f40 0x7f0aa4004810 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80cfe80 tx=0x7f0aa001cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766680b52d7fd6a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:36.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa000c0e0 con 0x7f0aa4003f40 2024-04-25T16:52:36.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa00a9020 con 0x7f0aa4003f40 2024-04-25T16:52:36.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0ab5d9b640 1 -- 172.21.15.94:0/1490477916 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0005b20 con 0x7f0aa4003f40 2024-04-25T16:52:36.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0abc954640 1 -- 172.21.15.94:0/1490477916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4003f40 msgr2=0x7f0aa4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0abc954640 1 --2- 172.21.15.94:0/1490477916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4003f40 0x7f0aa4004810 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80cfe80 tx=0x7f0aa001cbc0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0ab6d9d640 1 -- 172.21.15.94:0/1490477916 reap_dead start 2024-04-25T16:52:36.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0abc954640 1 -- 172.21.15.94:0/1490477916 shutdown_connections 2024-04-25T16:52:36.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0abc954640 1 -- 172.21.15.94:0/1490477916 >> 172.21.15.94:0/1490477916 conn(0x7f0ab80cfed0 msgr2=0x7f0ab80d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:36.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0abc954640 1 -- 172.21.15.94:0/1490477916 shutdown_connections 2024-04-25T16:52:36.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.511+0000 7f0abc954640 1 -- 172.21.15.94:0/1490477916 wait complete. 2024-04-25T16:52:36.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0abc954640 1 Processor -- start 2024-04-25T16:52:36.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0abc954640 1 -- start start 2024-04-25T16:52:36.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0abc954640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4003f40 0x7f0ab81fd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0abc954640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ab81f8a90 0x7f0ab8165900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0abc954640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab81f8ea0 0x7f0ab8165e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0abc954640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab80cf960 con 0x7f0aa4003f40 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0abc954640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab80cf7e0 con 0x7f0ab81f8ea0 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0abc954640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab8168530 con 0x7f0ab81f8a90 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0ab659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4003f40 0x7f0ab81fd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0ab659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4003f40 0x7f0ab81fd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59136/0 (socket says 172.21.15.94:59136) 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0ab659c640 1 -- 172.21.15.94:0/880746913 learned_addr learned my addr 172.21.15.94:0/880746913 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0ab659c640 1 -- 172.21.15.94:0/880746913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ab81f8a90 msgr2=0x7f0ab8165900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0ab659c640 1 --2- 172.21.15.94:0/880746913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ab81f8a90 0x7f0ab8165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0ab659c640 1 -- 172.21.15.94:0/880746913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab81f8ea0 msgr2=0x7f0ab8165e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0ab659c640 1 --2- 172.21.15.94:0/880746913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab81f8ea0 0x7f0ab8165e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0ab659c640 1 -- 172.21.15.94:0/880746913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab80c82e0 con 0x7f0aa4003f40 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0ab659c640 1 --2- 172.21.15.94:0/880746913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4003f40 0x7f0ab81fd260 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009fd0 tx=0x7f0aa8010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:36.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0aaf7fe640 1 -- 172.21.15.94:0/880746913 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa800f040 con 0x7f0aa4003f40 2024-04-25T16:52:36.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0aaf7fe640 1 -- 172.21.15.94:0/880746913 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa800eae0 con 0x7f0aa4003f40 2024-04-25T16:52:36.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0aaf7fe640 1 -- 172.21.15.94:0/880746913 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa801c400 con 0x7f0aa4003f40 2024-04-25T16:52:36.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0abc954640 1 -- 172.21.15.94:0/880746913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab80c8000 con 0x7f0aa4003f40 2024-04-25T16:52:36.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0abc954640 1 -- 172.21.15.94:0/880746913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab806e040 con 0x7f0aa4003f40 2024-04-25T16:52:36.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.515+0000 7f0abc954640 1 -- 172.21.15.94:0/880746913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab806a730 con 0x7f0aa4003f40 2024-04-25T16:52:36.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.519+0000 7f0aaf7fe640 1 -- 172.21.15.94:0/880746913 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0aa80143f0 con 0x7f0aa4003f40 2024-04-25T16:52:36.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.519+0000 7f0aaf7fe640 1 --2- 172.21.15.94:0/880746913 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a9806f400 0x7f0a98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.519+0000 7f0aaf7fe640 1 -- 172.21.15.94:0/880746913 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f0aa808fbb0 con 0x7f0aa4003f40 2024-04-25T16:52:36.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.519+0000 7f0aadd9b640 1 --2- 172.21.15.94:0/880746913 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a9806f400 0x7f0a98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.519+0000 7f0aadd9b640 1 --2- 172.21.15.94:0/880746913 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a9806f400 0x7f0a98071910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0001060 tx=0x7f0aa0000ff0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.523+0000 7f0aaf7fe640 1 -- 172.21.15.94:0/880746913 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ab80c82e0 con 0x7f0aa4003f40 2024-04-25T16:52:36.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.723+0000 7f0abc954640 1 -- 172.21.15.94:0/880746913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f0ab81561e0 con 0x7f0aa4003f40 2024-04-25T16:52:36.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.723+0000 7f0aaf7fe640 1 -- 172.21.15.94:0/880746913 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1013 (secure 0 0 0) 0x7f0aa8018030 con 0x7f0aa4003f40 2024-04-25T16:52:36.723 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:36.723 INFO:teuthology.orchestra.run.smithi094.stdout:{"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":127,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":40,"num_osds":8,"num_up_osds":8,"osd_up_since":1714063840,"num_in_osds":8,"osd_in_since":1714063831,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":264474624,"bytes_avail":772829638656,"bytes_total":773094113280,"write_bytes_sec":1791,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5442}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-25T16:51:59.684427+0000","services":{}},"progress_events":{}} 2024-04-25T16:52:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.727+0000 7f0abc954640 1 -- 172.21.15.94:0/880746913 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a9806f400 msgr2=0x7f0a98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.727+0000 7f0abc954640 1 --2- 172.21.15.94:0/880746913 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a9806f400 0x7f0a98071910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0001060 tx=0x7f0aa0000ff0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.727+0000 7f0abc954640 1 -- 172.21.15.94:0/880746913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4003f40 msgr2=0x7f0ab81fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.727+0000 7f0abc954640 1 --2- 172.21.15.94:0/880746913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4003f40 0x7f0ab81fd260 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009fd0 tx=0x7f0aa8010d60 comp rx=0 tx=0).stop 2024-04-25T16:52:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.727+0000 7f0abc954640 1 -- 172.21.15.94:0/880746913 shutdown_connections 2024-04-25T16:52:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.727+0000 7f0abc954640 1 --2- 172.21.15.94:0/880746913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab81f8ea0 0x7f0ab8165e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.727+0000 7f0abc954640 1 --2- 172.21.15.94:0/880746913 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f0a9806f400 0x7f0a98071910 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.727+0000 7f0abc954640 1 --2- 172.21.15.94:0/880746913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ab81f8a90 0x7f0ab8165900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.727+0000 7f0abc954640 1 --2- 172.21.15.94:0/880746913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4003f40 0x7f0ab81fd260 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.727+0000 7f0abc954640 1 -- 172.21.15.94:0/880746913 >> 172.21.15.94:0/880746913 conn(0x7f0ab80cfed0 msgr2=0x7f0ab814bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:36.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.727+0000 7f0abc954640 1 -- 172.21.15.94:0/880746913 shutdown_connections 2024-04-25T16:52:36.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.727+0000 7f0abc954640 1 -- 172.21.15.94:0/880746913 wait complete. 2024-04-25T16:52:36.744 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T16:52:36.744 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:52:36.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4ba6a3d640 1 Processor -- start 2024-04-25T16:52:36.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4ba6a3d640 1 -- start start 2024-04-25T16:52:36.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4ba6a3d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4ba00b5440 0x7f4ba00b5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4ba6a3d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ba0160430 0x7f4ba0160840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4ba6a3d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ba0160d80 con 0x7f4ba00d1c90 2024-04-25T16:52:36.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4ba6a3d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4ba00d0be0 con 0x7f4ba00b5440 2024-04-25T16:52:36.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4ba6a3d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4ba006f120 con 0x7f4ba0160430 2024-04-25T16:52:36.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4b9f7fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4ba00b5440 0x7f4ba00b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4ba4fda640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ba0160430 0x7f4ba0160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4b9f7fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4ba00b5440 0x7f4ba00b5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51792/0 (socket says 172.21.15.94:51792) 2024-04-25T16:52:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4b9f7fe640 1 -- 172.21.15.94:0/859104818 learned_addr learned my addr 172.21.15.94:0/859104818 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4ba4fda640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ba0160430 0x7f4ba0160840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:55278/0 (socket says 172.21.15.94:55278) 2024-04-25T16:52:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4b9ffff640 1 --1- 172.21.15.94:0/859104818 >> v1:172.21.15.138:6789/0 conn(0x7f4ba00d1c90 0x7f4ba00d1070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:52710/0 (socket says 172.21.15.94:52710) 2024-04-25T16:52:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4ba4fda640 1 -- 172.21.15.94:0/859104818 >> v2:172.21.15.138:3300/0 conn(0x7f4ba00b5440 msgr2=0x7f4ba00b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4ba4fda640 1 --2- 172.21.15.94:0/859104818 >> v2:172.21.15.138:3300/0 conn(0x7f4ba00b5440 0x7f4ba00b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4ba4fda640 1 -- 172.21.15.94:0/859104818 >> v1:172.21.15.138:6789/0 conn(0x7f4ba00d1c90 legacy=0x7f4ba00d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.883+0000 7f4ba4fda640 1 -- 172.21.15.94:0/859104818 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba006f400 con 0x7f4ba0160430 2024-04-25T16:52:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.887+0000 7f4ba4fda640 1 --2- 172.21.15.94:0/859104818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ba0160430 0x7f4ba0160840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009d90 tx=0x7f4b9801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=371b248551899aea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.887+0000 7f4b9effd640 1 -- 172.21.15.94:0/859104818 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b980b2070 con 0x7f4ba0160430 2024-04-25T16:52:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.887+0000 7f4b9effd640 1 -- 172.21.15.94:0/859104818 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b980a5900 con 0x7f4ba0160430 2024-04-25T16:52:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.887+0000 7f4b9effd640 1 -- 172.21.15.94:0/859104818 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b980a5cd0 con 0x7f4ba0160430 2024-04-25T16:52:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.887+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/859104818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ba0160430 msgr2=0x7f4ba0160840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.887+0000 7f4ba6a3d640 1 --2- 172.21.15.94:0/859104818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ba0160430 0x7f4ba0160840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009d90 tx=0x7f4b9801c970 comp rx=0 tx=0).stop 2024-04-25T16:52:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.887+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/859104818 shutdown_connections 2024-04-25T16:52:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.887+0000 7f4ba6a3d640 1 --2- 172.21.15.94:0/859104818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ba0160430 0x7f4ba0160840 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.887+0000 7f4ba6a3d640 1 --2- 172.21.15.94:0/859104818 >> v2:172.21.15.138:3300/0 conn(0x7f4ba00b5440 0x7f4ba00b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.887+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/859104818 >> 172.21.15.94:0/859104818 conn(0x7f4ba0071150 msgr2=0x7f4ba0071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.887+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/859104818 shutdown_connections 2024-04-25T16:52:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.887+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/859104818 wait complete. 2024-04-25T16:52:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4ba6a3d640 1 Processor -- start 2024-04-25T16:52:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4ba6a3d640 1 -- start start 2024-04-25T16:52:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4ba6a3d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ba00b5440 0x7f4ba0144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4ba6a3d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ba00d1c90 0x7f4ba01527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4ba6a3d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ba0160430 0x7f4ba0152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4ba6a3d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba00d1780 con 0x7f4ba00b5440 2024-04-25T16:52:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4ba6a3d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba00d20a0 con 0x7f4ba0160430 2024-04-25T16:52:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4ba6a3d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4ba00d1900 con 0x7f4ba00d1c90 2024-04-25T16:52:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4b9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ba00b5440 0x7f4ba0144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4b9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ba00b5440 0x7f4ba0144680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59146/0 (socket says 172.21.15.94:59146) 2024-04-25T16:52:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4b9ffff640 1 -- 172.21.15.94:0/532881238 learned_addr learned my addr 172.21.15.94:0/532881238 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4b9f7fe640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ba00d1c90 0x7f4ba01527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4ba4fda640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ba0160430 0x7f4ba0152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4b9ffff640 1 -- 172.21.15.94:0/532881238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ba00d1c90 msgr2=0x7f4ba01527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4b9ffff640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ba00d1c90 0x7f4ba01527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4b9ffff640 1 -- 172.21.15.94:0/532881238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ba0160430 msgr2=0x7f4ba0152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4b9ffff640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ba0160430 0x7f4ba0152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4b9ffff640 1 -- 172.21.15.94:0/532881238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba0144bc0 con 0x7f4ba00b5440 2024-04-25T16:52:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4ba4fda640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ba0160430 0x7f4ba0152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4b9ffff640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ba00b5440 0x7f4ba0144680 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000c340 tx=0x7f4b9000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4b9cff9640 1 -- 172.21.15.94:0/532881238 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b900074e0 con 0x7f4ba00b5440 2024-04-25T16:52:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4b9cff9640 1 -- 172.21.15.94:0/532881238 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b90007680 con 0x7f4ba00b5440 2024-04-25T16:52:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/532881238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba01460b0 con 0x7f4ba00b5440 2024-04-25T16:52:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4b9cff9640 1 -- 172.21.15.94:0/532881238 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90004a30 con 0x7f4ba00b5440 2024-04-25T16:52:36.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/532881238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba0146670 con 0x7f4ba00b5440 2024-04-25T16:52:36.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.891+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/532881238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba006a730 con 0x7f4ba00b5440 2024-04-25T16:52:36.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.895+0000 7f4b9cff9640 1 -- 172.21.15.94:0/532881238 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4b9001a020 con 0x7f4ba00b5440 2024-04-25T16:52:36.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.895+0000 7f4b9cff9640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4b8c06f400 0x7f4b8c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.895+0000 7f4b9cff9640 1 -- 172.21.15.94:0/532881238 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f4b9008ec00 con 0x7f4ba00b5440 2024-04-25T16:52:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.895+0000 7f4b9f7fe640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4b8c06f400 0x7f4b8c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.895+0000 7f4b9f7fe640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4b8c06f400 0x7f4b8c071910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00ceb40 tx=0x7f4b94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:36.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:36.899+0000 7f4b9cff9640 1 -- 172.21.15.94:0/532881238 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4b90095080 con 0x7f4ba00b5440 2024-04-25T16:52:37.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.043+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/532881238 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ba0145e60 con 0x7f4b8c06f400 2024-04-25T16:52:37.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.063+0000 7f4b9cff9640 1 -- 172.21.15.94:0/532881238 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251881 (secure 0 0 0) 0x7f4ba0145e60 con 0x7f4b8c06f400 2024-04-25T16:52:37.063 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:52:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.067+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/532881238 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4b8c06f400 msgr2=0x7f4b8c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.067+0000 7f4ba6a3d640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4b8c06f400 0x7f4b8c071910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00ceb40 tx=0x7f4b94008040 comp rx=0 tx=0).stop 2024-04-25T16:52:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.067+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/532881238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ba00b5440 msgr2=0x7f4ba0144680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.067+0000 7f4ba6a3d640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ba00b5440 0x7f4ba0144680 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000c340 tx=0x7f4b9000c800 comp rx=0 tx=0).stop 2024-04-25T16:52:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.067+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/532881238 shutdown_connections 2024-04-25T16:52:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.067+0000 7f4ba6a3d640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ba0160430 0x7f4ba0152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.067+0000 7f4ba6a3d640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4b8c06f400 0x7f4b8c071910 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.067+0000 7f4ba6a3d640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ba00d1c90 0x7f4ba01527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.067+0000 7f4ba6a3d640 1 --2- 172.21.15.94:0/532881238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ba00b5440 0x7f4ba0144680 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.067+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/532881238 >> 172.21.15.94:0/532881238 conn(0x7f4ba0071150 msgr2=0x7f4ba00738f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.067+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/532881238 shutdown_connections 2024-04-25T16:52:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:37.067+0000 7f4ba6a3d640 1 -- 172.21.15.94:0/532881238 wait complete. 2024-04-25T16:52:37.082 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-25T16:52:35.877728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258320,"kb_used_data":4708,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716400,"statfs":{"total":773094113280,"available":772829593600,"internally_reserved":0,"allocated":4820992,"data_stored":2737575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.100997"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873724+0000","last_change":"2024-04-25T16:52:26.825748+0000","last_active":"2024-04-25T16:52:34.873724+0000","last_peered":"2024-04-25T16:52:34.873724+0000","last_clean":"2024-04-25T16:52:34.873724+0000","last_became_active":"2024-04-25T16:52:26.825333+0000","last_became_peered":"2024-04-25T16:52:26.825333+0000","last_unstale":"2024-04-25T16:52:34.873724+0000","last_undegraded":"2024-04-25T16:52:34.873724+0000","last_fullsized":"2024-04-25T16:52:34.873724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:19.084563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873506+0000","last_change":"2024-04-25T16:52:26.795964+0000","last_active":"2024-04-25T16:52:34.873506+0000","last_peered":"2024-04-25T16:52:34.873506+0000","last_clean":"2024-04-25T16:52:34.873506+0000","last_became_active":"2024-04-25T16:52:26.794884+0000","last_became_peered":"2024-04-25T16:52:26.794884+0000","last_unstale":"2024-04-25T16:52:34.873506+0000","last_undegraded":"2024-04-25T16:52:34.873506+0000","last_fullsized":"2024-04-25T16:52:34.873506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878233+0000","last_change":"2024-04-25T16:52:26.802075+0000","last_active":"2024-04-25T16:52:34.878233+0000","last_peered":"2024-04-25T16:52:34.878233+0000","last_clean":"2024-04-25T16:52:34.878233+0000","last_became_active":"2024-04-25T16:52:26.801203+0000","last_became_peered":"2024-04-25T16:52:26.801203+0000","last_unstale":"2024-04-25T16:52:34.878233+0000","last_undegraded":"2024-04-25T16:52:34.878233+0000","last_fullsized":"2024-04-25T16:52:34.878233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782213+0000","last_change":"2024-04-25T16:52:26.793541+0000","last_active":"2024-04-25T16:52:27.782213+0000","last_peered":"2024-04-25T16:52:27.782213+0000","last_clean":"2024-04-25T16:52:27.782213+0000","last_became_active":"2024-04-25T16:52:26.789101+0000","last_became_peered":"2024-04-25T16:52:26.789101+0000","last_unstale":"2024-04-25T16:52:27.782213+0000","last_undegraded":"2024-04-25T16:52:27.782213+0000","last_fullsized":"2024-04-25T16:52:27.782213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:38.013058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781321+0000","last_change":"2024-04-25T16:52:26.784467+0000","last_active":"2024-04-25T16:52:27.781321+0000","last_peered":"2024-04-25T16:52:27.781321+0000","last_clean":"2024-04-25T16:52:27.781321+0000","last_became_active":"2024-04-25T16:52:26.783673+0000","last_became_peered":"2024-04-25T16:52:26.783673+0000","last_unstale":"2024-04-25T16:52:27.781321+0000","last_undegraded":"2024-04-25T16:52:27.781321+0000","last_fullsized":"2024-04-25T16:52:27.781321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:36.583897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831274+0000","last_change":"2024-04-25T16:52:26.823413+0000","last_active":"2024-04-25T16:52:27.831274+0000","last_peered":"2024-04-25T16:52:27.831274+0000","last_clean":"2024-04-25T16:52:27.831274+0000","last_became_active":"2024-04-25T16:52:26.823059+0000","last_became_peered":"2024-04-25T16:52:26.823059+0000","last_unstale":"2024-04-25T16:52:27.831274+0000","last_undegraded":"2024-04-25T16:52:27.831274+0000","last_fullsized":"2024-04-25T16:52:27.831274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.876836+0000","last_change":"2024-04-25T16:52:26.808661+0000","last_active":"2024-04-25T16:52:34.876836+0000","last_peered":"2024-04-25T16:52:34.876836+0000","last_clean":"2024-04-25T16:52:34.876836+0000","last_became_active":"2024-04-25T16:52:26.805296+0000","last_became_peered":"2024-04-25T16:52:26.805296+0000","last_unstale":"2024-04-25T16:52:34.876836+0000","last_undegraded":"2024-04-25T16:52:34.876836+0000","last_fullsized":"2024-04-25T16:52:34.876836+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"peering","last_fresh":"2024-04-25T16:52:30.709501+0000","last_change":"2024-04-25T16:52:34.873590+0000","last_active":"2024-04-25T16:52:34.873590+0000","last_peered":"2024-04-25T16:52:30.709501+0000","last_clean":"2024-04-25T16:52:30.709501+0000","last_became_active":"2024-04-25T16:52:26.820528+0000","last_became_peered":"2024-04-25T16:52:26.820528+0000","last_unstale":"2024-04-25T16:52:30.709501+0000","last_undegraded":"2024-04-25T16:52:30.709501+0000","last_fullsized":"2024-04-25T16:52:30.709501+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.879080+0000","last_change":"2024-04-25T16:52:26.803363+0000","last_active":"2024-04-25T16:52:34.879080+0000","last_peered":"2024-04-25T16:52:34.879080+0000","last_clean":"2024-04-25T16:52:34.879080+0000","last_became_active":"2024-04-25T16:52:26.802539+0000","last_became_peered":"2024-04-25T16:52:26.802539+0000","last_unstale":"2024-04-25T16:52:34.879080+0000","last_undegraded":"2024-04-25T16:52:34.879080+0000","last_fullsized":"2024-04-25T16:52:34.879080+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877890+0000","last_change":"2024-04-25T16:52:26.842728+0000","last_active":"2024-04-25T16:52:34.877890+0000","last_peered":"2024-04-25T16:52:34.877890+0000","last_clean":"2024-04-25T16:52:34.877890+0000","last_became_active":"2024-04-25T16:52:26.842041+0000","last_became_peered":"2024-04-25T16:52:26.842041+0000","last_unstale":"2024-04-25T16:52:34.877890+0000","last_undegraded":"2024-04-25T16:52:34.877890+0000","last_fullsized":"2024-04-25T16:52:34.877890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:33.962706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877607+0000","last_change":"2024-04-25T16:52:26.819329+0000","last_active":"2024-04-25T16:52:34.877607+0000","last_peered":"2024-04-25T16:52:34.877607+0000","last_clean":"2024-04-25T16:52:34.877607+0000","last_became_active":"2024-04-25T16:52:26.818528+0000","last_became_peered":"2024-04-25T16:52:26.818528+0000","last_unstale":"2024-04-25T16:52:34.877607+0000","last_undegraded":"2024-04-25T16:52:34.877607+0000","last_fullsized":"2024-04-25T16:52:34.877607+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878111+0000","last_change":"2024-04-25T16:52:26.795942+0000","last_active":"2024-04-25T16:52:34.878111+0000","last_peered":"2024-04-25T16:52:34.878111+0000","last_clean":"2024-04-25T16:52:34.878111+0000","last_became_active":"2024-04-25T16:52:26.795124+0000","last_became_peered":"2024-04-25T16:52:26.795124+0000","last_unstale":"2024-04-25T16:52:34.878111+0000","last_undegraded":"2024-04-25T16:52:34.878111+0000","last_fullsized":"2024-04-25T16:52:34.878111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877026+0000","last_change":"2024-04-25T16:52:26.814749+0000","last_active":"2024-04-25T16:52:34.877026+0000","last_peered":"2024-04-25T16:52:34.877026+0000","last_clean":"2024-04-25T16:52:34.877026+0000","last_became_active":"2024-04-25T16:52:26.814020+0000","last_became_peered":"2024-04-25T16:52:26.814020+0000","last_unstale":"2024-04-25T16:52:34.877026+0000","last_undegraded":"2024-04-25T16:52:34.877026+0000","last_fullsized":"2024-04-25T16:52:34.877026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871890+0000","last_change":"2024-04-25T16:52:26.813118+0000","last_active":"2024-04-25T16:52:34.871890+0000","last_peered":"2024-04-25T16:52:34.871890+0000","last_clean":"2024-04-25T16:52:34.871890+0000","last_became_active":"2024-04-25T16:52:26.812400+0000","last_became_peered":"2024-04-25T16:52:26.812400+0000","last_unstale":"2024-04-25T16:52:34.871890+0000","last_undegraded":"2024-04-25T16:52:34.871890+0000","last_fullsized":"2024-04-25T16:52:34.871890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:16.169654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877027+0000","last_change":"2024-04-25T16:52:26.795864+0000","last_active":"2024-04-25T16:52:34.877027+0000","last_peered":"2024-04-25T16:52:34.877027+0000","last_clean":"2024-04-25T16:52:34.877027+0000","last_became_active":"2024-04-25T16:52:26.794738+0000","last_became_peered":"2024-04-25T16:52:26.794738+0000","last_unstale":"2024-04-25T16:52:34.877027+0000","last_undegraded":"2024-04-25T16:52:34.877027+0000","last_fullsized":"2024-04-25T16:52:34.877027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340722+0000","last_change":"2024-04-25T16:52:26.836640+0000","last_active":"2024-04-25T16:52:35.340722+0000","last_peered":"2024-04-25T16:52:35.340722+0000","last_clean":"2024-04-25T16:52:35.340722+0000","last_became_active":"2024-04-25T16:52:26.833071+0000","last_became_peered":"2024-04-25T16:52:26.833071+0000","last_unstale":"2024-04-25T16:52:35.340722+0000","last_undegraded":"2024-04-25T16:52:35.340722+0000","last_fullsized":"2024-04-25T16:52:35.340722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341191+0000","last_change":"2024-04-25T16:52:26.841207+0000","last_active":"2024-04-25T16:52:35.341191+0000","last_peered":"2024-04-25T16:52:35.341191+0000","last_clean":"2024-04-25T16:52:35.341191+0000","last_became_active":"2024-04-25T16:52:26.838750+0000","last_became_peered":"2024-04-25T16:52:26.838750+0000","last_unstale":"2024-04-25T16:52:35.341191+0000","last_undegraded":"2024-04-25T16:52:35.341191+0000","last_fullsized":"2024-04-25T16:52:35.341191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875046+0000","last_change":"2024-04-25T16:52:26.795939+0000","last_active":"2024-04-25T16:52:34.875046+0000","last_peered":"2024-04-25T16:52:34.875046+0000","last_clean":"2024-04-25T16:52:34.875046+0000","last_became_active":"2024-04-25T16:52:26.795084+0000","last_became_peered":"2024-04-25T16:52:26.795084+0000","last_unstale":"2024-04-25T16:52:34.875046+0000","last_undegraded":"2024-04-25T16:52:34.875046+0000","last_fullsized":"2024-04-25T16:52:34.875046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781139+0000","last_change":"2024-04-25T16:52:26.818025+0000","last_active":"2024-04-25T16:52:27.781139+0000","last_peered":"2024-04-25T16:52:27.781139+0000","last_clean":"2024-04-25T16:52:27.781139+0000","last_became_active":"2024-04-25T16:52:26.817652+0000","last_became_peered":"2024-04-25T16:52:26.817652+0000","last_unstale":"2024-04-25T16:52:27.781139+0000","last_undegraded":"2024-04-25T16:52:27.781139+0000","last_fullsized":"2024-04-25T16:52:27.781139+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:46.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872811+0000","last_change":"2024-04-25T16:52:26.796384+0000","last_active":"2024-04-25T16:52:34.872811+0000","last_peered":"2024-04-25T16:52:34.872811+0000","last_clean":"2024-04-25T16:52:34.872811+0000","last_became_active":"2024-04-25T16:52:26.795507+0000","last_became_peered":"2024-04-25T16:52:26.795507+0000","last_unstale":"2024-04-25T16:52:34.872811+0000","last_undegraded":"2024-04-25T16:52:34.872811+0000","last_fullsized":"2024-04-25T16:52:34.872811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877104+0000","last_change":"2024-04-25T16:52:26.809988+0000","last_active":"2024-04-25T16:52:34.877104+0000","last_peered":"2024-04-25T16:52:34.877104+0000","last_clean":"2024-04-25T16:52:34.877104+0000","last_became_active":"2024-04-25T16:52:26.809107+0000","last_became_peered":"2024-04-25T16:52:26.809107+0000","last_unstale":"2024-04-25T16:52:34.877104+0000","last_undegraded":"2024-04-25T16:52:34.877104+0000","last_fullsized":"2024-04-25T16:52:34.877104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872865+0000","last_change":"2024-04-25T16:52:26.784466+0000","last_active":"2024-04-25T16:52:34.872865+0000","last_peered":"2024-04-25T16:52:34.872865+0000","last_clean":"2024-04-25T16:52:34.872865+0000","last_became_active":"2024-04-25T16:52:26.783550+0000","last_became_peered":"2024-04-25T16:52:26.783550+0000","last_unstale":"2024-04-25T16:52:34.872865+0000","last_undegraded":"2024-04-25T16:52:34.872865+0000","last_fullsized":"2024-04-25T16:52:34.872865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340668+0000","last_change":"2024-04-25T16:52:26.832452+0000","last_active":"2024-04-25T16:52:35.340668+0000","last_peered":"2024-04-25T16:52:35.340668+0000","last_clean":"2024-04-25T16:52:35.340668+0000","last_became_active":"2024-04-25T16:52:26.831753+0000","last_became_peered":"2024-04-25T16:52:26.831753+0000","last_unstale":"2024-04-25T16:52:35.340668+0000","last_undegraded":"2024-04-25T16:52:35.340668+0000","last_fullsized":"2024-04-25T16:52:35.340668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780752+0000","last_change":"2024-04-25T16:52:26.805633+0000","last_active":"2024-04-25T16:52:27.780752+0000","last_peered":"2024-04-25T16:52:27.780752+0000","last_clean":"2024-04-25T16:52:27.780752+0000","last_became_active":"2024-04-25T16:52:26.804789+0000","last_became_peered":"2024-04-25T16:52:26.804789+0000","last_unstale":"2024-04-25T16:52:27.780752+0000","last_undegraded":"2024-04-25T16:52:27.780752+0000","last_fullsized":"2024-04-25T16:52:27.780752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:08.931465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875174+0000","last_change":"2024-04-25T16:52:26.798693+0000","last_active":"2024-04-25T16:52:34.875174+0000","last_peered":"2024-04-25T16:52:34.875174+0000","last_clean":"2024-04-25T16:52:34.875174+0000","last_became_active":"2024-04-25T16:52:26.797852+0000","last_became_peered":"2024-04-25T16:52:26.797852+0000","last_unstale":"2024-04-25T16:52:34.875174+0000","last_undegraded":"2024-04-25T16:52:34.875174+0000","last_fullsized":"2024-04-25T16:52:34.875174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831376+0000","last_change":"2024-04-25T16:52:26.819630+0000","last_active":"2024-04-25T16:52:27.831376+0000","last_peered":"2024-04-25T16:52:27.831376+0000","last_clean":"2024-04-25T16:52:27.831376+0000","last_became_active":"2024-04-25T16:52:26.819274+0000","last_became_peered":"2024-04-25T16:52:26.819274+0000","last_unstale":"2024-04-25T16:52:27.831376+0000","last_undegraded":"2024-04-25T16:52:27.831376+0000","last_fullsized":"2024-04-25T16:52:27.831376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873718+0000","last_change":"2024-04-25T16:52:26.791433+0000","last_active":"2024-04-25T16:52:34.873718+0000","last_peered":"2024-04-25T16:52:34.873718+0000","last_clean":"2024-04-25T16:52:34.873718+0000","last_became_active":"2024-04-25T16:52:26.790483+0000","last_became_peered":"2024-04-25T16:52:26.790483+0000","last_unstale":"2024-04-25T16:52:34.873718+0000","last_undegraded":"2024-04-25T16:52:34.873718+0000","last_fullsized":"2024-04-25T16:52:34.873718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877762+0000","last_change":"2024-04-25T16:52:26.804860+0000","last_active":"2024-04-25T16:52:34.877762+0000","last_peered":"2024-04-25T16:52:34.877762+0000","last_clean":"2024-04-25T16:52:34.877762+0000","last_became_active":"2024-04-25T16:52:26.804065+0000","last_became_peered":"2024-04-25T16:52:26.804065+0000","last_unstale":"2024-04-25T16:52:34.877762+0000","last_undegraded":"2024-04-25T16:52:34.877762+0000","last_fullsized":"2024-04-25T16:52:34.877762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873009+0000","last_change":"2024-04-25T16:52:26.806705+0000","last_active":"2024-04-25T16:52:34.873009+0000","last_peered":"2024-04-25T16:52:34.873009+0000","last_clean":"2024-04-25T16:52:34.873009+0000","last_became_active":"2024-04-25T16:52:26.805889+0000","last_became_peered":"2024-04-25T16:52:26.805889+0000","last_unstale":"2024-04-25T16:52:34.873009+0000","last_undegraded":"2024-04-25T16:52:34.873009+0000","last_fullsized":"2024-04-25T16:52:34.873009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874258+0000","last_change":"2024-04-25T16:52:26.798903+0000","last_active":"2024-04-25T16:52:34.874258+0000","last_peered":"2024-04-25T16:52:34.874258+0000","last_clean":"2024-04-25T16:52:34.874258+0000","last_became_active":"2024-04-25T16:52:26.798011+0000","last_became_peered":"2024-04-25T16:52:26.798011+0000","last_unstale":"2024-04-25T16:52:34.874258+0000","last_undegraded":"2024-04-25T16:52:34.874258+0000","last_fullsized":"2024-04-25T16:52:34.874258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340837+0000","last_change":"2024-04-25T16:52:26.841037+0000","last_active":"2024-04-25T16:52:35.340837+0000","last_peered":"2024-04-25T16:52:35.340837+0000","last_clean":"2024-04-25T16:52:35.340837+0000","last_became_active":"2024-04-25T16:52:26.838194+0000","last_became_peered":"2024-04-25T16:52:26.838194+0000","last_unstale":"2024-04-25T16:52:35.340837+0000","last_undegraded":"2024-04-25T16:52:35.340837+0000","last_fullsized":"2024-04-25T16:52:35.340837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830883+0000","last_change":"2024-04-25T16:52:26.813243+0000","last_active":"2024-04-25T16:52:27.830883+0000","last_peered":"2024-04-25T16:52:27.830883+0000","last_clean":"2024-04-25T16:52:27.830883+0000","last_became_active":"2024-04-25T16:52:26.811807+0000","last_became_peered":"2024-04-25T16:52:26.811807+0000","last_unstale":"2024-04-25T16:52:27.830883+0000","last_undegraded":"2024-04-25T16:52:27.830883+0000","last_fullsized":"2024-04-25T16:52:27.830883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:42.553053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341094+0000","last_change":"2024-04-25T16:52:26.841017+0000","last_active":"2024-04-25T16:52:35.341094+0000","last_peered":"2024-04-25T16:52:35.341094+0000","last_clean":"2024-04-25T16:52:35.341094+0000","last_became_active":"2024-04-25T16:52:26.837543+0000","last_became_peered":"2024-04-25T16:52:26.837543+0000","last_unstale":"2024-04-25T16:52:35.341094+0000","last_undegraded":"2024-04-25T16:52:35.341094+0000","last_fullsized":"2024-04-25T16:52:35.341094+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875394+0000","last_change":"2024-04-25T16:52:25.818069+0000","last_active":"2024-04-25T16:52:34.875394+0000","last_peered":"2024-04-25T16:52:34.875394+0000","last_clean":"2024-04-25T16:52:34.875394+0000","last_became_active":"2024-04-25T16:52:25.817362+0000","last_became_peered":"2024-04-25T16:52:25.817362+0000","last_unstale":"2024-04-25T16:52:34.875394+0000","last_undegraded":"2024-04-25T16:52:34.875394+0000","last_fullsized":"2024-04-25T16:52:34.875394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:08.488136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833310+0000","last_change":"2024-04-25T16:52:26.829008+0000","last_active":"2024-04-25T16:52:27.833310+0000","last_peered":"2024-04-25T16:52:27.833310+0000","last_clean":"2024-04-25T16:52:27.833310+0000","last_became_active":"2024-04-25T16:52:26.828706+0000","last_became_peered":"2024-04-25T16:52:26.828706+0000","last_unstale":"2024-04-25T16:52:27.833310+0000","last_undegraded":"2024-04-25T16:52:27.833310+0000","last_fullsized":"2024-04-25T16:52:27.833310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878876+0000","last_change":"2024-04-25T16:52:25.795967+0000","last_active":"2024-04-25T16:52:34.878876+0000","last_peered":"2024-04-25T16:52:34.878876+0000","last_clean":"2024-04-25T16:52:34.878876+0000","last_became_active":"2024-04-25T16:52:25.795695+0000","last_became_peered":"2024-04-25T16:52:25.795695+0000","last_unstale":"2024-04-25T16:52:34.878876+0000","last_undegraded":"2024-04-25T16:52:34.878876+0000","last_fullsized":"2024-04-25T16:52:34.878876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875243+0000","last_change":"2024-04-25T16:52:26.789430+0000","last_active":"2024-04-25T16:52:34.875243+0000","last_peered":"2024-04-25T16:52:34.875243+0000","last_clean":"2024-04-25T16:52:34.875243+0000","last_became_active":"2024-04-25T16:52:26.782243+0000","last_became_peered":"2024-04-25T16:52:26.782243+0000","last_unstale":"2024-04-25T16:52:34.875243+0000","last_undegraded":"2024-04-25T16:52:34.875243+0000","last_fullsized":"2024-04-25T16:52:34.875243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833428+0000","last_change":"2024-04-25T16:52:25.819162+0000","last_active":"2024-04-25T16:52:27.833428+0000","last_peered":"2024-04-25T16:52:27.833428+0000","last_clean":"2024-04-25T16:52:27.833428+0000","last_became_active":"2024-04-25T16:52:25.818859+0000","last_became_peered":"2024-04-25T16:52:25.818859+0000","last_unstale":"2024-04-25T16:52:27.833428+0000","last_undegraded":"2024-04-25T16:52:27.833428+0000","last_fullsized":"2024-04-25T16:52:27.833428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874082+0000","last_change":"2024-04-25T16:52:26.805376+0000","last_active":"2024-04-25T16:52:34.874082+0000","last_peered":"2024-04-25T16:52:34.874082+0000","last_clean":"2024-04-25T16:52:34.874082+0000","last_became_active":"2024-04-25T16:52:26.804542+0000","last_became_peered":"2024-04-25T16:52:26.804542+0000","last_unstale":"2024-04-25T16:52:34.874082+0000","last_undegraded":"2024-04-25T16:52:34.874082+0000","last_fullsized":"2024-04-25T16:52:34.874082+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.620224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874004+0000","last_change":"2024-04-25T16:52:25.815751+0000","last_active":"2024-04-25T16:52:34.874004+0000","last_peered":"2024-04-25T16:52:34.874004+0000","last_clean":"2024-04-25T16:52:34.874004+0000","last_became_active":"2024-04-25T16:52:25.814927+0000","last_became_peered":"2024-04-25T16:52:25.814927+0000","last_unstale":"2024-04-25T16:52:34.874004+0000","last_undegraded":"2024-04-25T16:52:34.874004+0000","last_fullsized":"2024-04-25T16:52:34.874004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877461+0000","last_change":"2024-04-25T16:52:26.814640+0000","last_active":"2024-04-25T16:52:34.877461+0000","last_peered":"2024-04-25T16:52:34.877461+0000","last_clean":"2024-04-25T16:52:34.877461+0000","last_became_active":"2024-04-25T16:52:26.814016+0000","last_became_peered":"2024-04-25T16:52:26.814016+0000","last_unstale":"2024-04-25T16:52:34.877461+0000","last_undegraded":"2024-04-25T16:52:34.877461+0000","last_fullsized":"2024-04-25T16:52:34.877461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877431+0000","last_change":"2024-04-25T16:52:25.805323+0000","last_active":"2024-04-25T16:52:34.877431+0000","last_peered":"2024-04-25T16:52:34.877431+0000","last_clean":"2024-04-25T16:52:34.877431+0000","last_became_active":"2024-04-25T16:52:25.796444+0000","last_became_peered":"2024-04-25T16:52:25.796444+0000","last_unstale":"2024-04-25T16:52:34.877431+0000","last_undegraded":"2024-04-25T16:52:34.877431+0000","last_fullsized":"2024-04-25T16:52:34.877431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:28.996351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877181+0000","last_change":"2024-04-25T16:52:26.821648+0000","last_active":"2024-04-25T16:52:34.877181+0000","last_peered":"2024-04-25T16:52:34.877181+0000","last_clean":"2024-04-25T16:52:34.877181+0000","last_became_active":"2024-04-25T16:52:26.820835+0000","last_became_peered":"2024-04-25T16:52:26.820835+0000","last_unstale":"2024-04-25T16:52:34.877181+0000","last_undegraded":"2024-04-25T16:52:34.877181+0000","last_fullsized":"2024-04-25T16:52:34.877181+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340828+0000","last_change":"2024-04-25T16:52:25.816250+0000","last_active":"2024-04-25T16:52:35.340828+0000","last_peered":"2024-04-25T16:52:35.340828+0000","last_clean":"2024-04-25T16:52:35.340828+0000","last_became_active":"2024-04-25T16:52:25.815656+0000","last_became_peered":"2024-04-25T16:52:25.815656+0000","last_unstale":"2024-04-25T16:52:35.340828+0000","last_undegraded":"2024-04-25T16:52:35.340828+0000","last_fullsized":"2024-04-25T16:52:35.340828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872461+0000","last_change":"2024-04-25T16:52:26.813425+0000","last_active":"2024-04-25T16:52:34.872461+0000","last_peered":"2024-04-25T16:52:34.872461+0000","last_clean":"2024-04-25T16:52:34.872461+0000","last_became_active":"2024-04-25T16:52:26.812713+0000","last_became_peered":"2024-04-25T16:52:26.812713+0000","last_unstale":"2024-04-25T16:52:34.872461+0000","last_undegraded":"2024-04-25T16:52:34.872461+0000","last_fullsized":"2024-04-25T16:52:34.872461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831072+0000","last_change":"2024-04-25T16:52:25.803265+0000","last_active":"2024-04-25T16:52:27.831072+0000","last_peered":"2024-04-25T16:52:27.831072+0000","last_clean":"2024-04-25T16:52:27.831072+0000","last_became_active":"2024-04-25T16:52:25.802617+0000","last_became_peered":"2024-04-25T16:52:25.802617+0000","last_unstale":"2024-04-25T16:52:27.831072+0000","last_undegraded":"2024-04-25T16:52:27.831072+0000","last_fullsized":"2024-04-25T16:52:27.831072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872313+0000","last_change":"2024-04-25T16:52:26.819267+0000","last_active":"2024-04-25T16:52:34.872313+0000","last_peered":"2024-04-25T16:52:34.872313+0000","last_clean":"2024-04-25T16:52:34.872313+0000","last_became_active":"2024-04-25T16:52:26.818740+0000","last_became_peered":"2024-04-25T16:52:26.818740+0000","last_unstale":"2024-04-25T16:52:34.872313+0000","last_undegraded":"2024-04-25T16:52:34.872313+0000","last_fullsized":"2024-04-25T16:52:34.872313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871955+0000","last_change":"2024-04-25T16:52:25.816663+0000","last_active":"2024-04-25T16:52:34.871955+0000","last_peered":"2024-04-25T16:52:34.871955+0000","last_clean":"2024-04-25T16:52:34.871955+0000","last_became_active":"2024-04-25T16:52:25.815420+0000","last_became_peered":"2024-04-25T16:52:25.815420+0000","last_unstale":"2024-04-25T16:52:34.871955+0000","last_undegraded":"2024-04-25T16:52:34.871955+0000","last_fullsized":"2024-04-25T16:52:34.871955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:30.288538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782313+0000","last_change":"2024-04-25T16:52:26.789927+0000","last_active":"2024-04-25T16:52:27.782313+0000","last_peered":"2024-04-25T16:52:27.782313+0000","last_clean":"2024-04-25T16:52:27.782313+0000","last_became_active":"2024-04-25T16:52:26.788014+0000","last_became_peered":"2024-04-25T16:52:26.788014+0000","last_unstale":"2024-04-25T16:52:27.782313+0000","last_undegraded":"2024-04-25T16:52:27.782313+0000","last_fullsized":"2024-04-25T16:52:27.782313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:47.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873885+0000","last_change":"2024-04-25T16:52:25.798974+0000","last_active":"2024-04-25T16:52:34.873885+0000","last_peered":"2024-04-25T16:52:34.873885+0000","last_clean":"2024-04-25T16:52:34.873885+0000","last_became_active":"2024-04-25T16:52:25.798314+0000","last_became_peered":"2024-04-25T16:52:25.798314+0000","last_unstale":"2024-04-25T16:52:34.873885+0000","last_undegraded":"2024-04-25T16:52:34.873885+0000","last_fullsized":"2024-04-25T16:52:34.873885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.348530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341450+0000","last_change":"2024-04-25T16:52:26.842485+0000","last_active":"2024-04-25T16:52:35.341450+0000","last_peered":"2024-04-25T16:52:35.341450+0000","last_clean":"2024-04-25T16:52:35.341450+0000","last_became_active":"2024-04-25T16:52:26.838813+0000","last_became_peered":"2024-04-25T16:52:26.838813+0000","last_unstale":"2024-04-25T16:52:35.341450+0000","last_undegraded":"2024-04-25T16:52:35.341450+0000","last_fullsized":"2024-04-25T16:52:35.341450+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878521+0000","last_change":"2024-04-25T16:52:25.803289+0000","last_active":"2024-04-25T16:52:34.878521+0000","last_peered":"2024-04-25T16:52:34.878521+0000","last_clean":"2024-04-25T16:52:34.878521+0000","last_became_active":"2024-04-25T16:52:25.801415+0000","last_became_peered":"2024-04-25T16:52:25.801415+0000","last_unstale":"2024-04-25T16:52:34.878521+0000","last_undegraded":"2024-04-25T16:52:34.878521+0000","last_fullsized":"2024-04-25T16:52:34.878521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:45.310089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878182+0000","last_change":"2024-04-25T16:52:26.840379+0000","last_active":"2024-04-25T16:52:34.878182+0000","last_peered":"2024-04-25T16:52:34.878182+0000","last_clean":"2024-04-25T16:52:34.878182+0000","last_became_active":"2024-04-25T16:52:26.840062+0000","last_became_peered":"2024-04-25T16:52:26.840062+0000","last_unstale":"2024-04-25T16:52:34.878182+0000","last_undegraded":"2024-04-25T16:52:34.878182+0000","last_fullsized":"2024-04-25T16:52:34.878182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878024+0000","last_change":"2024-04-25T16:52:25.809391+0000","last_active":"2024-04-25T16:52:34.878024+0000","last_peered":"2024-04-25T16:52:34.878024+0000","last_clean":"2024-04-25T16:52:34.878024+0000","last_became_active":"2024-04-25T16:52:25.808455+0000","last_became_peered":"2024-04-25T16:52:25.808455+0000","last_unstale":"2024-04-25T16:52:34.878024+0000","last_undegraded":"2024-04-25T16:52:34.878024+0000","last_fullsized":"2024-04-25T16:52:34.878024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872072+0000","last_change":"2024-04-25T16:52:26.782954+0000","last_active":"2024-04-25T16:52:34.872072+0000","last_peered":"2024-04-25T16:52:34.872072+0000","last_clean":"2024-04-25T16:52:34.872072+0000","last_became_active":"2024-04-25T16:52:26.782108+0000","last_became_peered":"2024-04-25T16:52:26.782108+0000","last_unstale":"2024-04-25T16:52:34.872072+0000","last_undegraded":"2024-04-25T16:52:34.872072+0000","last_fullsized":"2024-04-25T16:52:34.872072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873363+0000","last_change":"2024-04-25T16:52:25.795020+0000","last_active":"2024-04-25T16:52:34.873363+0000","last_peered":"2024-04-25T16:52:34.873363+0000","last_clean":"2024-04-25T16:52:34.873363+0000","last_became_active":"2024-04-25T16:52:25.794633+0000","last_became_peered":"2024-04-25T16:52:25.794633+0000","last_unstale":"2024-04-25T16:52:34.873363+0000","last_undegraded":"2024-04-25T16:52:34.873363+0000","last_fullsized":"2024-04-25T16:52:34.873363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:38.694859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877510+0000","last_change":"2024-04-25T16:52:26.821081+0000","last_active":"2024-04-25T16:52:34.877510+0000","last_peered":"2024-04-25T16:52:34.877510+0000","last_clean":"2024-04-25T16:52:34.877510+0000","last_became_active":"2024-04-25T16:52:26.819337+0000","last_became_peered":"2024-04-25T16:52:26.819337+0000","last_unstale":"2024-04-25T16:52:34.877510+0000","last_undegraded":"2024-04-25T16:52:34.877510+0000","last_fullsized":"2024-04-25T16:52:34.877510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872999+0000","last_change":"2024-04-25T16:52:25.793039+0000","last_active":"2024-04-25T16:52:34.872999+0000","last_peered":"2024-04-25T16:52:34.872999+0000","last_clean":"2024-04-25T16:52:34.872999+0000","last_became_active":"2024-04-25T16:52:25.792552+0000","last_became_peered":"2024-04-25T16:52:25.792552+0000","last_unstale":"2024-04-25T16:52:34.872999+0000","last_undegraded":"2024-04-25T16:52:34.872999+0000","last_fullsized":"2024-04-25T16:52:34.872999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:30.954700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872507+0000","last_change":"2024-04-25T16:52:26.823643+0000","last_active":"2024-04-25T16:52:34.872507+0000","last_peered":"2024-04-25T16:52:34.872507+0000","last_clean":"2024-04-25T16:52:34.872507+0000","last_became_active":"2024-04-25T16:52:26.823348+0000","last_became_peered":"2024-04-25T16:52:26.823348+0000","last_unstale":"2024-04-25T16:52:34.872507+0000","last_undegraded":"2024-04-25T16:52:34.872507+0000","last_fullsized":"2024-04-25T16:52:34.872507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830926+0000","last_change":"2024-04-25T16:52:25.813706+0000","last_active":"2024-04-25T16:52:27.830926+0000","last_peered":"2024-04-25T16:52:27.830926+0000","last_clean":"2024-04-25T16:52:27.830926+0000","last_became_active":"2024-04-25T16:52:25.813387+0000","last_became_peered":"2024-04-25T16:52:25.813387+0000","last_unstale":"2024-04-25T16:52:27.830926+0000","last_undegraded":"2024-04-25T16:52:27.830926+0000","last_fullsized":"2024-04-25T16:52:27.830926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340755+0000","last_change":"2024-04-25T16:52:26.817332+0000","last_active":"2024-04-25T16:52:35.340755+0000","last_peered":"2024-04-25T16:52:35.340755+0000","last_clean":"2024-04-25T16:52:35.340755+0000","last_became_active":"2024-04-25T16:52:26.816020+0000","last_became_peered":"2024-04-25T16:52:26.816020+0000","last_unstale":"2024-04-25T16:52:35.340755+0000","last_undegraded":"2024-04-25T16:52:35.340755+0000","last_fullsized":"2024-04-25T16:52:35.340755+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872306+0000","last_change":"2024-04-25T16:52:25.815545+0000","last_active":"2024-04-25T16:52:34.872306+0000","last_peered":"2024-04-25T16:52:34.872306+0000","last_clean":"2024-04-25T16:52:34.872306+0000","last_became_active":"2024-04-25T16:52:25.814711+0000","last_became_peered":"2024-04-25T16:52:25.814711+0000","last_unstale":"2024-04-25T16:52:34.872306+0000","last_undegraded":"2024-04-25T16:52:34.872306+0000","last_fullsized":"2024-04-25T16:52:34.872306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872682+0000","last_change":"2024-04-25T16:52:26.826567+0000","last_active":"2024-04-25T16:52:34.872682+0000","last_peered":"2024-04-25T16:52:34.872682+0000","last_clean":"2024-04-25T16:52:34.872682+0000","last_became_active":"2024-04-25T16:52:26.826224+0000","last_became_peered":"2024-04-25T16:52:26.826224+0000","last_unstale":"2024-04-25T16:52:34.872682+0000","last_undegraded":"2024-04-25T16:52:34.872682+0000","last_fullsized":"2024-04-25T16:52:34.872682+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:06.823841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":24,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340795+0000","last_change":"2024-04-25T16:52:25.815153+0000","last_active":"2024-04-25T16:52:35.340795+0000","last_peered":"2024-04-25T16:52:35.340795+0000","last_clean":"2024-04-25T16:52:35.340795+0000","last_became_active":"2024-04-25T16:52:25.814360+0000","last_became_peered":"2024-04-25T16:52:25.814360+0000","last_unstale":"2024-04-25T16:52:35.340795+0000","last_undegraded":"2024-04-25T16:52:35.340795+0000","last_fullsized":"2024-04-25T16:52:35.340795+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873169+0000","last_change":"2024-04-25T16:52:26.804704+0000","last_active":"2024-04-25T16:52:34.873169+0000","last_peered":"2024-04-25T16:52:34.873169+0000","last_clean":"2024-04-25T16:52:34.873169+0000","last_became_active":"2024-04-25T16:52:26.803888+0000","last_became_peered":"2024-04-25T16:52:26.803888+0000","last_unstale":"2024-04-25T16:52:34.873169+0000","last_undegraded":"2024-04-25T16:52:34.873169+0000","last_fullsized":"2024-04-25T16:52:34.873169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.640204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878107+0000","last_change":"2024-04-25T16:52:25.818880+0000","last_active":"2024-04-25T16:52:34.878107+0000","last_peered":"2024-04-25T16:52:34.878107+0000","last_clean":"2024-04-25T16:52:34.878107+0000","last_became_active":"2024-04-25T16:52:25.817841+0000","last_became_peered":"2024-04-25T16:52:25.817841+0000","last_unstale":"2024-04-25T16:52:34.878107+0000","last_undegraded":"2024-04-25T16:52:34.878107+0000","last_fullsized":"2024-04-25T16:52:34.878107+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:26.817110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877704+0000","last_change":"2024-04-25T16:52:26.809554+0000","last_active":"2024-04-25T16:52:34.877704+0000","last_peered":"2024-04-25T16:52:34.877704+0000","last_clean":"2024-04-25T16:52:34.877704+0000","last_became_active":"2024-04-25T16:52:26.808709+0000","last_became_peered":"2024-04-25T16:52:26.808709+0000","last_unstale":"2024-04-25T16:52:34.877704+0000","last_undegraded":"2024-04-25T16:52:34.877704+0000","last_fullsized":"2024-04-25T16:52:34.877704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875584+0000","last_change":"2024-04-25T16:52:25.812269+0000","last_active":"2024-04-25T16:52:34.875584+0000","last_peered":"2024-04-25T16:52:34.875584+0000","last_clean":"2024-04-25T16:52:34.875584+0000","last_became_active":"2024-04-25T16:52:25.810831+0000","last_became_peered":"2024-04-25T16:52:25.810831+0000","last_unstale":"2024-04-25T16:52:34.875584+0000","last_undegraded":"2024-04-25T16:52:34.875584+0000","last_fullsized":"2024-04-25T16:52:34.875584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874980+0000","last_change":"2024-04-25T16:52:26.806939+0000","last_active":"2024-04-25T16:52:34.874980+0000","last_peered":"2024-04-25T16:52:34.874980+0000","last_clean":"2024-04-25T16:52:34.874980+0000","last_became_active":"2024-04-25T16:52:26.806053+0000","last_became_peered":"2024-04-25T16:52:26.806053+0000","last_unstale":"2024-04-25T16:52:34.874980+0000","last_undegraded":"2024-04-25T16:52:34.874980+0000","last_fullsized":"2024-04-25T16:52:34.874980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781168+0000","last_change":"2024-04-25T16:52:25.783759+0000","last_active":"2024-04-25T16:52:27.781168+0000","last_peered":"2024-04-25T16:52:27.781168+0000","last_clean":"2024-04-25T16:52:27.781168+0000","last_became_active":"2024-04-25T16:52:25.783353+0000","last_became_peered":"2024-04-25T16:52:25.783353+0000","last_unstale":"2024-04-25T16:52:27.781168+0000","last_undegraded":"2024-04-25T16:52:27.781168+0000","last_fullsized":"2024-04-25T16:52:27.781168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:34.281502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780945+0000","last_change":"2024-04-25T16:52:26.798434+0000","last_active":"2024-04-25T16:52:27.780945+0000","last_peered":"2024-04-25T16:52:27.780945+0000","last_clean":"2024-04-25T16:52:27.780945+0000","last_became_active":"2024-04-25T16:52:26.797631+0000","last_became_peered":"2024-04-25T16:52:26.797631+0000","last_unstale":"2024-04-25T16:52:27.780945+0000","last_undegraded":"2024-04-25T16:52:27.780945+0000","last_fullsized":"2024-04-25T16:52:27.780945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:16.248387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831612+0000","last_change":"2024-04-25T16:52:25.819136+0000","last_active":"2024-04-25T16:52:27.831612+0000","last_peered":"2024-04-25T16:52:27.831612+0000","last_clean":"2024-04-25T16:52:27.831612+0000","last_became_active":"2024-04-25T16:52:25.818809+0000","last_became_peered":"2024-04-25T16:52:25.818809+0000","last_unstale":"2024-04-25T16:52:27.831612+0000","last_undegraded":"2024-04-25T16:52:27.831612+0000","last_fullsized":"2024-04-25T16:52:27.831612+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341292+0000","last_change":"2024-04-25T16:52:26.819612+0000","last_active":"2024-04-25T16:52:35.341292+0000","last_peered":"2024-04-25T16:52:35.341292+0000","last_clean":"2024-04-25T16:52:35.341292+0000","last_became_active":"2024-04-25T16:52:26.818874+0000","last_became_peered":"2024-04-25T16:52:26.818874+0000","last_unstale":"2024-04-25T16:52:35.341292+0000","last_undegraded":"2024-04-25T16:52:35.341292+0000","last_fullsized":"2024-04-25T16:52:35.341292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781486+0000","last_change":"2024-04-25T16:52:25.780874+0000","last_active":"2024-04-25T16:52:27.781486+0000","last_peered":"2024-04-25T16:52:27.781486+0000","last_clean":"2024-04-25T16:52:27.781486+0000","last_became_active":"2024-04-25T16:52:25.780529+0000","last_became_peered":"2024-04-25T16:52:25.780529+0000","last_unstale":"2024-04-25T16:52:27.781486+0000","last_undegraded":"2024-04-25T16:52:27.781486+0000","last_fullsized":"2024-04-25T16:52:27.781486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:57.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878077+0000","last_change":"2024-04-25T16:52:26.845027+0000","last_active":"2024-04-25T16:52:34.878077+0000","last_peered":"2024-04-25T16:52:34.878077+0000","last_clean":"2024-04-25T16:52:34.878077+0000","last_became_active":"2024-04-25T16:52:26.844620+0000","last_became_peered":"2024-04-25T16:52:26.844620+0000","last_unstale":"2024-04-25T16:52:34.878077+0000","last_undegraded":"2024-04-25T16:52:34.878077+0000","last_fullsized":"2024-04-25T16:52:34.878077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780947+0000","last_change":"2024-04-25T16:52:25.821889+0000","last_active":"2024-04-25T16:52:27.780947+0000","last_peered":"2024-04-25T16:52:27.780947+0000","last_clean":"2024-04-25T16:52:27.780947+0000","last_became_active":"2024-04-25T16:52:25.821643+0000","last_became_peered":"2024-04-25T16:52:25.821643+0000","last_unstale":"2024-04-25T16:52:27.780947+0000","last_undegraded":"2024-04-25T16:52:27.780947+0000","last_fullsized":"2024-04-25T16:52:27.780947+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:27.786669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877414+0000","last_change":"2024-04-25T16:52:26.802864+0000","last_active":"2024-04-25T16:52:34.877414+0000","last_peered":"2024-04-25T16:52:34.877414+0000","last_clean":"2024-04-25T16:52:34.877414+0000","last_became_active":"2024-04-25T16:52:26.801913+0000","last_became_peered":"2024-04-25T16:52:26.801913+0000","last_unstale":"2024-04-25T16:52:34.877414+0000","last_undegraded":"2024-04-25T16:52:34.877414+0000","last_fullsized":"2024-04-25T16:52:34.877414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341042+0000","last_change":"2024-04-25T16:52:25.806124+0000","last_active":"2024-04-25T16:52:35.341042+0000","last_peered":"2024-04-25T16:52:35.341042+0000","last_clean":"2024-04-25T16:52:35.341042+0000","last_became_active":"2024-04-25T16:52:25.805306+0000","last_became_peered":"2024-04-25T16:52:25.805306+0000","last_unstale":"2024-04-25T16:52:35.341042+0000","last_undegraded":"2024-04-25T16:52:35.341042+0000","last_fullsized":"2024-04-25T16:52:35.341042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872688+0000","last_change":"2024-04-25T16:52:26.819372+0000","last_active":"2024-04-25T16:52:34.872688+0000","last_peered":"2024-04-25T16:52:34.872688+0000","last_clean":"2024-04-25T16:52:34.872688+0000","last_became_active":"2024-04-25T16:52:26.807182+0000","last_became_peered":"2024-04-25T16:52:26.807182+0000","last_unstale":"2024-04-25T16:52:34.872688+0000","last_undegraded":"2024-04-25T16:52:34.872688+0000","last_fullsized":"2024-04-25T16:52:34.872688+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:14.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780505+0000","last_change":"2024-04-25T16:52:25.788959+0000","last_active":"2024-04-25T16:52:27.780505+0000","last_peered":"2024-04-25T16:52:27.780505+0000","last_clean":"2024-04-25T16:52:27.780505+0000","last_became_active":"2024-04-25T16:52:25.788261+0000","last_became_peered":"2024-04-25T16:52:25.788261+0000","last_unstale":"2024-04-25T16:52:27.780505+0000","last_undegraded":"2024-04-25T16:52:27.780505+0000","last_fullsized":"2024-04-25T16:52:27.780505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:13.943131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831765+0000","last_change":"2024-04-25T16:52:26.819641+0000","last_active":"2024-04-25T16:52:27.831765+0000","last_peered":"2024-04-25T16:52:27.831765+0000","last_clean":"2024-04-25T16:52:27.831765+0000","last_became_active":"2024-04-25T16:52:26.819314+0000","last_became_peered":"2024-04-25T16:52:26.819314+0000","last_unstale":"2024-04-25T16:52:27.831765+0000","last_undegraded":"2024-04-25T16:52:27.831765+0000","last_fullsized":"2024-04-25T16:52:27.831765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878486+0000","last_change":"2024-04-25T16:52:25.806269+0000","last_active":"2024-04-25T16:52:34.878486+0000","last_peered":"2024-04-25T16:52:34.878486+0000","last_clean":"2024-04-25T16:52:34.878486+0000","last_became_active":"2024-04-25T16:52:25.804183+0000","last_became_peered":"2024-04-25T16:52:25.804183+0000","last_unstale":"2024-04-25T16:52:34.878486+0000","last_undegraded":"2024-04-25T16:52:34.878486+0000","last_fullsized":"2024-04-25T16:52:34.878486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:07.902516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341557+0000","last_change":"2024-04-25T16:52:26.841753+0000","last_active":"2024-04-25T16:52:35.341557+0000","last_peered":"2024-04-25T16:52:35.341557+0000","last_clean":"2024-04-25T16:52:35.341557+0000","last_became_active":"2024-04-25T16:52:26.841112+0000","last_became_peered":"2024-04-25T16:52:26.841112+0000","last_unstale":"2024-04-25T16:52:35.341557+0000","last_undegraded":"2024-04-25T16:52:35.341557+0000","last_fullsized":"2024-04-25T16:52:35.341557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781212+0000","last_change":"2024-04-25T16:52:25.803168+0000","last_active":"2024-04-25T16:52:27.781212+0000","last_peered":"2024-04-25T16:52:27.781212+0000","last_clean":"2024-04-25T16:52:27.781212+0000","last_became_active":"2024-04-25T16:52:25.802880+0000","last_became_peered":"2024-04-25T16:52:25.802880+0000","last_unstale":"2024-04-25T16:52:27.781212+0000","last_undegraded":"2024-04-25T16:52:27.781212+0000","last_fullsized":"2024-04-25T16:52:27.781212+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.210482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873797+0000","last_change":"2024-04-25T16:52:26.819781+0000","last_active":"2024-04-25T16:52:34.873797+0000","last_peered":"2024-04-25T16:52:34.873797+0000","last_clean":"2024-04-25T16:52:34.873797+0000","last_became_active":"2024-04-25T16:52:26.819467+0000","last_became_peered":"2024-04-25T16:52:26.819467+0000","last_unstale":"2024-04-25T16:52:34.873797+0000","last_undegraded":"2024-04-25T16:52:34.873797+0000","last_fullsized":"2024-04-25T16:52:34.873797+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878534+0000","last_change":"2024-04-25T16:52:25.799953+0000","last_active":"2024-04-25T16:52:34.878534+0000","last_peered":"2024-04-25T16:52:34.878534+0000","last_clean":"2024-04-25T16:52:34.878534+0000","last_became_active":"2024-04-25T16:52:25.799689+0000","last_became_peered":"2024-04-25T16:52:25.799689+0000","last_unstale":"2024-04-25T16:52:34.878534+0000","last_undegraded":"2024-04-25T16:52:34.878534+0000","last_fullsized":"2024-04-25T16:52:34.878534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877833+0000","last_change":"2024-04-25T16:52:26.799367+0000","last_active":"2024-04-25T16:52:34.877833+0000","last_peered":"2024-04-25T16:52:34.877833+0000","last_clean":"2024-04-25T16:52:34.877833+0000","last_became_active":"2024-04-25T16:52:26.797337+0000","last_became_peered":"2024-04-25T16:52:26.797337+0000","last_unstale":"2024-04-25T16:52:34.877833+0000","last_undegraded":"2024-04-25T16:52:34.877833+0000","last_fullsized":"2024-04-25T16:52:34.877833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878430+0000","last_change":"2024-04-25T16:52:25.806956+0000","last_active":"2024-04-25T16:52:34.878430+0000","last_peered":"2024-04-25T16:52:34.878430+0000","last_clean":"2024-04-25T16:52:34.878430+0000","last_became_active":"2024-04-25T16:52:25.806604+0000","last_became_peered":"2024-04-25T16:52:25.806604+0000","last_unstale":"2024-04-25T16:52:34.878430+0000","last_undegraded":"2024-04-25T16:52:34.878430+0000","last_fullsized":"2024-04-25T16:52:34.878430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831008+0000","last_change":"2024-04-25T16:52:26.820845+0000","last_active":"2024-04-25T16:52:27.831008+0000","last_peered":"2024-04-25T16:52:27.831008+0000","last_clean":"2024-04-25T16:52:27.831008+0000","last_became_active":"2024-04-25T16:52:26.820484+0000","last_became_peered":"2024-04-25T16:52:26.820484+0000","last_unstale":"2024-04-25T16:52:27.831008+0000","last_undegraded":"2024-04-25T16:52:27.831008+0000","last_fullsized":"2024-04-25T16:52:27.831008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:44.359144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874630+0000","last_change":"2024-04-25T16:52:25.785723+0000","last_active":"2024-04-25T16:52:34.874630+0000","last_peered":"2024-04-25T16:52:34.874630+0000","last_clean":"2024-04-25T16:52:34.874630+0000","last_became_active":"2024-04-25T16:52:25.784930+0000","last_became_peered":"2024-04-25T16:52:25.784930+0000","last_unstale":"2024-04-25T16:52:34.874630+0000","last_undegraded":"2024-04-25T16:52:34.874630+0000","last_fullsized":"2024-04-25T16:52:34.874630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872198+0000","last_change":"2024-04-25T16:52:26.799277+0000","last_active":"2024-04-25T16:52:34.872198+0000","last_peered":"2024-04-25T16:52:34.872198+0000","last_clean":"2024-04-25T16:52:34.872198+0000","last_became_active":"2024-04-25T16:52:26.797817+0000","last_became_peered":"2024-04-25T16:52:26.797817+0000","last_unstale":"2024-04-25T16:52:34.872198+0000","last_undegraded":"2024-04-25T16:52:34.872198+0000","last_fullsized":"2024-04-25T16:52:34.872198+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871616+0000","last_change":"2024-04-25T16:52:25.810628+0000","last_active":"2024-04-25T16:52:34.871616+0000","last_peered":"2024-04-25T16:52:34.871616+0000","last_clean":"2024-04-25T16:52:34.871616+0000","last_became_active":"2024-04-25T16:52:25.809669+0000","last_became_peered":"2024-04-25T16:52:25.809669+0000","last_unstale":"2024-04-25T16:52:34.871616+0000","last_undegraded":"2024-04-25T16:52:34.871616+0000","last_fullsized":"2024-04-25T16:52:34.871616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:36.638682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341549+0000","last_change":"2024-04-25T16:52:26.832124+0000","last_active":"2024-04-25T16:52:35.341549+0000","last_peered":"2024-04-25T16:52:35.341549+0000","last_clean":"2024-04-25T16:52:35.341549+0000","last_became_active":"2024-04-25T16:52:26.831437+0000","last_became_peered":"2024-04-25T16:52:26.831437+0000","last_unstale":"2024-04-25T16:52:35.341549+0000","last_undegraded":"2024-04-25T16:52:35.341549+0000","last_fullsized":"2024-04-25T16:52:35.341549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872502+0000","last_change":"2024-04-25T16:52:25.793944+0000","last_active":"2024-04-25T16:52:34.872502+0000","last_peered":"2024-04-25T16:52:34.872502+0000","last_clean":"2024-04-25T16:52:34.872502+0000","last_became_active":"2024-04-25T16:52:25.793522+0000","last_became_peered":"2024-04-25T16:52:25.793522+0000","last_unstale":"2024-04-25T16:52:34.872502+0000","last_undegraded":"2024-04-25T16:52:34.872502+0000","last_fullsized":"2024-04-25T16:52:34.872502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:41.542439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877722+0000","last_change":"2024-04-25T16:52:26.804643+0000","last_active":"2024-04-25T16:52:34.877722+0000","last_peered":"2024-04-25T16:52:34.877722+0000","last_clean":"2024-04-25T16:52:34.877722+0000","last_became_active":"2024-04-25T16:52:26.802297+0000","last_became_peered":"2024-04-25T16:52:26.802297+0000","last_unstale":"2024-04-25T16:52:34.877722+0000","last_undegraded":"2024-04-25T16:52:34.877722+0000","last_fullsized":"2024-04-25T16:52:34.877722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341255+0000","last_change":"2024-04-25T16:52:25.796075+0000","last_active":"2024-04-25T16:52:35.341255+0000","last_peered":"2024-04-25T16:52:35.341255+0000","last_clean":"2024-04-25T16:52:35.341255+0000","last_became_active":"2024-04-25T16:52:25.795226+0000","last_became_peered":"2024-04-25T16:52:25.795226+0000","last_unstale":"2024-04-25T16:52:35.341255+0000","last_undegraded":"2024-04-25T16:52:35.341255+0000","last_fullsized":"2024-04-25T16:52:35.341255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":145,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340402+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:52:35.340402+0000","last_peered":"2024-04-25T16:52:35.340402+0000","last_clean":"2024-04-25T16:52:35.340402+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:52:35.340402+0000","last_undegraded":"2024-04-25T16:52:35.340402+0000","last_fullsized":"2024-04-25T16:52:35.340402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":196969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":196499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":458752,"data_stored":196991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35796,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336044,"statfs":{"total":96636764160,"available":96600109056,"internally_reserved":0,"allocated":466944,"data_stored":198033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":190098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35776,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336064,"statfs":{"total":96636764160,"available":96600129536,"internally_reserved":0,"allocated":442368,"data_stored":190118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335476,"statfs":{"total":96636764160,"available":96599527424,"internally_reserved":0,"allocated":1048576,"data_stored":786789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:52:40.087 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:52:40.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc184367640 1 Processor -- start 2024-04-25T16:52:40.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc184367640 1 -- start start 2024-04-25T16:52:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc184367640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0d0be0 0x7fc17c15e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc184367640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc17c15dec0 con 0x7fc17c0cefb0 2024-04-25T16:52:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc184367640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc17c0d11a0 con 0x7fc17c06e5b0 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc184367640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc17c0d1480 con 0x7fc17c0d0be0 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181902640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc17c0cefb0 0x7fc17c06dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:33582/0 (socket says 172.21.15.94:33582) 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181902640 1 -- 172.21.15.94:0/413104603 learned_addr learned my addr 172.21.15.94:0/413104603 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 796411054 0 0) 0x7fc17c15dec0 con 0x7fc17c0cefb0 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc182103640 1 --2- 172.21.15.94:0/413104603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0d0be0 0x7fc17c15e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc168003530 con 0x7fc17c0cefb0 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2444293606 0 0) 0x7fc168003530 con 0x7fc17c0cefb0 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0d0be0 msgr2=0x7fc17c15e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 --2- 172.21.15.94:0/413104603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0d0be0 0x7fc17c15e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 >> v1:172.21.15.138:6789/0 conn(0x7fc17c06e5b0 legacy=0x7fc17c15f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17c0d1760 con 0x7fc17c0cefb0 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7fc16c0016a0 con 0x7fc17c0cefb0 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 >> v1:172.21.15.94:6789/0 conn(0x7fc17c0cefb0 legacy=0x7fc17c06dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 --2- 172.21.15.94:0/413104603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1680039f0 0x7fc168003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 --2- 172.21.15.94:0/413104603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0d0be0 0x7fc168004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 --2- 172.21.15.94:0/413104603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc168004d50 0x7fc168005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c0d1480 con 0x7fc168004d50 2024-04-25T16:52:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc16800bb00 con 0x7fc1680039f0 2024-04-25T16:52:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc16800bd00 con 0x7fc17c0d0be0 2024-04-25T16:52:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc182904640 1 --2- 172.21.15.94:0/413104603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc168004d50 0x7fc168005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc182103640 1 --2- 172.21.15.94:0/413104603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1680039f0 0x7fc168003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc181902640 1 --2- 172.21.15.94:0/413104603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0d0be0 0x7fc168004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc182904640 1 -- 172.21.15.94:0/413104603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0d0be0 msgr2=0x7fc168004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc182904640 1 --2- 172.21.15.94:0/413104603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c0d0be0 0x7fc168004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc182904640 1 -- 172.21.15.94:0/413104603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1680039f0 msgr2=0x7fc168003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc182904640 1 --2- 172.21.15.94:0/413104603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1680039f0 0x7fc168003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.251+0000 7fc182904640 1 -- 172.21.15.94:0/413104603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc168003530 con 0x7fc168004d50 2024-04-25T16:52:40.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.255+0000 7fc182904640 1 --2- 172.21.15.94:0/413104603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc168004d50 0x7fc168005160 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc174001e40 tx=0x7fc17401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f27aab80e0a612 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:40.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.255+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174001a90 con 0x7fc168004d50 2024-04-25T16:52:40.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.255+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc174003b80 con 0x7fc168004d50 2024-04-25T16:52:40.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.255+0000 7fc181101640 1 -- 172.21.15.94:0/413104603 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174005080 con 0x7fc168004d50 2024-04-25T16:52:40.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.255+0000 7fc184367640 1 -- 172.21.15.94:0/413104603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc168004d50 msgr2=0x7fc168005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:40.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.255+0000 7fc184367640 1 --2- 172.21.15.94:0/413104603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc168004d50 0x7fc168005160 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc174001e40 tx=0x7fc17401c760 comp rx=0 tx=0).stop 2024-04-25T16:52:40.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.255+0000 7fc182904640 1 -- 172.21.15.94:0/413104603 reap_dead start 2024-04-25T16:52:40.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.255+0000 7fc184367640 1 -- 172.21.15.94:0/413104603 shutdown_connections 2024-04-25T16:52:40.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.255+0000 7fc184367640 1 -- 172.21.15.94:0/413104603 >> 172.21.15.94:0/413104603 conn(0x7fc17c072fd0 msgr2=0x7fc17c0733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:40.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.255+0000 7fc184367640 1 -- 172.21.15.94:0/413104603 shutdown_connections 2024-04-25T16:52:40.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.255+0000 7fc184367640 1 -- 172.21.15.94:0/413104603 wait complete. 2024-04-25T16:52:40.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc184367640 1 Processor -- start 2024-04-25T16:52:40.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc184367640 1 -- start start 2024-04-25T16:52:40.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc184367640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc17c1501e0 0x7fc17c151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc184367640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c152ee0 0x7fc17c1532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc184367640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c158580 0x7fc17c158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc184367640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c15f910 con 0x7fc17c1501e0 2024-04-25T16:52:40.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc184367640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c15f790 con 0x7fc17c158580 2024-04-25T16:52:40.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc184367640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc17c153830 con 0x7fc17c152ee0 2024-04-25T16:52:40.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc182103640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc17c1501e0 0x7fc17c151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:40.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc182103640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc17c1501e0 0x7fc17c151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49752/0 (socket says 172.21.15.94:49752) 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc182103640 1 -- 172.21.15.94:0/2397524526 learned_addr learned my addr 172.21.15.94:0/2397524526 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc182904640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c158580 0x7fc17c158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc181902640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c152ee0 0x7fc17c1532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc182103640 1 -- 172.21.15.94:0/2397524526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c152ee0 msgr2=0x7fc17c1532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc182103640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c152ee0 0x7fc17c1532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc182103640 1 -- 172.21.15.94:0/2397524526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c158580 msgr2=0x7fc17c158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc182103640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c158580 0x7fc17c158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc182103640 1 -- 172.21.15.94:0/2397524526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17c152170 con 0x7fc17c1501e0 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc182904640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c158580 0x7fc17c158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc182103640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc17c1501e0 0x7fc17c151790 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc178011560 tx=0x7fc178011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc172ffd640 1 -- 172.21.15.94:0/2397524526 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178019070 con 0x7fc17c1501e0 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc172ffd640 1 -- 172.21.15.94:0/2397524526 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc178003620 con 0x7fc17c1501e0 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc184367640 1 -- 172.21.15.94:0/2397524526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17c152450 con 0x7fc17c1501e0 2024-04-25T16:52:40.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc172ffd640 1 -- 172.21.15.94:0/2397524526 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178003a10 con 0x7fc17c1501e0 2024-04-25T16:52:40.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc184367640 1 -- 172.21.15.94:0/2397524526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc17c15b820 con 0x7fc17c1501e0 2024-04-25T16:52:40.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.259+0000 7fc184367640 1 -- 172.21.15.94:0/2397524526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc17c076830 con 0x7fc17c1501e0 2024-04-25T16:52:40.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.263+0000 7fc172ffd640 1 -- 172.21.15.94:0/2397524526 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc17801c070 con 0x7fc17c1501e0 2024-04-25T16:52:40.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.263+0000 7fc172ffd640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc16006f400 0x7fc160071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.263+0000 7fc172ffd640 1 -- 172.21.15.94:0/2397524526 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fc17808f3f0 con 0x7fc17c1501e0 2024-04-25T16:52:40.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.263+0000 7fc181902640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc16006f400 0x7fc160071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:40.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.263+0000 7fc181902640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc16006f400 0x7fc160071910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0d0950 tx=0x7fc16c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:40.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.267+0000 7fc172ffd640 1 -- 172.21.15.94:0/2397524526 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc17805c250 con 0x7fc17c1501e0 2024-04-25T16:52:40.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.423+0000 7fc184367640 1 -- 172.21.15.94:0/2397524526 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc17c152ad0 con 0x7fc16006f400 2024-04-25T16:52:40.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.435+0000 7fc172ffd640 1 -- 172.21.15.94:0/2397524526 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251872 (secure 0 0 0) 0x7fc17c152ad0 con 0x7fc16006f400 2024-04-25T16:52:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:40.437 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:52:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.439+0000 7fc184367640 1 -- 172.21.15.94:0/2397524526 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc16006f400 msgr2=0x7fc160071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.439+0000 7fc184367640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc16006f400 0x7fc160071910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0d0950 tx=0x7fc16c009040 comp rx=0 tx=0).stop 2024-04-25T16:52:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.439+0000 7fc184367640 1 -- 172.21.15.94:0/2397524526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc17c1501e0 msgr2=0x7fc17c151790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.439+0000 7fc184367640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc17c1501e0 0x7fc17c151790 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc178011560 tx=0x7fc178011a20 comp rx=0 tx=0).stop 2024-04-25T16:52:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.439+0000 7fc184367640 1 -- 172.21.15.94:0/2397524526 shutdown_connections 2024-04-25T16:52:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.439+0000 7fc184367640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc17c158580 0x7fc17c158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.439+0000 7fc184367640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc16006f400 0x7fc160071910 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.439+0000 7fc184367640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc17c152ee0 0x7fc17c1532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.439+0000 7fc184367640 1 --2- 172.21.15.94:0/2397524526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc17c1501e0 0x7fc17c151790 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.439+0000 7fc184367640 1 -- 172.21.15.94:0/2397524526 >> 172.21.15.94:0/2397524526 conn(0x7fc17c072fd0 msgr2=0x7fc17c0b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.439+0000 7fc184367640 1 -- 172.21.15.94:0/2397524526 shutdown_connections 2024-04-25T16:52:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.439+0000 7fc184367640 1 -- 172.21.15.94:0/2397524526 wait complete. 2024-04-25T16:52:40.457 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-04-25T16:52:39.696569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258320,"kb_used_data":4708,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716400,"statfs":{"total":773094113280,"available":772829593600,"internally_reserved":0,"allocated":4820992,"data_stored":2737575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002542"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873724+0000","last_change":"2024-04-25T16:52:26.825748+0000","last_active":"2024-04-25T16:52:34.873724+0000","last_peered":"2024-04-25T16:52:34.873724+0000","last_clean":"2024-04-25T16:52:34.873724+0000","last_became_active":"2024-04-25T16:52:26.825333+0000","last_became_peered":"2024-04-25T16:52:26.825333+0000","last_unstale":"2024-04-25T16:52:34.873724+0000","last_undegraded":"2024-04-25T16:52:34.873724+0000","last_fullsized":"2024-04-25T16:52:34.873724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:19.084563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873506+0000","last_change":"2024-04-25T16:52:26.795964+0000","last_active":"2024-04-25T16:52:34.873506+0000","last_peered":"2024-04-25T16:52:34.873506+0000","last_clean":"2024-04-25T16:52:34.873506+0000","last_became_active":"2024-04-25T16:52:26.794884+0000","last_became_peered":"2024-04-25T16:52:26.794884+0000","last_unstale":"2024-04-25T16:52:34.873506+0000","last_undegraded":"2024-04-25T16:52:34.873506+0000","last_fullsized":"2024-04-25T16:52:34.873506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878233+0000","last_change":"2024-04-25T16:52:26.802075+0000","last_active":"2024-04-25T16:52:34.878233+0000","last_peered":"2024-04-25T16:52:34.878233+0000","last_clean":"2024-04-25T16:52:34.878233+0000","last_became_active":"2024-04-25T16:52:26.801203+0000","last_became_peered":"2024-04-25T16:52:26.801203+0000","last_unstale":"2024-04-25T16:52:34.878233+0000","last_undegraded":"2024-04-25T16:52:34.878233+0000","last_fullsized":"2024-04-25T16:52:34.878233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782213+0000","last_change":"2024-04-25T16:52:26.793541+0000","last_active":"2024-04-25T16:52:27.782213+0000","last_peered":"2024-04-25T16:52:27.782213+0000","last_clean":"2024-04-25T16:52:27.782213+0000","last_became_active":"2024-04-25T16:52:26.789101+0000","last_became_peered":"2024-04-25T16:52:26.789101+0000","last_unstale":"2024-04-25T16:52:27.782213+0000","last_undegraded":"2024-04-25T16:52:27.782213+0000","last_fullsized":"2024-04-25T16:52:27.782213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:38.013058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781321+0000","last_change":"2024-04-25T16:52:26.784467+0000","last_active":"2024-04-25T16:52:27.781321+0000","last_peered":"2024-04-25T16:52:27.781321+0000","last_clean":"2024-04-25T16:52:27.781321+0000","last_became_active":"2024-04-25T16:52:26.783673+0000","last_became_peered":"2024-04-25T16:52:26.783673+0000","last_unstale":"2024-04-25T16:52:27.781321+0000","last_undegraded":"2024-04-25T16:52:27.781321+0000","last_fullsized":"2024-04-25T16:52:27.781321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:36.583897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831274+0000","last_change":"2024-04-25T16:52:26.823413+0000","last_active":"2024-04-25T16:52:27.831274+0000","last_peered":"2024-04-25T16:52:27.831274+0000","last_clean":"2024-04-25T16:52:27.831274+0000","last_became_active":"2024-04-25T16:52:26.823059+0000","last_became_peered":"2024-04-25T16:52:26.823059+0000","last_unstale":"2024-04-25T16:52:27.831274+0000","last_undegraded":"2024-04-25T16:52:27.831274+0000","last_fullsized":"2024-04-25T16:52:27.831274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.876836+0000","last_change":"2024-04-25T16:52:26.808661+0000","last_active":"2024-04-25T16:52:34.876836+0000","last_peered":"2024-04-25T16:52:34.876836+0000","last_clean":"2024-04-25T16:52:34.876836+0000","last_became_active":"2024-04-25T16:52:26.805296+0000","last_became_peered":"2024-04-25T16:52:26.805296+0000","last_unstale":"2024-04-25T16:52:34.876836+0000","last_undegraded":"2024-04-25T16:52:34.876836+0000","last_fullsized":"2024-04-25T16:52:34.876836+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"peering","last_fresh":"2024-04-25T16:52:30.709501+0000","last_change":"2024-04-25T16:52:34.873590+0000","last_active":"2024-04-25T16:52:34.873590+0000","last_peered":"2024-04-25T16:52:30.709501+0000","last_clean":"2024-04-25T16:52:30.709501+0000","last_became_active":"2024-04-25T16:52:26.820528+0000","last_became_peered":"2024-04-25T16:52:26.820528+0000","last_unstale":"2024-04-25T16:52:30.709501+0000","last_undegraded":"2024-04-25T16:52:30.709501+0000","last_fullsized":"2024-04-25T16:52:30.709501+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.879080+0000","last_change":"2024-04-25T16:52:26.803363+0000","last_active":"2024-04-25T16:52:34.879080+0000","last_peered":"2024-04-25T16:52:34.879080+0000","last_clean":"2024-04-25T16:52:34.879080+0000","last_became_active":"2024-04-25T16:52:26.802539+0000","last_became_peered":"2024-04-25T16:52:26.802539+0000","last_unstale":"2024-04-25T16:52:34.879080+0000","last_undegraded":"2024-04-25T16:52:34.879080+0000","last_fullsized":"2024-04-25T16:52:34.879080+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877890+0000","last_change":"2024-04-25T16:52:26.842728+0000","last_active":"2024-04-25T16:52:34.877890+0000","last_peered":"2024-04-25T16:52:34.877890+0000","last_clean":"2024-04-25T16:52:34.877890+0000","last_became_active":"2024-04-25T16:52:26.842041+0000","last_became_peered":"2024-04-25T16:52:26.842041+0000","last_unstale":"2024-04-25T16:52:34.877890+0000","last_undegraded":"2024-04-25T16:52:34.877890+0000","last_fullsized":"2024-04-25T16:52:34.877890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:33.962706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877607+0000","last_change":"2024-04-25T16:52:26.819329+0000","last_active":"2024-04-25T16:52:34.877607+0000","last_peered":"2024-04-25T16:52:34.877607+0000","last_clean":"2024-04-25T16:52:34.877607+0000","last_became_active":"2024-04-25T16:52:26.818528+0000","last_became_peered":"2024-04-25T16:52:26.818528+0000","last_unstale":"2024-04-25T16:52:34.877607+0000","last_undegraded":"2024-04-25T16:52:34.877607+0000","last_fullsized":"2024-04-25T16:52:34.877607+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878111+0000","last_change":"2024-04-25T16:52:26.795942+0000","last_active":"2024-04-25T16:52:34.878111+0000","last_peered":"2024-04-25T16:52:34.878111+0000","last_clean":"2024-04-25T16:52:34.878111+0000","last_became_active":"2024-04-25T16:52:26.795124+0000","last_became_peered":"2024-04-25T16:52:26.795124+0000","last_unstale":"2024-04-25T16:52:34.878111+0000","last_undegraded":"2024-04-25T16:52:34.878111+0000","last_fullsized":"2024-04-25T16:52:34.878111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877026+0000","last_change":"2024-04-25T16:52:26.814749+0000","last_active":"2024-04-25T16:52:34.877026+0000","last_peered":"2024-04-25T16:52:34.877026+0000","last_clean":"2024-04-25T16:52:34.877026+0000","last_became_active":"2024-04-25T16:52:26.814020+0000","last_became_peered":"2024-04-25T16:52:26.814020+0000","last_unstale":"2024-04-25T16:52:34.877026+0000","last_undegraded":"2024-04-25T16:52:34.877026+0000","last_fullsized":"2024-04-25T16:52:34.877026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871890+0000","last_change":"2024-04-25T16:52:26.813118+0000","last_active":"2024-04-25T16:52:34.871890+0000","last_peered":"2024-04-25T16:52:34.871890+0000","last_clean":"2024-04-25T16:52:34.871890+0000","last_became_active":"2024-04-25T16:52:26.812400+0000","last_became_peered":"2024-04-25T16:52:26.812400+0000","last_unstale":"2024-04-25T16:52:34.871890+0000","last_undegraded":"2024-04-25T16:52:34.871890+0000","last_fullsized":"2024-04-25T16:52:34.871890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:16.169654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877027+0000","last_change":"2024-04-25T16:52:26.795864+0000","last_active":"2024-04-25T16:52:34.877027+0000","last_peered":"2024-04-25T16:52:34.877027+0000","last_clean":"2024-04-25T16:52:34.877027+0000","last_became_active":"2024-04-25T16:52:26.794738+0000","last_became_peered":"2024-04-25T16:52:26.794738+0000","last_unstale":"2024-04-25T16:52:34.877027+0000","last_undegraded":"2024-04-25T16:52:34.877027+0000","last_fullsized":"2024-04-25T16:52:34.877027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340722+0000","last_change":"2024-04-25T16:52:26.836640+0000","last_active":"2024-04-25T16:52:35.340722+0000","last_peered":"2024-04-25T16:52:35.340722+0000","last_clean":"2024-04-25T16:52:35.340722+0000","last_became_active":"2024-04-25T16:52:26.833071+0000","last_became_peered":"2024-04-25T16:52:26.833071+0000","last_unstale":"2024-04-25T16:52:35.340722+0000","last_undegraded":"2024-04-25T16:52:35.340722+0000","last_fullsized":"2024-04-25T16:52:35.340722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341191+0000","last_change":"2024-04-25T16:52:26.841207+0000","last_active":"2024-04-25T16:52:35.341191+0000","last_peered":"2024-04-25T16:52:35.341191+0000","last_clean":"2024-04-25T16:52:35.341191+0000","last_became_active":"2024-04-25T16:52:26.838750+0000","last_became_peered":"2024-04-25T16:52:26.838750+0000","last_unstale":"2024-04-25T16:52:35.341191+0000","last_undegraded":"2024-04-25T16:52:35.341191+0000","last_fullsized":"2024-04-25T16:52:35.341191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875046+0000","last_change":"2024-04-25T16:52:26.795939+0000","last_active":"2024-04-25T16:52:34.875046+0000","last_peered":"2024-04-25T16:52:34.875046+0000","last_clean":"2024-04-25T16:52:34.875046+0000","last_became_active":"2024-04-25T16:52:26.795084+0000","last_became_peered":"2024-04-25T16:52:26.795084+0000","last_unstale":"2024-04-25T16:52:34.875046+0000","last_undegraded":"2024-04-25T16:52:34.875046+0000","last_fullsized":"2024-04-25T16:52:34.875046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781139+0000","last_change":"2024-04-25T16:52:26.818025+0000","last_active":"2024-04-25T16:52:27.781139+0000","last_peered":"2024-04-25T16:52:27.781139+0000","last_clean":"2024-04-25T16:52:27.781139+0000","last_became_active":"2024-04-25T16:52:26.817652+0000","last_became_peered":"2024-04-25T16:52:26.817652+0000","last_unstale":"2024-04-25T16:52:27.781139+0000","last_undegraded":"2024-04-25T16:52:27.781139+0000","last_fullsized":"2024-04-25T16:52:27.781139+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:46.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872811+0000","last_change":"2024-04-25T16:52:26.796384+0000","last_active":"2024-04-25T16:52:34.872811+0000","last_peered":"2024-04-25T16:52:34.872811+0000","last_clean":"2024-04-25T16:52:34.872811+0000","last_became_active":"2024-04-25T16:52:26.795507+0000","last_became_peered":"2024-04-25T16:52:26.795507+0000","last_unstale":"2024-04-25T16:52:34.872811+0000","last_undegraded":"2024-04-25T16:52:34.872811+0000","last_fullsized":"2024-04-25T16:52:34.872811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877104+0000","last_change":"2024-04-25T16:52:26.809988+0000","last_active":"2024-04-25T16:52:34.877104+0000","last_peered":"2024-04-25T16:52:34.877104+0000","last_clean":"2024-04-25T16:52:34.877104+0000","last_became_active":"2024-04-25T16:52:26.809107+0000","last_became_peered":"2024-04-25T16:52:26.809107+0000","last_unstale":"2024-04-25T16:52:34.877104+0000","last_undegraded":"2024-04-25T16:52:34.877104+0000","last_fullsized":"2024-04-25T16:52:34.877104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872865+0000","last_change":"2024-04-25T16:52:26.784466+0000","last_active":"2024-04-25T16:52:34.872865+0000","last_peered":"2024-04-25T16:52:34.872865+0000","last_clean":"2024-04-25T16:52:34.872865+0000","last_became_active":"2024-04-25T16:52:26.783550+0000","last_became_peered":"2024-04-25T16:52:26.783550+0000","last_unstale":"2024-04-25T16:52:34.872865+0000","last_undegraded":"2024-04-25T16:52:34.872865+0000","last_fullsized":"2024-04-25T16:52:34.872865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340668+0000","last_change":"2024-04-25T16:52:26.832452+0000","last_active":"2024-04-25T16:52:35.340668+0000","last_peered":"2024-04-25T16:52:35.340668+0000","last_clean":"2024-04-25T16:52:35.340668+0000","last_became_active":"2024-04-25T16:52:26.831753+0000","last_became_peered":"2024-04-25T16:52:26.831753+0000","last_unstale":"2024-04-25T16:52:35.340668+0000","last_undegraded":"2024-04-25T16:52:35.340668+0000","last_fullsized":"2024-04-25T16:52:35.340668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780752+0000","last_change":"2024-04-25T16:52:26.805633+0000","last_active":"2024-04-25T16:52:27.780752+0000","last_peered":"2024-04-25T16:52:27.780752+0000","last_clean":"2024-04-25T16:52:27.780752+0000","last_became_active":"2024-04-25T16:52:26.804789+0000","last_became_peered":"2024-04-25T16:52:26.804789+0000","last_unstale":"2024-04-25T16:52:27.780752+0000","last_undegraded":"2024-04-25T16:52:27.780752+0000","last_fullsized":"2024-04-25T16:52:27.780752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:08.931465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875174+0000","last_change":"2024-04-25T16:52:26.798693+0000","last_active":"2024-04-25T16:52:34.875174+0000","last_peered":"2024-04-25T16:52:34.875174+0000","last_clean":"2024-04-25T16:52:34.875174+0000","last_became_active":"2024-04-25T16:52:26.797852+0000","last_became_peered":"2024-04-25T16:52:26.797852+0000","last_unstale":"2024-04-25T16:52:34.875174+0000","last_undegraded":"2024-04-25T16:52:34.875174+0000","last_fullsized":"2024-04-25T16:52:34.875174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831376+0000","last_change":"2024-04-25T16:52:26.819630+0000","last_active":"2024-04-25T16:52:27.831376+0000","last_peered":"2024-04-25T16:52:27.831376+0000","last_clean":"2024-04-25T16:52:27.831376+0000","last_became_active":"2024-04-25T16:52:26.819274+0000","last_became_peered":"2024-04-25T16:52:26.819274+0000","last_unstale":"2024-04-25T16:52:27.831376+0000","last_undegraded":"2024-04-25T16:52:27.831376+0000","last_fullsized":"2024-04-25T16:52:27.831376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873718+0000","last_change":"2024-04-25T16:52:26.791433+0000","last_active":"2024-04-25T16:52:34.873718+0000","last_peered":"2024-04-25T16:52:34.873718+0000","last_clean":"2024-04-25T16:52:34.873718+0000","last_became_active":"2024-04-25T16:52:26.790483+0000","last_became_peered":"2024-04-25T16:52:26.790483+0000","last_unstale":"2024-04-25T16:52:34.873718+0000","last_undegraded":"2024-04-25T16:52:34.873718+0000","last_fullsized":"2024-04-25T16:52:34.873718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877762+0000","last_change":"2024-04-25T16:52:26.804860+0000","last_active":"2024-04-25T16:52:34.877762+0000","last_peered":"2024-04-25T16:52:34.877762+0000","last_clean":"2024-04-25T16:52:34.877762+0000","last_became_active":"2024-04-25T16:52:26.804065+0000","last_became_peered":"2024-04-25T16:52:26.804065+0000","last_unstale":"2024-04-25T16:52:34.877762+0000","last_undegraded":"2024-04-25T16:52:34.877762+0000","last_fullsized":"2024-04-25T16:52:34.877762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873009+0000","last_change":"2024-04-25T16:52:26.806705+0000","last_active":"2024-04-25T16:52:34.873009+0000","last_peered":"2024-04-25T16:52:34.873009+0000","last_clean":"2024-04-25T16:52:34.873009+0000","last_became_active":"2024-04-25T16:52:26.805889+0000","last_became_peered":"2024-04-25T16:52:26.805889+0000","last_unstale":"2024-04-25T16:52:34.873009+0000","last_undegraded":"2024-04-25T16:52:34.873009+0000","last_fullsized":"2024-04-25T16:52:34.873009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874258+0000","last_change":"2024-04-25T16:52:26.798903+0000","last_active":"2024-04-25T16:52:34.874258+0000","last_peered":"2024-04-25T16:52:34.874258+0000","last_clean":"2024-04-25T16:52:34.874258+0000","last_became_active":"2024-04-25T16:52:26.798011+0000","last_became_peered":"2024-04-25T16:52:26.798011+0000","last_unstale":"2024-04-25T16:52:34.874258+0000","last_undegraded":"2024-04-25T16:52:34.874258+0000","last_fullsized":"2024-04-25T16:52:34.874258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340837+0000","last_change":"2024-04-25T16:52:26.841037+0000","last_active":"2024-04-25T16:52:35.340837+0000","last_peered":"2024-04-25T16:52:35.340837+0000","last_clean":"2024-04-25T16:52:35.340837+0000","last_became_active":"2024-04-25T16:52:26.838194+0000","last_became_peered":"2024-04-25T16:52:26.838194+0000","last_unstale":"2024-04-25T16:52:35.340837+0000","last_undegraded":"2024-04-25T16:52:35.340837+0000","last_fullsized":"2024-04-25T16:52:35.340837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830883+0000","last_change":"2024-04-25T16:52:26.813243+0000","last_active":"2024-04-25T16:52:27.830883+0000","last_peered":"2024-04-25T16:52:27.830883+0000","last_clean":"2024-04-25T16:52:27.830883+0000","last_became_active":"2024-04-25T16:52:26.811807+0000","last_became_peered":"2024-04-25T16:52:26.811807+0000","last_unstale":"2024-04-25T16:52:27.830883+0000","last_undegraded":"2024-04-25T16:52:27.830883+0000","last_fullsized":"2024-04-25T16:52:27.830883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:42.553053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341094+0000","last_change":"2024-04-25T16:52:26.841017+0000","last_active":"2024-04-25T16:52:35.341094+0000","last_peered":"2024-04-25T16:52:35.341094+0000","last_clean":"2024-04-25T16:52:35.341094+0000","last_became_active":"2024-04-25T16:52:26.837543+0000","last_became_peered":"2024-04-25T16:52:26.837543+0000","last_unstale":"2024-04-25T16:52:35.341094+0000","last_undegraded":"2024-04-25T16:52:35.341094+0000","last_fullsized":"2024-04-25T16:52:35.341094+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875394+0000","last_change":"2024-04-25T16:52:25.818069+0000","last_active":"2024-04-25T16:52:34.875394+0000","last_peered":"2024-04-25T16:52:34.875394+0000","last_clean":"2024-04-25T16:52:34.875394+0000","last_became_active":"2024-04-25T16:52:25.817362+0000","last_became_peered":"2024-04-25T16:52:25.817362+0000","last_unstale":"2024-04-25T16:52:34.875394+0000","last_undegraded":"2024-04-25T16:52:34.875394+0000","last_fullsized":"2024-04-25T16:52:34.875394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:08.488136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833310+0000","last_change":"2024-04-25T16:52:26.829008+0000","last_active":"2024-04-25T16:52:27.833310+0000","last_peered":"2024-04-25T16:52:27.833310+0000","last_clean":"2024-04-25T16:52:27.833310+0000","last_became_active":"2024-04-25T16:52:26.828706+0000","last_became_peered":"2024-04-25T16:52:26.828706+0000","last_unstale":"2024-04-25T16:52:27.833310+0000","last_undegraded":"2024-04-25T16:52:27.833310+0000","last_fullsized":"2024-04-25T16:52:27.833310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878876+0000","last_change":"2024-04-25T16:52:25.795967+0000","last_active":"2024-04-25T16:52:34.878876+0000","last_peered":"2024-04-25T16:52:34.878876+0000","last_clean":"2024-04-25T16:52:34.878876+0000","last_became_active":"2024-04-25T16:52:25.795695+0000","last_became_peered":"2024-04-25T16:52:25.795695+0000","last_unstale":"2024-04-25T16:52:34.878876+0000","last_undegraded":"2024-04-25T16:52:34.878876+0000","last_fullsized":"2024-04-25T16:52:34.878876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875243+0000","last_change":"2024-04-25T16:52:26.789430+0000","last_active":"2024-04-25T16:52:34.875243+0000","last_peered":"2024-04-25T16:52:34.875243+0000","last_clean":"2024-04-25T16:52:34.875243+0000","last_became_active":"2024-04-25T16:52:26.782243+0000","last_became_peered":"2024-04-25T16:52:26.782243+0000","last_unstale":"2024-04-25T16:52:34.875243+0000","last_undegraded":"2024-04-25T16:52:34.875243+0000","last_fullsized":"2024-04-25T16:52:34.875243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833428+0000","last_change":"2024-04-25T16:52:25.819162+0000","last_active":"2024-04-25T16:52:27.833428+0000","last_peered":"2024-04-25T16:52:27.833428+0000","last_clean":"2024-04-25T16:52:27.833428+0000","last_became_active":"2024-04-25T16:52:25.818859+0000","last_became_peered":"2024-04-25T16:52:25.818859+0000","last_unstale":"2024-04-25T16:52:27.833428+0000","last_undegraded":"2024-04-25T16:52:27.833428+0000","last_fullsized":"2024-04-25T16:52:27.833428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874082+0000","last_change":"2024-04-25T16:52:26.805376+0000","last_active":"2024-04-25T16:52:34.874082+0000","last_peered":"2024-04-25T16:52:34.874082+0000","last_clean":"2024-04-25T16:52:34.874082+0000","last_became_active":"2024-04-25T16:52:26.804542+0000","last_became_peered":"2024-04-25T16:52:26.804542+0000","last_unstale":"2024-04-25T16:52:34.874082+0000","last_undegraded":"2024-04-25T16:52:34.874082+0000","last_fullsized":"2024-04-25T16:52:34.874082+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.620224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874004+0000","last_change":"2024-04-25T16:52:25.815751+0000","last_active":"2024-04-25T16:52:34.874004+0000","last_peered":"2024-04-25T16:52:34.874004+0000","last_clean":"2024-04-25T16:52:34.874004+0000","last_became_active":"2024-04-25T16:52:25.814927+0000","last_became_peered":"2024-04-25T16:52:25.814927+0000","last_unstale":"2024-04-25T16:52:34.874004+0000","last_undegraded":"2024-04-25T16:52:34.874004+0000","last_fullsized":"2024-04-25T16:52:34.874004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877461+0000","last_change":"2024-04-25T16:52:26.814640+0000","last_active":"2024-04-25T16:52:34.877461+0000","last_peered":"2024-04-25T16:52:34.877461+0000","last_clean":"2024-04-25T16:52:34.877461+0000","last_became_active":"2024-04-25T16:52:26.814016+0000","last_became_peered":"2024-04-25T16:52:26.814016+0000","last_unstale":"2024-04-25T16:52:34.877461+0000","last_undegraded":"2024-04-25T16:52:34.877461+0000","last_fullsized":"2024-04-25T16:52:34.877461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877431+0000","last_change":"2024-04-25T16:52:25.805323+0000","last_active":"2024-04-25T16:52:34.877431+0000","last_peered":"2024-04-25T16:52:34.877431+0000","last_clean":"2024-04-25T16:52:34.877431+0000","last_became_active":"2024-04-25T16:52:25.796444+0000","last_became_peered":"2024-04-25T16:52:25.796444+0000","last_unstale":"2024-04-25T16:52:34.877431+0000","last_undegraded":"2024-04-25T16:52:34.877431+0000","last_fullsized":"2024-04-25T16:52:34.877431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:28.996351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877181+0000","last_change":"2024-04-25T16:52:26.821648+0000","last_active":"2024-04-25T16:52:34.877181+0000","last_peered":"2024-04-25T16:52:34.877181+0000","last_clean":"2024-04-25T16:52:34.877181+0000","last_became_active":"2024-04-25T16:52:26.820835+0000","last_became_peered":"2024-04-25T16:52:26.820835+0000","last_unstale":"2024-04-25T16:52:34.877181+0000","last_undegraded":"2024-04-25T16:52:34.877181+0000","last_fullsized":"2024-04-25T16:52:34.877181+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340828+0000","last_change":"2024-04-25T16:52:25.816250+0000","last_active":"2024-04-25T16:52:35.340828+0000","last_peered":"2024-04-25T16:52:35.340828+0000","last_clean":"2024-04-25T16:52:35.340828+0000","last_became_active":"2024-04-25T16:52:25.815656+0000","last_became_peered":"2024-04-25T16:52:25.815656+0000","last_unstale":"2024-04-25T16:52:35.340828+0000","last_undegraded":"2024-04-25T16:52:35.340828+0000","last_fullsized":"2024-04-25T16:52:35.340828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872461+0000","last_change":"2024-04-25T16:52:26.813425+0000","last_active":"2024-04-25T16:52:34.872461+0000","last_peered":"2024-04-25T16:52:34.872461+0000","last_clean":"2024-04-25T16:52:34.872461+0000","last_became_active":"2024-04-25T16:52:26.812713+0000","last_became_peered":"2024-04-25T16:52:26.812713+0000","last_unstale":"2024-04-25T16:52:34.872461+0000","last_undegraded":"2024-04-25T16:52:34.872461+0000","last_fullsized":"2024-04-25T16:52:34.872461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831072+0000","last_change":"2024-04-25T16:52:25.803265+0000","last_active":"2024-04-25T16:52:27.831072+0000","last_peered":"2024-04-25T16:52:27.831072+0000","last_clean":"2024-04-25T16:52:27.831072+0000","last_became_active":"2024-04-25T16:52:25.802617+0000","last_became_peered":"2024-04-25T16:52:25.802617+0000","last_unstale":"2024-04-25T16:52:27.831072+0000","last_undegraded":"2024-04-25T16:52:27.831072+0000","last_fullsized":"2024-04-25T16:52:27.831072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872313+0000","last_change":"2024-04-25T16:52:26.819267+0000","last_active":"2024-04-25T16:52:34.872313+0000","last_peered":"2024-04-25T16:52:34.872313+0000","last_clean":"2024-04-25T16:52:34.872313+0000","last_became_active":"2024-04-25T16:52:26.818740+0000","last_became_peered":"2024-04-25T16:52:26.818740+0000","last_unstale":"2024-04-25T16:52:34.872313+0000","last_undegraded":"2024-04-25T16:52:34.872313+0000","last_fullsized":"2024-04-25T16:52:34.872313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871955+0000","last_change":"2024-04-25T16:52:25.816663+0000","last_active":"2024-04-25T16:52:34.871955+0000","last_peered":"2024-04-25T16:52:34.871955+0000","last_clean":"2024-04-25T16:52:34.871955+0000","last_became_active":"2024-04-25T16:52:25.815420+0000","last_became_peered":"2024-04-25T16:52:25.815420+0000","last_unstale":"2024-04-25T16:52:34.871955+0000","last_undegraded":"2024-04-25T16:52:34.871955+0000","last_fullsized":"2024-04-25T16:52:34.871955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:30.288538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782313+0000","last_change":"2024-04-25T16:52:26.789927+0000","last_active":"2024-04-25T16:52:27.782313+0000","last_peered":"2024-04-25T16:52:27.782313+0000","last_clean":"2024-04-25T16:52:27.782313+0000","last_became_active":"2024-04-25T16:52:26.788014+0000","last_became_peered":"2024-04-25T16:52:26.788014+0000","last_unstale":"2024-04-25T16:52:27.782313+0000","last_undegraded":"2024-04-25T16:52:27.782313+0000","last_fullsized":"2024-04-25T16:52:27.782313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:47.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873885+0000","last_change":"2024-04-25T16:52:25.798974+0000","last_active":"2024-04-25T16:52:34.873885+0000","last_peered":"2024-04-25T16:52:34.873885+0000","last_clean":"2024-04-25T16:52:34.873885+0000","last_became_active":"2024-04-25T16:52:25.798314+0000","last_became_peered":"2024-04-25T16:52:25.798314+0000","last_unstale":"2024-04-25T16:52:34.873885+0000","last_undegraded":"2024-04-25T16:52:34.873885+0000","last_fullsized":"2024-04-25T16:52:34.873885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.348530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341450+0000","last_change":"2024-04-25T16:52:26.842485+0000","last_active":"2024-04-25T16:52:35.341450+0000","last_peered":"2024-04-25T16:52:35.341450+0000","last_clean":"2024-04-25T16:52:35.341450+0000","last_became_active":"2024-04-25T16:52:26.838813+0000","last_became_peered":"2024-04-25T16:52:26.838813+0000","last_unstale":"2024-04-25T16:52:35.341450+0000","last_undegraded":"2024-04-25T16:52:35.341450+0000","last_fullsized":"2024-04-25T16:52:35.341450+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878521+0000","last_change":"2024-04-25T16:52:25.803289+0000","last_active":"2024-04-25T16:52:34.878521+0000","last_peered":"2024-04-25T16:52:34.878521+0000","last_clean":"2024-04-25T16:52:34.878521+0000","last_became_active":"2024-04-25T16:52:25.801415+0000","last_became_peered":"2024-04-25T16:52:25.801415+0000","last_unstale":"2024-04-25T16:52:34.878521+0000","last_undegraded":"2024-04-25T16:52:34.878521+0000","last_fullsized":"2024-04-25T16:52:34.878521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:45.310089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878182+0000","last_change":"2024-04-25T16:52:26.840379+0000","last_active":"2024-04-25T16:52:34.878182+0000","last_peered":"2024-04-25T16:52:34.878182+0000","last_clean":"2024-04-25T16:52:34.878182+0000","last_became_active":"2024-04-25T16:52:26.840062+0000","last_became_peered":"2024-04-25T16:52:26.840062+0000","last_unstale":"2024-04-25T16:52:34.878182+0000","last_undegraded":"2024-04-25T16:52:34.878182+0000","last_fullsized":"2024-04-25T16:52:34.878182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878024+0000","last_change":"2024-04-25T16:52:25.809391+0000","last_active":"2024-04-25T16:52:34.878024+0000","last_peered":"2024-04-25T16:52:34.878024+0000","last_clean":"2024-04-25T16:52:34.878024+0000","last_became_active":"2024-04-25T16:52:25.808455+0000","last_became_peered":"2024-04-25T16:52:25.808455+0000","last_unstale":"2024-04-25T16:52:34.878024+0000","last_undegraded":"2024-04-25T16:52:34.878024+0000","last_fullsized":"2024-04-25T16:52:34.878024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872072+0000","last_change":"2024-04-25T16:52:26.782954+0000","last_active":"2024-04-25T16:52:34.872072+0000","last_peered":"2024-04-25T16:52:34.872072+0000","last_clean":"2024-04-25T16:52:34.872072+0000","last_became_active":"2024-04-25T16:52:26.782108+0000","last_became_peered":"2024-04-25T16:52:26.782108+0000","last_unstale":"2024-04-25T16:52:34.872072+0000","last_undegraded":"2024-04-25T16:52:34.872072+0000","last_fullsized":"2024-04-25T16:52:34.872072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873363+0000","last_change":"2024-04-25T16:52:25.795020+0000","last_active":"2024-04-25T16:52:34.873363+0000","last_peered":"2024-04-25T16:52:34.873363+0000","last_clean":"2024-04-25T16:52:34.873363+0000","last_became_active":"2024-04-25T16:52:25.794633+0000","last_became_peered":"2024-04-25T16:52:25.794633+0000","last_unstale":"2024-04-25T16:52:34.873363+0000","last_undegraded":"2024-04-25T16:52:34.873363+0000","last_fullsized":"2024-04-25T16:52:34.873363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:38.694859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877510+0000","last_change":"2024-04-25T16:52:26.821081+0000","last_active":"2024-04-25T16:52:34.877510+0000","last_peered":"2024-04-25T16:52:34.877510+0000","last_clean":"2024-04-25T16:52:34.877510+0000","last_became_active":"2024-04-25T16:52:26.819337+0000","last_became_peered":"2024-04-25T16:52:26.819337+0000","last_unstale":"2024-04-25T16:52:34.877510+0000","last_undegraded":"2024-04-25T16:52:34.877510+0000","last_fullsized":"2024-04-25T16:52:34.877510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872999+0000","last_change":"2024-04-25T16:52:25.793039+0000","last_active":"2024-04-25T16:52:34.872999+0000","last_peered":"2024-04-25T16:52:34.872999+0000","last_clean":"2024-04-25T16:52:34.872999+0000","last_became_active":"2024-04-25T16:52:25.792552+0000","last_became_peered":"2024-04-25T16:52:25.792552+0000","last_unstale":"2024-04-25T16:52:34.872999+0000","last_undegraded":"2024-04-25T16:52:34.872999+0000","last_fullsized":"2024-04-25T16:52:34.872999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:30.954700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872507+0000","last_change":"2024-04-25T16:52:26.823643+0000","last_active":"2024-04-25T16:52:34.872507+0000","last_peered":"2024-04-25T16:52:34.872507+0000","last_clean":"2024-04-25T16:52:34.872507+0000","last_became_active":"2024-04-25T16:52:26.823348+0000","last_became_peered":"2024-04-25T16:52:26.823348+0000","last_unstale":"2024-04-25T16:52:34.872507+0000","last_undegraded":"2024-04-25T16:52:34.872507+0000","last_fullsized":"2024-04-25T16:52:34.872507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830926+0000","last_change":"2024-04-25T16:52:25.813706+0000","last_active":"2024-04-25T16:52:27.830926+0000","last_peered":"2024-04-25T16:52:27.830926+0000","last_clean":"2024-04-25T16:52:27.830926+0000","last_became_active":"2024-04-25T16:52:25.813387+0000","last_became_peered":"2024-04-25T16:52:25.813387+0000","last_unstale":"2024-04-25T16:52:27.830926+0000","last_undegraded":"2024-04-25T16:52:27.830926+0000","last_fullsized":"2024-04-25T16:52:27.830926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340755+0000","last_change":"2024-04-25T16:52:26.817332+0000","last_active":"2024-04-25T16:52:35.340755+0000","last_peered":"2024-04-25T16:52:35.340755+0000","last_clean":"2024-04-25T16:52:35.340755+0000","last_became_active":"2024-04-25T16:52:26.816020+0000","last_became_peered":"2024-04-25T16:52:26.816020+0000","last_unstale":"2024-04-25T16:52:35.340755+0000","last_undegraded":"2024-04-25T16:52:35.340755+0000","last_fullsized":"2024-04-25T16:52:35.340755+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872306+0000","last_change":"2024-04-25T16:52:25.815545+0000","last_active":"2024-04-25T16:52:34.872306+0000","last_peered":"2024-04-25T16:52:34.872306+0000","last_clean":"2024-04-25T16:52:34.872306+0000","last_became_active":"2024-04-25T16:52:25.814711+0000","last_became_peered":"2024-04-25T16:52:25.814711+0000","last_unstale":"2024-04-25T16:52:34.872306+0000","last_undegraded":"2024-04-25T16:52:34.872306+0000","last_fullsized":"2024-04-25T16:52:34.872306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872682+0000","last_change":"2024-04-25T16:52:26.826567+0000","last_active":"2024-04-25T16:52:34.872682+0000","last_peered":"2024-04-25T16:52:34.872682+0000","last_clean":"2024-04-25T16:52:34.872682+0000","last_became_active":"2024-04-25T16:52:26.826224+0000","last_became_peered":"2024-04-25T16:52:26.826224+0000","last_unstale":"2024-04-25T16:52:34.872682+0000","last_undegraded":"2024-04-25T16:52:34.872682+0000","last_fullsized":"2024-04-25T16:52:34.872682+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:06.823841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":24,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340795+0000","last_change":"2024-04-25T16:52:25.815153+0000","last_active":"2024-04-25T16:52:35.340795+0000","last_peered":"2024-04-25T16:52:35.340795+0000","last_clean":"2024-04-25T16:52:35.340795+0000","last_became_active":"2024-04-25T16:52:25.814360+0000","last_became_peered":"2024-04-25T16:52:25.814360+0000","last_unstale":"2024-04-25T16:52:35.340795+0000","last_undegraded":"2024-04-25T16:52:35.340795+0000","last_fullsized":"2024-04-25T16:52:35.340795+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873169+0000","last_change":"2024-04-25T16:52:26.804704+0000","last_active":"2024-04-25T16:52:34.873169+0000","last_peered":"2024-04-25T16:52:34.873169+0000","last_clean":"2024-04-25T16:52:34.873169+0000","last_became_active":"2024-04-25T16:52:26.803888+0000","last_became_peered":"2024-04-25T16:52:26.803888+0000","last_unstale":"2024-04-25T16:52:34.873169+0000","last_undegraded":"2024-04-25T16:52:34.873169+0000","last_fullsized":"2024-04-25T16:52:34.873169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.640204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878107+0000","last_change":"2024-04-25T16:52:25.818880+0000","last_active":"2024-04-25T16:52:34.878107+0000","last_peered":"2024-04-25T16:52:34.878107+0000","last_clean":"2024-04-25T16:52:34.878107+0000","last_became_active":"2024-04-25T16:52:25.817841+0000","last_became_peered":"2024-04-25T16:52:25.817841+0000","last_unstale":"2024-04-25T16:52:34.878107+0000","last_undegraded":"2024-04-25T16:52:34.878107+0000","last_fullsized":"2024-04-25T16:52:34.878107+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:26.817110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877704+0000","last_change":"2024-04-25T16:52:26.809554+0000","last_active":"2024-04-25T16:52:34.877704+0000","last_peered":"2024-04-25T16:52:34.877704+0000","last_clean":"2024-04-25T16:52:34.877704+0000","last_became_active":"2024-04-25T16:52:26.808709+0000","last_became_peered":"2024-04-25T16:52:26.808709+0000","last_unstale":"2024-04-25T16:52:34.877704+0000","last_undegraded":"2024-04-25T16:52:34.877704+0000","last_fullsized":"2024-04-25T16:52:34.877704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875584+0000","last_change":"2024-04-25T16:52:25.812269+0000","last_active":"2024-04-25T16:52:34.875584+0000","last_peered":"2024-04-25T16:52:34.875584+0000","last_clean":"2024-04-25T16:52:34.875584+0000","last_became_active":"2024-04-25T16:52:25.810831+0000","last_became_peered":"2024-04-25T16:52:25.810831+0000","last_unstale":"2024-04-25T16:52:34.875584+0000","last_undegraded":"2024-04-25T16:52:34.875584+0000","last_fullsized":"2024-04-25T16:52:34.875584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874980+0000","last_change":"2024-04-25T16:52:26.806939+0000","last_active":"2024-04-25T16:52:34.874980+0000","last_peered":"2024-04-25T16:52:34.874980+0000","last_clean":"2024-04-25T16:52:34.874980+0000","last_became_active":"2024-04-25T16:52:26.806053+0000","last_became_peered":"2024-04-25T16:52:26.806053+0000","last_unstale":"2024-04-25T16:52:34.874980+0000","last_undegraded":"2024-04-25T16:52:34.874980+0000","last_fullsized":"2024-04-25T16:52:34.874980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781168+0000","last_change":"2024-04-25T16:52:25.783759+0000","last_active":"2024-04-25T16:52:27.781168+0000","last_peered":"2024-04-25T16:52:27.781168+0000","last_clean":"2024-04-25T16:52:27.781168+0000","last_became_active":"2024-04-25T16:52:25.783353+0000","last_became_peered":"2024-04-25T16:52:25.783353+0000","last_unstale":"2024-04-25T16:52:27.781168+0000","last_undegraded":"2024-04-25T16:52:27.781168+0000","last_fullsized":"2024-04-25T16:52:27.781168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:34.281502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780945+0000","last_change":"2024-04-25T16:52:26.798434+0000","last_active":"2024-04-25T16:52:27.780945+0000","last_peered":"2024-04-25T16:52:27.780945+0000","last_clean":"2024-04-25T16:52:27.780945+0000","last_became_active":"2024-04-25T16:52:26.797631+0000","last_became_peered":"2024-04-25T16:52:26.797631+0000","last_unstale":"2024-04-25T16:52:27.780945+0000","last_undegraded":"2024-04-25T16:52:27.780945+0000","last_fullsized":"2024-04-25T16:52:27.780945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:16.248387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831612+0000","last_change":"2024-04-25T16:52:25.819136+0000","last_active":"2024-04-25T16:52:27.831612+0000","last_peered":"2024-04-25T16:52:27.831612+0000","last_clean":"2024-04-25T16:52:27.831612+0000","last_became_active":"2024-04-25T16:52:25.818809+0000","last_became_peered":"2024-04-25T16:52:25.818809+0000","last_unstale":"2024-04-25T16:52:27.831612+0000","last_undegraded":"2024-04-25T16:52:27.831612+0000","last_fullsized":"2024-04-25T16:52:27.831612+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341292+0000","last_change":"2024-04-25T16:52:26.819612+0000","last_active":"2024-04-25T16:52:35.341292+0000","last_peered":"2024-04-25T16:52:35.341292+0000","last_clean":"2024-04-25T16:52:35.341292+0000","last_became_active":"2024-04-25T16:52:26.818874+0000","last_became_peered":"2024-04-25T16:52:26.818874+0000","last_unstale":"2024-04-25T16:52:35.341292+0000","last_undegraded":"2024-04-25T16:52:35.341292+0000","last_fullsized":"2024-04-25T16:52:35.341292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781486+0000","last_change":"2024-04-25T16:52:25.780874+0000","last_active":"2024-04-25T16:52:27.781486+0000","last_peered":"2024-04-25T16:52:27.781486+0000","last_clean":"2024-04-25T16:52:27.781486+0000","last_became_active":"2024-04-25T16:52:25.780529+0000","last_became_peered":"2024-04-25T16:52:25.780529+0000","last_unstale":"2024-04-25T16:52:27.781486+0000","last_undegraded":"2024-04-25T16:52:27.781486+0000","last_fullsized":"2024-04-25T16:52:27.781486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:57.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878077+0000","last_change":"2024-04-25T16:52:26.845027+0000","last_active":"2024-04-25T16:52:34.878077+0000","last_peered":"2024-04-25T16:52:34.878077+0000","last_clean":"2024-04-25T16:52:34.878077+0000","last_became_active":"2024-04-25T16:52:26.844620+0000","last_became_peered":"2024-04-25T16:52:26.844620+0000","last_unstale":"2024-04-25T16:52:34.878077+0000","last_undegraded":"2024-04-25T16:52:34.878077+0000","last_fullsized":"2024-04-25T16:52:34.878077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780947+0000","last_change":"2024-04-25T16:52:25.821889+0000","last_active":"2024-04-25T16:52:27.780947+0000","last_peered":"2024-04-25T16:52:27.780947+0000","last_clean":"2024-04-25T16:52:27.780947+0000","last_became_active":"2024-04-25T16:52:25.821643+0000","last_became_peered":"2024-04-25T16:52:25.821643+0000","last_unstale":"2024-04-25T16:52:27.780947+0000","last_undegraded":"2024-04-25T16:52:27.780947+0000","last_fullsized":"2024-04-25T16:52:27.780947+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:27.786669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877414+0000","last_change":"2024-04-25T16:52:26.802864+0000","last_active":"2024-04-25T16:52:34.877414+0000","last_peered":"2024-04-25T16:52:34.877414+0000","last_clean":"2024-04-25T16:52:34.877414+0000","last_became_active":"2024-04-25T16:52:26.801913+0000","last_became_peered":"2024-04-25T16:52:26.801913+0000","last_unstale":"2024-04-25T16:52:34.877414+0000","last_undegraded":"2024-04-25T16:52:34.877414+0000","last_fullsized":"2024-04-25T16:52:34.877414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341042+0000","last_change":"2024-04-25T16:52:25.806124+0000","last_active":"2024-04-25T16:52:35.341042+0000","last_peered":"2024-04-25T16:52:35.341042+0000","last_clean":"2024-04-25T16:52:35.341042+0000","last_became_active":"2024-04-25T16:52:25.805306+0000","last_became_peered":"2024-04-25T16:52:25.805306+0000","last_unstale":"2024-04-25T16:52:35.341042+0000","last_undegraded":"2024-04-25T16:52:35.341042+0000","last_fullsized":"2024-04-25T16:52:35.341042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872688+0000","last_change":"2024-04-25T16:52:26.819372+0000","last_active":"2024-04-25T16:52:34.872688+0000","last_peered":"2024-04-25T16:52:34.872688+0000","last_clean":"2024-04-25T16:52:34.872688+0000","last_became_active":"2024-04-25T16:52:26.807182+0000","last_became_peered":"2024-04-25T16:52:26.807182+0000","last_unstale":"2024-04-25T16:52:34.872688+0000","last_undegraded":"2024-04-25T16:52:34.872688+0000","last_fullsized":"2024-04-25T16:52:34.872688+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:14.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780505+0000","last_change":"2024-04-25T16:52:25.788959+0000","last_active":"2024-04-25T16:52:27.780505+0000","last_peered":"2024-04-25T16:52:27.780505+0000","last_clean":"2024-04-25T16:52:27.780505+0000","last_became_active":"2024-04-25T16:52:25.788261+0000","last_became_peered":"2024-04-25T16:52:25.788261+0000","last_unstale":"2024-04-25T16:52:27.780505+0000","last_undegraded":"2024-04-25T16:52:27.780505+0000","last_fullsized":"2024-04-25T16:52:27.780505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:13.943131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831765+0000","last_change":"2024-04-25T16:52:26.819641+0000","last_active":"2024-04-25T16:52:27.831765+0000","last_peered":"2024-04-25T16:52:27.831765+0000","last_clean":"2024-04-25T16:52:27.831765+0000","last_became_active":"2024-04-25T16:52:26.819314+0000","last_became_peered":"2024-04-25T16:52:26.819314+0000","last_unstale":"2024-04-25T16:52:27.831765+0000","last_undegraded":"2024-04-25T16:52:27.831765+0000","last_fullsized":"2024-04-25T16:52:27.831765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878486+0000","last_change":"2024-04-25T16:52:25.806269+0000","last_active":"2024-04-25T16:52:34.878486+0000","last_peered":"2024-04-25T16:52:34.878486+0000","last_clean":"2024-04-25T16:52:34.878486+0000","last_became_active":"2024-04-25T16:52:25.804183+0000","last_became_peered":"2024-04-25T16:52:25.804183+0000","last_unstale":"2024-04-25T16:52:34.878486+0000","last_undegraded":"2024-04-25T16:52:34.878486+0000","last_fullsized":"2024-04-25T16:52:34.878486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:07.902516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341557+0000","last_change":"2024-04-25T16:52:26.841753+0000","last_active":"2024-04-25T16:52:35.341557+0000","last_peered":"2024-04-25T16:52:35.341557+0000","last_clean":"2024-04-25T16:52:35.341557+0000","last_became_active":"2024-04-25T16:52:26.841112+0000","last_became_peered":"2024-04-25T16:52:26.841112+0000","last_unstale":"2024-04-25T16:52:35.341557+0000","last_undegraded":"2024-04-25T16:52:35.341557+0000","last_fullsized":"2024-04-25T16:52:35.341557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781212+0000","last_change":"2024-04-25T16:52:25.803168+0000","last_active":"2024-04-25T16:52:27.781212+0000","last_peered":"2024-04-25T16:52:27.781212+0000","last_clean":"2024-04-25T16:52:27.781212+0000","last_became_active":"2024-04-25T16:52:25.802880+0000","last_became_peered":"2024-04-25T16:52:25.802880+0000","last_unstale":"2024-04-25T16:52:27.781212+0000","last_undegraded":"2024-04-25T16:52:27.781212+0000","last_fullsized":"2024-04-25T16:52:27.781212+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.210482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873797+0000","last_change":"2024-04-25T16:52:26.819781+0000","last_active":"2024-04-25T16:52:34.873797+0000","last_peered":"2024-04-25T16:52:34.873797+0000","last_clean":"2024-04-25T16:52:34.873797+0000","last_became_active":"2024-04-25T16:52:26.819467+0000","last_became_peered":"2024-04-25T16:52:26.819467+0000","last_unstale":"2024-04-25T16:52:34.873797+0000","last_undegraded":"2024-04-25T16:52:34.873797+0000","last_fullsized":"2024-04-25T16:52:34.873797+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878534+0000","last_change":"2024-04-25T16:52:25.799953+0000","last_active":"2024-04-25T16:52:34.878534+0000","last_peered":"2024-04-25T16:52:34.878534+0000","last_clean":"2024-04-25T16:52:34.878534+0000","last_became_active":"2024-04-25T16:52:25.799689+0000","last_became_peered":"2024-04-25T16:52:25.799689+0000","last_unstale":"2024-04-25T16:52:34.878534+0000","last_undegraded":"2024-04-25T16:52:34.878534+0000","last_fullsized":"2024-04-25T16:52:34.878534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877833+0000","last_change":"2024-04-25T16:52:26.799367+0000","last_active":"2024-04-25T16:52:34.877833+0000","last_peered":"2024-04-25T16:52:34.877833+0000","last_clean":"2024-04-25T16:52:34.877833+0000","last_became_active":"2024-04-25T16:52:26.797337+0000","last_became_peered":"2024-04-25T16:52:26.797337+0000","last_unstale":"2024-04-25T16:52:34.877833+0000","last_undegraded":"2024-04-25T16:52:34.877833+0000","last_fullsized":"2024-04-25T16:52:34.877833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878430+0000","last_change":"2024-04-25T16:52:25.806956+0000","last_active":"2024-04-25T16:52:34.878430+0000","last_peered":"2024-04-25T16:52:34.878430+0000","last_clean":"2024-04-25T16:52:34.878430+0000","last_became_active":"2024-04-25T16:52:25.806604+0000","last_became_peered":"2024-04-25T16:52:25.806604+0000","last_unstale":"2024-04-25T16:52:34.878430+0000","last_undegraded":"2024-04-25T16:52:34.878430+0000","last_fullsized":"2024-04-25T16:52:34.878430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831008+0000","last_change":"2024-04-25T16:52:26.820845+0000","last_active":"2024-04-25T16:52:27.831008+0000","last_peered":"2024-04-25T16:52:27.831008+0000","last_clean":"2024-04-25T16:52:27.831008+0000","last_became_active":"2024-04-25T16:52:26.820484+0000","last_became_peered":"2024-04-25T16:52:26.820484+0000","last_unstale":"2024-04-25T16:52:27.831008+0000","last_undegraded":"2024-04-25T16:52:27.831008+0000","last_fullsized":"2024-04-25T16:52:27.831008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:44.359144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874630+0000","last_change":"2024-04-25T16:52:25.785723+0000","last_active":"2024-04-25T16:52:34.874630+0000","last_peered":"2024-04-25T16:52:34.874630+0000","last_clean":"2024-04-25T16:52:34.874630+0000","last_became_active":"2024-04-25T16:52:25.784930+0000","last_became_peered":"2024-04-25T16:52:25.784930+0000","last_unstale":"2024-04-25T16:52:34.874630+0000","last_undegraded":"2024-04-25T16:52:34.874630+0000","last_fullsized":"2024-04-25T16:52:34.874630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872198+0000","last_change":"2024-04-25T16:52:26.799277+0000","last_active":"2024-04-25T16:52:34.872198+0000","last_peered":"2024-04-25T16:52:34.872198+0000","last_clean":"2024-04-25T16:52:34.872198+0000","last_became_active":"2024-04-25T16:52:26.797817+0000","last_became_peered":"2024-04-25T16:52:26.797817+0000","last_unstale":"2024-04-25T16:52:34.872198+0000","last_undegraded":"2024-04-25T16:52:34.872198+0000","last_fullsized":"2024-04-25T16:52:34.872198+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871616+0000","last_change":"2024-04-25T16:52:25.810628+0000","last_active":"2024-04-25T16:52:34.871616+0000","last_peered":"2024-04-25T16:52:34.871616+0000","last_clean":"2024-04-25T16:52:34.871616+0000","last_became_active":"2024-04-25T16:52:25.809669+0000","last_became_peered":"2024-04-25T16:52:25.809669+0000","last_unstale":"2024-04-25T16:52:34.871616+0000","last_undegraded":"2024-04-25T16:52:34.871616+0000","last_fullsized":"2024-04-25T16:52:34.871616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:36.638682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341549+0000","last_change":"2024-04-25T16:52:26.832124+0000","last_active":"2024-04-25T16:52:35.341549+0000","last_peered":"2024-04-25T16:52:35.341549+0000","last_clean":"2024-04-25T16:52:35.341549+0000","last_became_active":"2024-04-25T16:52:26.831437+0000","last_became_peered":"2024-04-25T16:52:26.831437+0000","last_unstale":"2024-04-25T16:52:35.341549+0000","last_undegraded":"2024-04-25T16:52:35.341549+0000","last_fullsized":"2024-04-25T16:52:35.341549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872502+0000","last_change":"2024-04-25T16:52:25.793944+0000","last_active":"2024-04-25T16:52:34.872502+0000","last_peered":"2024-04-25T16:52:34.872502+0000","last_clean":"2024-04-25T16:52:34.872502+0000","last_became_active":"2024-04-25T16:52:25.793522+0000","last_became_peered":"2024-04-25T16:52:25.793522+0000","last_unstale":"2024-04-25T16:52:34.872502+0000","last_undegraded":"2024-04-25T16:52:34.872502+0000","last_fullsized":"2024-04-25T16:52:34.872502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:41.542439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877722+0000","last_change":"2024-04-25T16:52:26.804643+0000","last_active":"2024-04-25T16:52:34.877722+0000","last_peered":"2024-04-25T16:52:34.877722+0000","last_clean":"2024-04-25T16:52:34.877722+0000","last_became_active":"2024-04-25T16:52:26.802297+0000","last_became_peered":"2024-04-25T16:52:26.802297+0000","last_unstale":"2024-04-25T16:52:34.877722+0000","last_undegraded":"2024-04-25T16:52:34.877722+0000","last_fullsized":"2024-04-25T16:52:34.877722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341255+0000","last_change":"2024-04-25T16:52:25.796075+0000","last_active":"2024-04-25T16:52:35.341255+0000","last_peered":"2024-04-25T16:52:35.341255+0000","last_clean":"2024-04-25T16:52:35.341255+0000","last_became_active":"2024-04-25T16:52:25.795226+0000","last_became_peered":"2024-04-25T16:52:25.795226+0000","last_unstale":"2024-04-25T16:52:35.341255+0000","last_undegraded":"2024-04-25T16:52:35.341255+0000","last_fullsized":"2024-04-25T16:52:35.341255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":145,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340402+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:52:35.340402+0000","last_peered":"2024-04-25T16:52:35.340402+0000","last_clean":"2024-04-25T16:52:35.340402+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:52:35.340402+0000","last_undegraded":"2024-04-25T16:52:35.340402+0000","last_fullsized":"2024-04-25T16:52:35.340402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":196969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":196499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":458752,"data_stored":196991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35796,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336044,"statfs":{"total":96636764160,"available":96600109056,"internally_reserved":0,"allocated":466944,"data_stored":198033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":190098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35776,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336064,"statfs":{"total":96636764160,"available":96600129536,"internally_reserved":0,"allocated":442368,"data_stored":190118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335476,"statfs":{"total":96636764160,"available":96599527424,"internally_reserved":0,"allocated":1048576,"data_stored":786789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:52:40.461 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:52:40.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.639+0000 7f6668bf9640 1 Processor -- start 2024-04-25T16:52:40.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.639+0000 7f6668bf9640 1 -- start start 2024-04-25T16:52:40.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f6668bf9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6664070080 0x7f666415e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f6668bf9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666415e7c0 0x7f666406c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f6668bf9640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666406fc60 con 0x7f666415ebd0 2024-04-25T16:52:40.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f6668bf9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f6664158550 con 0x7f6664070080 2024-04-25T16:52:40.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f6668bf9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f666406f7d0 con 0x7f666415e7c0 2024-04-25T16:52:40.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f666259c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6664070080 0x7f666415e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:40.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f6662d9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f666415ebd0 0x7f666406cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:33592/0 (socket says 172.21.15.94:33592) 2024-04-25T16:52:40.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f6662d9d640 1 -- 172.21.15.94:0/725288893 learned_addr learned my addr 172.21.15.94:0/725288893 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:40.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f666259c640 1 --2- 172.21.15.94:0/725288893 >> v2:172.21.15.94:3300/0 conn(0x7f6664070080 0x7f666415e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49754/0 (socket says 172.21.15.94:49754) 2024-04-25T16:52:40.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f6661d9b640 1 --2- 172.21.15.94:0/725288893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666415e7c0 0x7f666406c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:40.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f666259c640 1 -- 172.21.15.94:0/725288893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666415e7c0 msgr2=0x7f666406c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:40.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f666259c640 1 --2- 172.21.15.94:0/725288893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666415e7c0 0x7f666406c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f666259c640 1 -- 172.21.15.94:0/725288893 >> v1:172.21.15.94:6789/0 conn(0x7f666415ebd0 legacy=0x7f666406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:40.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f666159a640 1 -- 172.21.15.94:0/725288893 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1964594248 0 0) 0x7f666406fc60 con 0x7f666415ebd0 2024-04-25T16:52:40.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.643+0000 7f666259c640 1 -- 172.21.15.94:0/725288893 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66640aef70 con 0x7f6664070080 2024-04-25T16:52:40.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f666259c640 1 --2- 172.21.15.94:0/725288893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6664070080 0x7f666415e280 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f665c009d90 tx=0x7f665c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585cb54e7b94f56d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:40.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f666159a640 1 -- 172.21.15.94:0/725288893 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0b2070 con 0x7f6664070080 2024-04-25T16:52:40.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f666159a640 1 -- 172.21.15.94:0/725288893 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665c0a5910 con 0x7f6664070080 2024-04-25T16:52:40.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f666159a640 1 -- 172.21.15.94:0/725288893 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0a5ce0 con 0x7f6664070080 2024-04-25T16:52:40.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f6668bf9640 1 -- 172.21.15.94:0/725288893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6664070080 msgr2=0x7f666415e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:40.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f6668bf9640 1 --2- 172.21.15.94:0/725288893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6664070080 0x7f666415e280 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f665c009d90 tx=0x7f665c01c940 comp rx=0 tx=0).stop 2024-04-25T16:52:40.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f6668bf9640 1 -- 172.21.15.94:0/725288893 shutdown_connections 2024-04-25T16:52:40.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f6668bf9640 1 --2- 172.21.15.94:0/725288893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f666415e7c0 0x7f666406c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f6668bf9640 1 --2- 172.21.15.94:0/725288893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6664070080 0x7f666415e280 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f6668bf9640 1 -- 172.21.15.94:0/725288893 >> 172.21.15.94:0/725288893 conn(0x7f66640b47f0 msgr2=0x7f66640b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f6668bf9640 1 -- 172.21.15.94:0/725288893 shutdown_connections 2024-04-25T16:52:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f6668bf9640 1 -- 172.21.15.94:0/725288893 wait complete. 2024-04-25T16:52:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f6668bf9640 1 Processor -- start 2024-04-25T16:52:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.647+0000 7f6668bf9640 1 -- start start 2024-04-25T16:52:40.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6668bf9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6664070080 0x7f666409aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6668bf9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666415e7c0 0x7f6664157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6668bf9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666415ebd0 0x7f66640b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6668bf9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6664070490 con 0x7f666415ebd0 2024-04-25T16:52:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6668bf9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66640ae6c0 con 0x7f666415e7c0 2024-04-25T16:52:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6668bf9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f666416ae90 con 0x7f6664070080 2024-04-25T16:52:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6662d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666415ebd0 0x7f66640b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6662d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666415ebd0 0x7f66640b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49770/0 (socket says 172.21.15.94:49770) 2024-04-25T16:52:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6662d9d640 1 -- 172.21.15.94:0/2509406238 learned_addr learned my addr 172.21.15.94:0/2509406238 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6661d9b640 1 --2- 172.21.15.94:0/2509406238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666415e7c0 0x7f6664157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6662d9d640 1 -- 172.21.15.94:0/2509406238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6664070080 msgr2=0x7f666409aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6662d9d640 1 --2- 172.21.15.94:0/2509406238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6664070080 0x7f666409aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6662d9d640 1 -- 172.21.15.94:0/2509406238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666415e7c0 msgr2=0x7f6664157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6662d9d640 1 --2- 172.21.15.94:0/2509406238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666415e7c0 0x7f6664157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6662d9d640 1 -- 172.21.15.94:0/2509406238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66641f75a0 con 0x7f666415ebd0 2024-04-25T16:52:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6662d9d640 1 --2- 172.21.15.94:0/2509406238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666415ebd0 0x7f66640b0d40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f665400c410 tx=0x7f665400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6652ffd640 1 -- 172.21.15.94:0/2509406238 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66540074e0 con 0x7f666415ebd0 2024-04-25T16:52:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6652ffd640 1 -- 172.21.15.94:0/2509406238 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66540047f0 con 0x7f666415ebd0 2024-04-25T16:52:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6652ffd640 1 -- 172.21.15.94:0/2509406238 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654004c20 con 0x7f666415ebd0 2024-04-25T16:52:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6668bf9640 1 -- 172.21.15.94:0/2509406238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66641f7880 con 0x7f666415ebd0 2024-04-25T16:52:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6668bf9640 1 -- 172.21.15.94:0/2509406238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66641f7e40 con 0x7f666415ebd0 2024-04-25T16:52:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.651+0000 7f6668bf9640 1 -- 172.21.15.94:0/2509406238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f666414c650 con 0x7f666415ebd0 2024-04-25T16:52:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.655+0000 7f6652ffd640 1 -- 172.21.15.94:0/2509406238 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6654011070 con 0x7f666415ebd0 2024-04-25T16:52:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.655+0000 7f6652ffd640 1 --2- 172.21.15.94:0/2509406238 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f664406f430 0x7f6644071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.655+0000 7f666259c640 1 --2- 172.21.15.94:0/2509406238 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f664406f430 0x7f6644071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.655+0000 7f6652ffd640 1 -- 172.21.15.94:0/2509406238 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f665408f5c0 con 0x7f666415ebd0 2024-04-25T16:52:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.655+0000 7f666259c640 1 --2- 172.21.15.94:0/2509406238 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f664406f430 0x7f6644071940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f665c0af000 tx=0x7f665c004ac0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.659+0000 7f6652ffd640 1 -- 172.21.15.94:0/2509406238 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f665405c370 con 0x7f666415ebd0 2024-04-25T16:52:40.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.803+0000 7f6668bf9640 1 -- 172.21.15.94:0/2509406238 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6664076830 con 0x7f664406f430 2024-04-25T16:52:40.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.811+0000 7f6652ffd640 1 -- 172.21.15.94:0/2509406238 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251872 (secure 0 0 0) 0x7f6664076830 con 0x7f664406f430 2024-04-25T16:52:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:40.815 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:52:40.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.815+0000 7f6668bf9640 1 -- 172.21.15.94:0/2509406238 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f664406f430 msgr2=0x7f6644071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:40.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.815+0000 7f6668bf9640 1 --2- 172.21.15.94:0/2509406238 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f664406f430 0x7f6644071940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f665c0af000 tx=0x7f665c004ac0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.815+0000 7f6668bf9640 1 -- 172.21.15.94:0/2509406238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666415ebd0 msgr2=0x7f66640b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:40.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.815+0000 7f6668bf9640 1 --2- 172.21.15.94:0/2509406238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666415ebd0 0x7f66640b0d40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f665400c410 tx=0x7f665400c810 comp rx=0 tx=0).stop 2024-04-25T16:52:40.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.815+0000 7f6668bf9640 1 -- 172.21.15.94:0/2509406238 shutdown_connections 2024-04-25T16:52:40.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.815+0000 7f6668bf9640 1 --2- 172.21.15.94:0/2509406238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666415ebd0 0x7f66640b0d40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.815+0000 7f6668bf9640 1 --2- 172.21.15.94:0/2509406238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666415e7c0 0x7f6664157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.815+0000 7f6668bf9640 1 --2- 172.21.15.94:0/2509406238 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f664406f430 0x7f6644071940 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.815+0000 7f6668bf9640 1 --2- 172.21.15.94:0/2509406238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6664070080 0x7f666409aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:40.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.815+0000 7f6668bf9640 1 -- 172.21.15.94:0/2509406238 >> 172.21.15.94:0/2509406238 conn(0x7f66640b47f0 msgr2=0x7f66640b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:40.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.815+0000 7f6668bf9640 1 -- 172.21.15.94:0/2509406238 shutdown_connections 2024-04-25T16:52:40.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:40.815+0000 7f6668bf9640 1 -- 172.21.15.94:0/2509406238 wait complete. 2024-04-25T16:52:40.832 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-04-25T16:52:39.696569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258320,"kb_used_data":4708,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716400,"statfs":{"total":773094113280,"available":772829593600,"internally_reserved":0,"allocated":4820992,"data_stored":2737575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002542"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873724+0000","last_change":"2024-04-25T16:52:26.825748+0000","last_active":"2024-04-25T16:52:34.873724+0000","last_peered":"2024-04-25T16:52:34.873724+0000","last_clean":"2024-04-25T16:52:34.873724+0000","last_became_active":"2024-04-25T16:52:26.825333+0000","last_became_peered":"2024-04-25T16:52:26.825333+0000","last_unstale":"2024-04-25T16:52:34.873724+0000","last_undegraded":"2024-04-25T16:52:34.873724+0000","last_fullsized":"2024-04-25T16:52:34.873724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:19.084563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873506+0000","last_change":"2024-04-25T16:52:26.795964+0000","last_active":"2024-04-25T16:52:34.873506+0000","last_peered":"2024-04-25T16:52:34.873506+0000","last_clean":"2024-04-25T16:52:34.873506+0000","last_became_active":"2024-04-25T16:52:26.794884+0000","last_became_peered":"2024-04-25T16:52:26.794884+0000","last_unstale":"2024-04-25T16:52:34.873506+0000","last_undegraded":"2024-04-25T16:52:34.873506+0000","last_fullsized":"2024-04-25T16:52:34.873506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878233+0000","last_change":"2024-04-25T16:52:26.802075+0000","last_active":"2024-04-25T16:52:34.878233+0000","last_peered":"2024-04-25T16:52:34.878233+0000","last_clean":"2024-04-25T16:52:34.878233+0000","last_became_active":"2024-04-25T16:52:26.801203+0000","last_became_peered":"2024-04-25T16:52:26.801203+0000","last_unstale":"2024-04-25T16:52:34.878233+0000","last_undegraded":"2024-04-25T16:52:34.878233+0000","last_fullsized":"2024-04-25T16:52:34.878233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782213+0000","last_change":"2024-04-25T16:52:26.793541+0000","last_active":"2024-04-25T16:52:27.782213+0000","last_peered":"2024-04-25T16:52:27.782213+0000","last_clean":"2024-04-25T16:52:27.782213+0000","last_became_active":"2024-04-25T16:52:26.789101+0000","last_became_peered":"2024-04-25T16:52:26.789101+0000","last_unstale":"2024-04-25T16:52:27.782213+0000","last_undegraded":"2024-04-25T16:52:27.782213+0000","last_fullsized":"2024-04-25T16:52:27.782213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:38.013058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781321+0000","last_change":"2024-04-25T16:52:26.784467+0000","last_active":"2024-04-25T16:52:27.781321+0000","last_peered":"2024-04-25T16:52:27.781321+0000","last_clean":"2024-04-25T16:52:27.781321+0000","last_became_active":"2024-04-25T16:52:26.783673+0000","last_became_peered":"2024-04-25T16:52:26.783673+0000","last_unstale":"2024-04-25T16:52:27.781321+0000","last_undegraded":"2024-04-25T16:52:27.781321+0000","last_fullsized":"2024-04-25T16:52:27.781321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:36.583897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831274+0000","last_change":"2024-04-25T16:52:26.823413+0000","last_active":"2024-04-25T16:52:27.831274+0000","last_peered":"2024-04-25T16:52:27.831274+0000","last_clean":"2024-04-25T16:52:27.831274+0000","last_became_active":"2024-04-25T16:52:26.823059+0000","last_became_peered":"2024-04-25T16:52:26.823059+0000","last_unstale":"2024-04-25T16:52:27.831274+0000","last_undegraded":"2024-04-25T16:52:27.831274+0000","last_fullsized":"2024-04-25T16:52:27.831274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.876836+0000","last_change":"2024-04-25T16:52:26.808661+0000","last_active":"2024-04-25T16:52:34.876836+0000","last_peered":"2024-04-25T16:52:34.876836+0000","last_clean":"2024-04-25T16:52:34.876836+0000","last_became_active":"2024-04-25T16:52:26.805296+0000","last_became_peered":"2024-04-25T16:52:26.805296+0000","last_unstale":"2024-04-25T16:52:34.876836+0000","last_undegraded":"2024-04-25T16:52:34.876836+0000","last_fullsized":"2024-04-25T16:52:34.876836+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"peering","last_fresh":"2024-04-25T16:52:30.709501+0000","last_change":"2024-04-25T16:52:34.873590+0000","last_active":"2024-04-25T16:52:34.873590+0000","last_peered":"2024-04-25T16:52:30.709501+0000","last_clean":"2024-04-25T16:52:30.709501+0000","last_became_active":"2024-04-25T16:52:26.820528+0000","last_became_peered":"2024-04-25T16:52:26.820528+0000","last_unstale":"2024-04-25T16:52:30.709501+0000","last_undegraded":"2024-04-25T16:52:30.709501+0000","last_fullsized":"2024-04-25T16:52:30.709501+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.879080+0000","last_change":"2024-04-25T16:52:26.803363+0000","last_active":"2024-04-25T16:52:34.879080+0000","last_peered":"2024-04-25T16:52:34.879080+0000","last_clean":"2024-04-25T16:52:34.879080+0000","last_became_active":"2024-04-25T16:52:26.802539+0000","last_became_peered":"2024-04-25T16:52:26.802539+0000","last_unstale":"2024-04-25T16:52:34.879080+0000","last_undegraded":"2024-04-25T16:52:34.879080+0000","last_fullsized":"2024-04-25T16:52:34.879080+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877890+0000","last_change":"2024-04-25T16:52:26.842728+0000","last_active":"2024-04-25T16:52:34.877890+0000","last_peered":"2024-04-25T16:52:34.877890+0000","last_clean":"2024-04-25T16:52:34.877890+0000","last_became_active":"2024-04-25T16:52:26.842041+0000","last_became_peered":"2024-04-25T16:52:26.842041+0000","last_unstale":"2024-04-25T16:52:34.877890+0000","last_undegraded":"2024-04-25T16:52:34.877890+0000","last_fullsized":"2024-04-25T16:52:34.877890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:33.962706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877607+0000","last_change":"2024-04-25T16:52:26.819329+0000","last_active":"2024-04-25T16:52:34.877607+0000","last_peered":"2024-04-25T16:52:34.877607+0000","last_clean":"2024-04-25T16:52:34.877607+0000","last_became_active":"2024-04-25T16:52:26.818528+0000","last_became_peered":"2024-04-25T16:52:26.818528+0000","last_unstale":"2024-04-25T16:52:34.877607+0000","last_undegraded":"2024-04-25T16:52:34.877607+0000","last_fullsized":"2024-04-25T16:52:34.877607+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878111+0000","last_change":"2024-04-25T16:52:26.795942+0000","last_active":"2024-04-25T16:52:34.878111+0000","last_peered":"2024-04-25T16:52:34.878111+0000","last_clean":"2024-04-25T16:52:34.878111+0000","last_became_active":"2024-04-25T16:52:26.795124+0000","last_became_peered":"2024-04-25T16:52:26.795124+0000","last_unstale":"2024-04-25T16:52:34.878111+0000","last_undegraded":"2024-04-25T16:52:34.878111+0000","last_fullsized":"2024-04-25T16:52:34.878111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877026+0000","last_change":"2024-04-25T16:52:26.814749+0000","last_active":"2024-04-25T16:52:34.877026+0000","last_peered":"2024-04-25T16:52:34.877026+0000","last_clean":"2024-04-25T16:52:34.877026+0000","last_became_active":"2024-04-25T16:52:26.814020+0000","last_became_peered":"2024-04-25T16:52:26.814020+0000","last_unstale":"2024-04-25T16:52:34.877026+0000","last_undegraded":"2024-04-25T16:52:34.877026+0000","last_fullsized":"2024-04-25T16:52:34.877026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871890+0000","last_change":"2024-04-25T16:52:26.813118+0000","last_active":"2024-04-25T16:52:34.871890+0000","last_peered":"2024-04-25T16:52:34.871890+0000","last_clean":"2024-04-25T16:52:34.871890+0000","last_became_active":"2024-04-25T16:52:26.812400+0000","last_became_peered":"2024-04-25T16:52:26.812400+0000","last_unstale":"2024-04-25T16:52:34.871890+0000","last_undegraded":"2024-04-25T16:52:34.871890+0000","last_fullsized":"2024-04-25T16:52:34.871890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:16.169654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877027+0000","last_change":"2024-04-25T16:52:26.795864+0000","last_active":"2024-04-25T16:52:34.877027+0000","last_peered":"2024-04-25T16:52:34.877027+0000","last_clean":"2024-04-25T16:52:34.877027+0000","last_became_active":"2024-04-25T16:52:26.794738+0000","last_became_peered":"2024-04-25T16:52:26.794738+0000","last_unstale":"2024-04-25T16:52:34.877027+0000","last_undegraded":"2024-04-25T16:52:34.877027+0000","last_fullsized":"2024-04-25T16:52:34.877027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340722+0000","last_change":"2024-04-25T16:52:26.836640+0000","last_active":"2024-04-25T16:52:35.340722+0000","last_peered":"2024-04-25T16:52:35.340722+0000","last_clean":"2024-04-25T16:52:35.340722+0000","last_became_active":"2024-04-25T16:52:26.833071+0000","last_became_peered":"2024-04-25T16:52:26.833071+0000","last_unstale":"2024-04-25T16:52:35.340722+0000","last_undegraded":"2024-04-25T16:52:35.340722+0000","last_fullsized":"2024-04-25T16:52:35.340722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341191+0000","last_change":"2024-04-25T16:52:26.841207+0000","last_active":"2024-04-25T16:52:35.341191+0000","last_peered":"2024-04-25T16:52:35.341191+0000","last_clean":"2024-04-25T16:52:35.341191+0000","last_became_active":"2024-04-25T16:52:26.838750+0000","last_became_peered":"2024-04-25T16:52:26.838750+0000","last_unstale":"2024-04-25T16:52:35.341191+0000","last_undegraded":"2024-04-25T16:52:35.341191+0000","last_fullsized":"2024-04-25T16:52:35.341191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875046+0000","last_change":"2024-04-25T16:52:26.795939+0000","last_active":"2024-04-25T16:52:34.875046+0000","last_peered":"2024-04-25T16:52:34.875046+0000","last_clean":"2024-04-25T16:52:34.875046+0000","last_became_active":"2024-04-25T16:52:26.795084+0000","last_became_peered":"2024-04-25T16:52:26.795084+0000","last_unstale":"2024-04-25T16:52:34.875046+0000","last_undegraded":"2024-04-25T16:52:34.875046+0000","last_fullsized":"2024-04-25T16:52:34.875046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781139+0000","last_change":"2024-04-25T16:52:26.818025+0000","last_active":"2024-04-25T16:52:27.781139+0000","last_peered":"2024-04-25T16:52:27.781139+0000","last_clean":"2024-04-25T16:52:27.781139+0000","last_became_active":"2024-04-25T16:52:26.817652+0000","last_became_peered":"2024-04-25T16:52:26.817652+0000","last_unstale":"2024-04-25T16:52:27.781139+0000","last_undegraded":"2024-04-25T16:52:27.781139+0000","last_fullsized":"2024-04-25T16:52:27.781139+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:46.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872811+0000","last_change":"2024-04-25T16:52:26.796384+0000","last_active":"2024-04-25T16:52:34.872811+0000","last_peered":"2024-04-25T16:52:34.872811+0000","last_clean":"2024-04-25T16:52:34.872811+0000","last_became_active":"2024-04-25T16:52:26.795507+0000","last_became_peered":"2024-04-25T16:52:26.795507+0000","last_unstale":"2024-04-25T16:52:34.872811+0000","last_undegraded":"2024-04-25T16:52:34.872811+0000","last_fullsized":"2024-04-25T16:52:34.872811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877104+0000","last_change":"2024-04-25T16:52:26.809988+0000","last_active":"2024-04-25T16:52:34.877104+0000","last_peered":"2024-04-25T16:52:34.877104+0000","last_clean":"2024-04-25T16:52:34.877104+0000","last_became_active":"2024-04-25T16:52:26.809107+0000","last_became_peered":"2024-04-25T16:52:26.809107+0000","last_unstale":"2024-04-25T16:52:34.877104+0000","last_undegraded":"2024-04-25T16:52:34.877104+0000","last_fullsized":"2024-04-25T16:52:34.877104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872865+0000","last_change":"2024-04-25T16:52:26.784466+0000","last_active":"2024-04-25T16:52:34.872865+0000","last_peered":"2024-04-25T16:52:34.872865+0000","last_clean":"2024-04-25T16:52:34.872865+0000","last_became_active":"2024-04-25T16:52:26.783550+0000","last_became_peered":"2024-04-25T16:52:26.783550+0000","last_unstale":"2024-04-25T16:52:34.872865+0000","last_undegraded":"2024-04-25T16:52:34.872865+0000","last_fullsized":"2024-04-25T16:52:34.872865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340668+0000","last_change":"2024-04-25T16:52:26.832452+0000","last_active":"2024-04-25T16:52:35.340668+0000","last_peered":"2024-04-25T16:52:35.340668+0000","last_clean":"2024-04-25T16:52:35.340668+0000","last_became_active":"2024-04-25T16:52:26.831753+0000","last_became_peered":"2024-04-25T16:52:26.831753+0000","last_unstale":"2024-04-25T16:52:35.340668+0000","last_undegraded":"2024-04-25T16:52:35.340668+0000","last_fullsized":"2024-04-25T16:52:35.340668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780752+0000","last_change":"2024-04-25T16:52:26.805633+0000","last_active":"2024-04-25T16:52:27.780752+0000","last_peered":"2024-04-25T16:52:27.780752+0000","last_clean":"2024-04-25T16:52:27.780752+0000","last_became_active":"2024-04-25T16:52:26.804789+0000","last_became_peered":"2024-04-25T16:52:26.804789+0000","last_unstale":"2024-04-25T16:52:27.780752+0000","last_undegraded":"2024-04-25T16:52:27.780752+0000","last_fullsized":"2024-04-25T16:52:27.780752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:08.931465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875174+0000","last_change":"2024-04-25T16:52:26.798693+0000","last_active":"2024-04-25T16:52:34.875174+0000","last_peered":"2024-04-25T16:52:34.875174+0000","last_clean":"2024-04-25T16:52:34.875174+0000","last_became_active":"2024-04-25T16:52:26.797852+0000","last_became_peered":"2024-04-25T16:52:26.797852+0000","last_unstale":"2024-04-25T16:52:34.875174+0000","last_undegraded":"2024-04-25T16:52:34.875174+0000","last_fullsized":"2024-04-25T16:52:34.875174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831376+0000","last_change":"2024-04-25T16:52:26.819630+0000","last_active":"2024-04-25T16:52:27.831376+0000","last_peered":"2024-04-25T16:52:27.831376+0000","last_clean":"2024-04-25T16:52:27.831376+0000","last_became_active":"2024-04-25T16:52:26.819274+0000","last_became_peered":"2024-04-25T16:52:26.819274+0000","last_unstale":"2024-04-25T16:52:27.831376+0000","last_undegraded":"2024-04-25T16:52:27.831376+0000","last_fullsized":"2024-04-25T16:52:27.831376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873718+0000","last_change":"2024-04-25T16:52:26.791433+0000","last_active":"2024-04-25T16:52:34.873718+0000","last_peered":"2024-04-25T16:52:34.873718+0000","last_clean":"2024-04-25T16:52:34.873718+0000","last_became_active":"2024-04-25T16:52:26.790483+0000","last_became_peered":"2024-04-25T16:52:26.790483+0000","last_unstale":"2024-04-25T16:52:34.873718+0000","last_undegraded":"2024-04-25T16:52:34.873718+0000","last_fullsized":"2024-04-25T16:52:34.873718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877762+0000","last_change":"2024-04-25T16:52:26.804860+0000","last_active":"2024-04-25T16:52:34.877762+0000","last_peered":"2024-04-25T16:52:34.877762+0000","last_clean":"2024-04-25T16:52:34.877762+0000","last_became_active":"2024-04-25T16:52:26.804065+0000","last_became_peered":"2024-04-25T16:52:26.804065+0000","last_unstale":"2024-04-25T16:52:34.877762+0000","last_undegraded":"2024-04-25T16:52:34.877762+0000","last_fullsized":"2024-04-25T16:52:34.877762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873009+0000","last_change":"2024-04-25T16:52:26.806705+0000","last_active":"2024-04-25T16:52:34.873009+0000","last_peered":"2024-04-25T16:52:34.873009+0000","last_clean":"2024-04-25T16:52:34.873009+0000","last_became_active":"2024-04-25T16:52:26.805889+0000","last_became_peered":"2024-04-25T16:52:26.805889+0000","last_unstale":"2024-04-25T16:52:34.873009+0000","last_undegraded":"2024-04-25T16:52:34.873009+0000","last_fullsized":"2024-04-25T16:52:34.873009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874258+0000","last_change":"2024-04-25T16:52:26.798903+0000","last_active":"2024-04-25T16:52:34.874258+0000","last_peered":"2024-04-25T16:52:34.874258+0000","last_clean":"2024-04-25T16:52:34.874258+0000","last_became_active":"2024-04-25T16:52:26.798011+0000","last_became_peered":"2024-04-25T16:52:26.798011+0000","last_unstale":"2024-04-25T16:52:34.874258+0000","last_undegraded":"2024-04-25T16:52:34.874258+0000","last_fullsized":"2024-04-25T16:52:34.874258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340837+0000","last_change":"2024-04-25T16:52:26.841037+0000","last_active":"2024-04-25T16:52:35.340837+0000","last_peered":"2024-04-25T16:52:35.340837+0000","last_clean":"2024-04-25T16:52:35.340837+0000","last_became_active":"2024-04-25T16:52:26.838194+0000","last_became_peered":"2024-04-25T16:52:26.838194+0000","last_unstale":"2024-04-25T16:52:35.340837+0000","last_undegraded":"2024-04-25T16:52:35.340837+0000","last_fullsized":"2024-04-25T16:52:35.340837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830883+0000","last_change":"2024-04-25T16:52:26.813243+0000","last_active":"2024-04-25T16:52:27.830883+0000","last_peered":"2024-04-25T16:52:27.830883+0000","last_clean":"2024-04-25T16:52:27.830883+0000","last_became_active":"2024-04-25T16:52:26.811807+0000","last_became_peered":"2024-04-25T16:52:26.811807+0000","last_unstale":"2024-04-25T16:52:27.830883+0000","last_undegraded":"2024-04-25T16:52:27.830883+0000","last_fullsized":"2024-04-25T16:52:27.830883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:42.553053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341094+0000","last_change":"2024-04-25T16:52:26.841017+0000","last_active":"2024-04-25T16:52:35.341094+0000","last_peered":"2024-04-25T16:52:35.341094+0000","last_clean":"2024-04-25T16:52:35.341094+0000","last_became_active":"2024-04-25T16:52:26.837543+0000","last_became_peered":"2024-04-25T16:52:26.837543+0000","last_unstale":"2024-04-25T16:52:35.341094+0000","last_undegraded":"2024-04-25T16:52:35.341094+0000","last_fullsized":"2024-04-25T16:52:35.341094+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875394+0000","last_change":"2024-04-25T16:52:25.818069+0000","last_active":"2024-04-25T16:52:34.875394+0000","last_peered":"2024-04-25T16:52:34.875394+0000","last_clean":"2024-04-25T16:52:34.875394+0000","last_became_active":"2024-04-25T16:52:25.817362+0000","last_became_peered":"2024-04-25T16:52:25.817362+0000","last_unstale":"2024-04-25T16:52:34.875394+0000","last_undegraded":"2024-04-25T16:52:34.875394+0000","last_fullsized":"2024-04-25T16:52:34.875394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:08.488136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833310+0000","last_change":"2024-04-25T16:52:26.829008+0000","last_active":"2024-04-25T16:52:27.833310+0000","last_peered":"2024-04-25T16:52:27.833310+0000","last_clean":"2024-04-25T16:52:27.833310+0000","last_became_active":"2024-04-25T16:52:26.828706+0000","last_became_peered":"2024-04-25T16:52:26.828706+0000","last_unstale":"2024-04-25T16:52:27.833310+0000","last_undegraded":"2024-04-25T16:52:27.833310+0000","last_fullsized":"2024-04-25T16:52:27.833310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878876+0000","last_change":"2024-04-25T16:52:25.795967+0000","last_active":"2024-04-25T16:52:34.878876+0000","last_peered":"2024-04-25T16:52:34.878876+0000","last_clean":"2024-04-25T16:52:34.878876+0000","last_became_active":"2024-04-25T16:52:25.795695+0000","last_became_peered":"2024-04-25T16:52:25.795695+0000","last_unstale":"2024-04-25T16:52:34.878876+0000","last_undegraded":"2024-04-25T16:52:34.878876+0000","last_fullsized":"2024-04-25T16:52:34.878876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875243+0000","last_change":"2024-04-25T16:52:26.789430+0000","last_active":"2024-04-25T16:52:34.875243+0000","last_peered":"2024-04-25T16:52:34.875243+0000","last_clean":"2024-04-25T16:52:34.875243+0000","last_became_active":"2024-04-25T16:52:26.782243+0000","last_became_peered":"2024-04-25T16:52:26.782243+0000","last_unstale":"2024-04-25T16:52:34.875243+0000","last_undegraded":"2024-04-25T16:52:34.875243+0000","last_fullsized":"2024-04-25T16:52:34.875243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833428+0000","last_change":"2024-04-25T16:52:25.819162+0000","last_active":"2024-04-25T16:52:27.833428+0000","last_peered":"2024-04-25T16:52:27.833428+0000","last_clean":"2024-04-25T16:52:27.833428+0000","last_became_active":"2024-04-25T16:52:25.818859+0000","last_became_peered":"2024-04-25T16:52:25.818859+0000","last_unstale":"2024-04-25T16:52:27.833428+0000","last_undegraded":"2024-04-25T16:52:27.833428+0000","last_fullsized":"2024-04-25T16:52:27.833428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874082+0000","last_change":"2024-04-25T16:52:26.805376+0000","last_active":"2024-04-25T16:52:34.874082+0000","last_peered":"2024-04-25T16:52:34.874082+0000","last_clean":"2024-04-25T16:52:34.874082+0000","last_became_active":"2024-04-25T16:52:26.804542+0000","last_became_peered":"2024-04-25T16:52:26.804542+0000","last_unstale":"2024-04-25T16:52:34.874082+0000","last_undegraded":"2024-04-25T16:52:34.874082+0000","last_fullsized":"2024-04-25T16:52:34.874082+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.620224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874004+0000","last_change":"2024-04-25T16:52:25.815751+0000","last_active":"2024-04-25T16:52:34.874004+0000","last_peered":"2024-04-25T16:52:34.874004+0000","last_clean":"2024-04-25T16:52:34.874004+0000","last_became_active":"2024-04-25T16:52:25.814927+0000","last_became_peered":"2024-04-25T16:52:25.814927+0000","last_unstale":"2024-04-25T16:52:34.874004+0000","last_undegraded":"2024-04-25T16:52:34.874004+0000","last_fullsized":"2024-04-25T16:52:34.874004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877461+0000","last_change":"2024-04-25T16:52:26.814640+0000","last_active":"2024-04-25T16:52:34.877461+0000","last_peered":"2024-04-25T16:52:34.877461+0000","last_clean":"2024-04-25T16:52:34.877461+0000","last_became_active":"2024-04-25T16:52:26.814016+0000","last_became_peered":"2024-04-25T16:52:26.814016+0000","last_unstale":"2024-04-25T16:52:34.877461+0000","last_undegraded":"2024-04-25T16:52:34.877461+0000","last_fullsized":"2024-04-25T16:52:34.877461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877431+0000","last_change":"2024-04-25T16:52:25.805323+0000","last_active":"2024-04-25T16:52:34.877431+0000","last_peered":"2024-04-25T16:52:34.877431+0000","last_clean":"2024-04-25T16:52:34.877431+0000","last_became_active":"2024-04-25T16:52:25.796444+0000","last_became_peered":"2024-04-25T16:52:25.796444+0000","last_unstale":"2024-04-25T16:52:34.877431+0000","last_undegraded":"2024-04-25T16:52:34.877431+0000","last_fullsized":"2024-04-25T16:52:34.877431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:28.996351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877181+0000","last_change":"2024-04-25T16:52:26.821648+0000","last_active":"2024-04-25T16:52:34.877181+0000","last_peered":"2024-04-25T16:52:34.877181+0000","last_clean":"2024-04-25T16:52:34.877181+0000","last_became_active":"2024-04-25T16:52:26.820835+0000","last_became_peered":"2024-04-25T16:52:26.820835+0000","last_unstale":"2024-04-25T16:52:34.877181+0000","last_undegraded":"2024-04-25T16:52:34.877181+0000","last_fullsized":"2024-04-25T16:52:34.877181+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340828+0000","last_change":"2024-04-25T16:52:25.816250+0000","last_active":"2024-04-25T16:52:35.340828+0000","last_peered":"2024-04-25T16:52:35.340828+0000","last_clean":"2024-04-25T16:52:35.340828+0000","last_became_active":"2024-04-25T16:52:25.815656+0000","last_became_peered":"2024-04-25T16:52:25.815656+0000","last_unstale":"2024-04-25T16:52:35.340828+0000","last_undegraded":"2024-04-25T16:52:35.340828+0000","last_fullsized":"2024-04-25T16:52:35.340828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872461+0000","last_change":"2024-04-25T16:52:26.813425+0000","last_active":"2024-04-25T16:52:34.872461+0000","last_peered":"2024-04-25T16:52:34.872461+0000","last_clean":"2024-04-25T16:52:34.872461+0000","last_became_active":"2024-04-25T16:52:26.812713+0000","last_became_peered":"2024-04-25T16:52:26.812713+0000","last_unstale":"2024-04-25T16:52:34.872461+0000","last_undegraded":"2024-04-25T16:52:34.872461+0000","last_fullsized":"2024-04-25T16:52:34.872461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831072+0000","last_change":"2024-04-25T16:52:25.803265+0000","last_active":"2024-04-25T16:52:27.831072+0000","last_peered":"2024-04-25T16:52:27.831072+0000","last_clean":"2024-04-25T16:52:27.831072+0000","last_became_active":"2024-04-25T16:52:25.802617+0000","last_became_peered":"2024-04-25T16:52:25.802617+0000","last_unstale":"2024-04-25T16:52:27.831072+0000","last_undegraded":"2024-04-25T16:52:27.831072+0000","last_fullsized":"2024-04-25T16:52:27.831072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872313+0000","last_change":"2024-04-25T16:52:26.819267+0000","last_active":"2024-04-25T16:52:34.872313+0000","last_peered":"2024-04-25T16:52:34.872313+0000","last_clean":"2024-04-25T16:52:34.872313+0000","last_became_active":"2024-04-25T16:52:26.818740+0000","last_became_peered":"2024-04-25T16:52:26.818740+0000","last_unstale":"2024-04-25T16:52:34.872313+0000","last_undegraded":"2024-04-25T16:52:34.872313+0000","last_fullsized":"2024-04-25T16:52:34.872313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871955+0000","last_change":"2024-04-25T16:52:25.816663+0000","last_active":"2024-04-25T16:52:34.871955+0000","last_peered":"2024-04-25T16:52:34.871955+0000","last_clean":"2024-04-25T16:52:34.871955+0000","last_became_active":"2024-04-25T16:52:25.815420+0000","last_became_peered":"2024-04-25T16:52:25.815420+0000","last_unstale":"2024-04-25T16:52:34.871955+0000","last_undegraded":"2024-04-25T16:52:34.871955+0000","last_fullsized":"2024-04-25T16:52:34.871955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:30.288538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782313+0000","last_change":"2024-04-25T16:52:26.789927+0000","last_active":"2024-04-25T16:52:27.782313+0000","last_peered":"2024-04-25T16:52:27.782313+0000","last_clean":"2024-04-25T16:52:27.782313+0000","last_became_active":"2024-04-25T16:52:26.788014+0000","last_became_peered":"2024-04-25T16:52:26.788014+0000","last_unstale":"2024-04-25T16:52:27.782313+0000","last_undegraded":"2024-04-25T16:52:27.782313+0000","last_fullsized":"2024-04-25T16:52:27.782313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:47.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873885+0000","last_change":"2024-04-25T16:52:25.798974+0000","last_active":"2024-04-25T16:52:34.873885+0000","last_peered":"2024-04-25T16:52:34.873885+0000","last_clean":"2024-04-25T16:52:34.873885+0000","last_became_active":"2024-04-25T16:52:25.798314+0000","last_became_peered":"2024-04-25T16:52:25.798314+0000","last_unstale":"2024-04-25T16:52:34.873885+0000","last_undegraded":"2024-04-25T16:52:34.873885+0000","last_fullsized":"2024-04-25T16:52:34.873885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.348530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341450+0000","last_change":"2024-04-25T16:52:26.842485+0000","last_active":"2024-04-25T16:52:35.341450+0000","last_peered":"2024-04-25T16:52:35.341450+0000","last_clean":"2024-04-25T16:52:35.341450+0000","last_became_active":"2024-04-25T16:52:26.838813+0000","last_became_peered":"2024-04-25T16:52:26.838813+0000","last_unstale":"2024-04-25T16:52:35.341450+0000","last_undegraded":"2024-04-25T16:52:35.341450+0000","last_fullsized":"2024-04-25T16:52:35.341450+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878521+0000","last_change":"2024-04-25T16:52:25.803289+0000","last_active":"2024-04-25T16:52:34.878521+0000","last_peered":"2024-04-25T16:52:34.878521+0000","last_clean":"2024-04-25T16:52:34.878521+0000","last_became_active":"2024-04-25T16:52:25.801415+0000","last_became_peered":"2024-04-25T16:52:25.801415+0000","last_unstale":"2024-04-25T16:52:34.878521+0000","last_undegraded":"2024-04-25T16:52:34.878521+0000","last_fullsized":"2024-04-25T16:52:34.878521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:45.310089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878182+0000","last_change":"2024-04-25T16:52:26.840379+0000","last_active":"2024-04-25T16:52:34.878182+0000","last_peered":"2024-04-25T16:52:34.878182+0000","last_clean":"2024-04-25T16:52:34.878182+0000","last_became_active":"2024-04-25T16:52:26.840062+0000","last_became_peered":"2024-04-25T16:52:26.840062+0000","last_unstale":"2024-04-25T16:52:34.878182+0000","last_undegraded":"2024-04-25T16:52:34.878182+0000","last_fullsized":"2024-04-25T16:52:34.878182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878024+0000","last_change":"2024-04-25T16:52:25.809391+0000","last_active":"2024-04-25T16:52:34.878024+0000","last_peered":"2024-04-25T16:52:34.878024+0000","last_clean":"2024-04-25T16:52:34.878024+0000","last_became_active":"2024-04-25T16:52:25.808455+0000","last_became_peered":"2024-04-25T16:52:25.808455+0000","last_unstale":"2024-04-25T16:52:34.878024+0000","last_undegraded":"2024-04-25T16:52:34.878024+0000","last_fullsized":"2024-04-25T16:52:34.878024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872072+0000","last_change":"2024-04-25T16:52:26.782954+0000","last_active":"2024-04-25T16:52:34.872072+0000","last_peered":"2024-04-25T16:52:34.872072+0000","last_clean":"2024-04-25T16:52:34.872072+0000","last_became_active":"2024-04-25T16:52:26.782108+0000","last_became_peered":"2024-04-25T16:52:26.782108+0000","last_unstale":"2024-04-25T16:52:34.872072+0000","last_undegraded":"2024-04-25T16:52:34.872072+0000","last_fullsized":"2024-04-25T16:52:34.872072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873363+0000","last_change":"2024-04-25T16:52:25.795020+0000","last_active":"2024-04-25T16:52:34.873363+0000","last_peered":"2024-04-25T16:52:34.873363+0000","last_clean":"2024-04-25T16:52:34.873363+0000","last_became_active":"2024-04-25T16:52:25.794633+0000","last_became_peered":"2024-04-25T16:52:25.794633+0000","last_unstale":"2024-04-25T16:52:34.873363+0000","last_undegraded":"2024-04-25T16:52:34.873363+0000","last_fullsized":"2024-04-25T16:52:34.873363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:38.694859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877510+0000","last_change":"2024-04-25T16:52:26.821081+0000","last_active":"2024-04-25T16:52:34.877510+0000","last_peered":"2024-04-25T16:52:34.877510+0000","last_clean":"2024-04-25T16:52:34.877510+0000","last_became_active":"2024-04-25T16:52:26.819337+0000","last_became_peered":"2024-04-25T16:52:26.819337+0000","last_unstale":"2024-04-25T16:52:34.877510+0000","last_undegraded":"2024-04-25T16:52:34.877510+0000","last_fullsized":"2024-04-25T16:52:34.877510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872999+0000","last_change":"2024-04-25T16:52:25.793039+0000","last_active":"2024-04-25T16:52:34.872999+0000","last_peered":"2024-04-25T16:52:34.872999+0000","last_clean":"2024-04-25T16:52:34.872999+0000","last_became_active":"2024-04-25T16:52:25.792552+0000","last_became_peered":"2024-04-25T16:52:25.792552+0000","last_unstale":"2024-04-25T16:52:34.872999+0000","last_undegraded":"2024-04-25T16:52:34.872999+0000","last_fullsized":"2024-04-25T16:52:34.872999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:30.954700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872507+0000","last_change":"2024-04-25T16:52:26.823643+0000","last_active":"2024-04-25T16:52:34.872507+0000","last_peered":"2024-04-25T16:52:34.872507+0000","last_clean":"2024-04-25T16:52:34.872507+0000","last_became_active":"2024-04-25T16:52:26.823348+0000","last_became_peered":"2024-04-25T16:52:26.823348+0000","last_unstale":"2024-04-25T16:52:34.872507+0000","last_undegraded":"2024-04-25T16:52:34.872507+0000","last_fullsized":"2024-04-25T16:52:34.872507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830926+0000","last_change":"2024-04-25T16:52:25.813706+0000","last_active":"2024-04-25T16:52:27.830926+0000","last_peered":"2024-04-25T16:52:27.830926+0000","last_clean":"2024-04-25T16:52:27.830926+0000","last_became_active":"2024-04-25T16:52:25.813387+0000","last_became_peered":"2024-04-25T16:52:25.813387+0000","last_unstale":"2024-04-25T16:52:27.830926+0000","last_undegraded":"2024-04-25T16:52:27.830926+0000","last_fullsized":"2024-04-25T16:52:27.830926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340755+0000","last_change":"2024-04-25T16:52:26.817332+0000","last_active":"2024-04-25T16:52:35.340755+0000","last_peered":"2024-04-25T16:52:35.340755+0000","last_clean":"2024-04-25T16:52:35.340755+0000","last_became_active":"2024-04-25T16:52:26.816020+0000","last_became_peered":"2024-04-25T16:52:26.816020+0000","last_unstale":"2024-04-25T16:52:35.340755+0000","last_undegraded":"2024-04-25T16:52:35.340755+0000","last_fullsized":"2024-04-25T16:52:35.340755+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872306+0000","last_change":"2024-04-25T16:52:25.815545+0000","last_active":"2024-04-25T16:52:34.872306+0000","last_peered":"2024-04-25T16:52:34.872306+0000","last_clean":"2024-04-25T16:52:34.872306+0000","last_became_active":"2024-04-25T16:52:25.814711+0000","last_became_peered":"2024-04-25T16:52:25.814711+0000","last_unstale":"2024-04-25T16:52:34.872306+0000","last_undegraded":"2024-04-25T16:52:34.872306+0000","last_fullsized":"2024-04-25T16:52:34.872306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872682+0000","last_change":"2024-04-25T16:52:26.826567+0000","last_active":"2024-04-25T16:52:34.872682+0000","last_peered":"2024-04-25T16:52:34.872682+0000","last_clean":"2024-04-25T16:52:34.872682+0000","last_became_active":"2024-04-25T16:52:26.826224+0000","last_became_peered":"2024-04-25T16:52:26.826224+0000","last_unstale":"2024-04-25T16:52:34.872682+0000","last_undegraded":"2024-04-25T16:52:34.872682+0000","last_fullsized":"2024-04-25T16:52:34.872682+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:06.823841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":24,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340795+0000","last_change":"2024-04-25T16:52:25.815153+0000","last_active":"2024-04-25T16:52:35.340795+0000","last_peered":"2024-04-25T16:52:35.340795+0000","last_clean":"2024-04-25T16:52:35.340795+0000","last_became_active":"2024-04-25T16:52:25.814360+0000","last_became_peered":"2024-04-25T16:52:25.814360+0000","last_unstale":"2024-04-25T16:52:35.340795+0000","last_undegraded":"2024-04-25T16:52:35.340795+0000","last_fullsized":"2024-04-25T16:52:35.340795+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873169+0000","last_change":"2024-04-25T16:52:26.804704+0000","last_active":"2024-04-25T16:52:34.873169+0000","last_peered":"2024-04-25T16:52:34.873169+0000","last_clean":"2024-04-25T16:52:34.873169+0000","last_became_active":"2024-04-25T16:52:26.803888+0000","last_became_peered":"2024-04-25T16:52:26.803888+0000","last_unstale":"2024-04-25T16:52:34.873169+0000","last_undegraded":"2024-04-25T16:52:34.873169+0000","last_fullsized":"2024-04-25T16:52:34.873169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.640204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878107+0000","last_change":"2024-04-25T16:52:25.818880+0000","last_active":"2024-04-25T16:52:34.878107+0000","last_peered":"2024-04-25T16:52:34.878107+0000","last_clean":"2024-04-25T16:52:34.878107+0000","last_became_active":"2024-04-25T16:52:25.817841+0000","last_became_peered":"2024-04-25T16:52:25.817841+0000","last_unstale":"2024-04-25T16:52:34.878107+0000","last_undegraded":"2024-04-25T16:52:34.878107+0000","last_fullsized":"2024-04-25T16:52:34.878107+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:26.817110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877704+0000","last_change":"2024-04-25T16:52:26.809554+0000","last_active":"2024-04-25T16:52:34.877704+0000","last_peered":"2024-04-25T16:52:34.877704+0000","last_clean":"2024-04-25T16:52:34.877704+0000","last_became_active":"2024-04-25T16:52:26.808709+0000","last_became_peered":"2024-04-25T16:52:26.808709+0000","last_unstale":"2024-04-25T16:52:34.877704+0000","last_undegraded":"2024-04-25T16:52:34.877704+0000","last_fullsized":"2024-04-25T16:52:34.877704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875584+0000","last_change":"2024-04-25T16:52:25.812269+0000","last_active":"2024-04-25T16:52:34.875584+0000","last_peered":"2024-04-25T16:52:34.875584+0000","last_clean":"2024-04-25T16:52:34.875584+0000","last_became_active":"2024-04-25T16:52:25.810831+0000","last_became_peered":"2024-04-25T16:52:25.810831+0000","last_unstale":"2024-04-25T16:52:34.875584+0000","last_undegraded":"2024-04-25T16:52:34.875584+0000","last_fullsized":"2024-04-25T16:52:34.875584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874980+0000","last_change":"2024-04-25T16:52:26.806939+0000","last_active":"2024-04-25T16:52:34.874980+0000","last_peered":"2024-04-25T16:52:34.874980+0000","last_clean":"2024-04-25T16:52:34.874980+0000","last_became_active":"2024-04-25T16:52:26.806053+0000","last_became_peered":"2024-04-25T16:52:26.806053+0000","last_unstale":"2024-04-25T16:52:34.874980+0000","last_undegraded":"2024-04-25T16:52:34.874980+0000","last_fullsized":"2024-04-25T16:52:34.874980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781168+0000","last_change":"2024-04-25T16:52:25.783759+0000","last_active":"2024-04-25T16:52:27.781168+0000","last_peered":"2024-04-25T16:52:27.781168+0000","last_clean":"2024-04-25T16:52:27.781168+0000","last_became_active":"2024-04-25T16:52:25.783353+0000","last_became_peered":"2024-04-25T16:52:25.783353+0000","last_unstale":"2024-04-25T16:52:27.781168+0000","last_undegraded":"2024-04-25T16:52:27.781168+0000","last_fullsized":"2024-04-25T16:52:27.781168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:34.281502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780945+0000","last_change":"2024-04-25T16:52:26.798434+0000","last_active":"2024-04-25T16:52:27.780945+0000","last_peered":"2024-04-25T16:52:27.780945+0000","last_clean":"2024-04-25T16:52:27.780945+0000","last_became_active":"2024-04-25T16:52:26.797631+0000","last_became_peered":"2024-04-25T16:52:26.797631+0000","last_unstale":"2024-04-25T16:52:27.780945+0000","last_undegraded":"2024-04-25T16:52:27.780945+0000","last_fullsized":"2024-04-25T16:52:27.780945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:16.248387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831612+0000","last_change":"2024-04-25T16:52:25.819136+0000","last_active":"2024-04-25T16:52:27.831612+0000","last_peered":"2024-04-25T16:52:27.831612+0000","last_clean":"2024-04-25T16:52:27.831612+0000","last_became_active":"2024-04-25T16:52:25.818809+0000","last_became_peered":"2024-04-25T16:52:25.818809+0000","last_unstale":"2024-04-25T16:52:27.831612+0000","last_undegraded":"2024-04-25T16:52:27.831612+0000","last_fullsized":"2024-04-25T16:52:27.831612+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341292+0000","last_change":"2024-04-25T16:52:26.819612+0000","last_active":"2024-04-25T16:52:35.341292+0000","last_peered":"2024-04-25T16:52:35.341292+0000","last_clean":"2024-04-25T16:52:35.341292+0000","last_became_active":"2024-04-25T16:52:26.818874+0000","last_became_peered":"2024-04-25T16:52:26.818874+0000","last_unstale":"2024-04-25T16:52:35.341292+0000","last_undegraded":"2024-04-25T16:52:35.341292+0000","last_fullsized":"2024-04-25T16:52:35.341292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781486+0000","last_change":"2024-04-25T16:52:25.780874+0000","last_active":"2024-04-25T16:52:27.781486+0000","last_peered":"2024-04-25T16:52:27.781486+0000","last_clean":"2024-04-25T16:52:27.781486+0000","last_became_active":"2024-04-25T16:52:25.780529+0000","last_became_peered":"2024-04-25T16:52:25.780529+0000","last_unstale":"2024-04-25T16:52:27.781486+0000","last_undegraded":"2024-04-25T16:52:27.781486+0000","last_fullsized":"2024-04-25T16:52:27.781486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:57.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878077+0000","last_change":"2024-04-25T16:52:26.845027+0000","last_active":"2024-04-25T16:52:34.878077+0000","last_peered":"2024-04-25T16:52:34.878077+0000","last_clean":"2024-04-25T16:52:34.878077+0000","last_became_active":"2024-04-25T16:52:26.844620+0000","last_became_peered":"2024-04-25T16:52:26.844620+0000","last_unstale":"2024-04-25T16:52:34.878077+0000","last_undegraded":"2024-04-25T16:52:34.878077+0000","last_fullsized":"2024-04-25T16:52:34.878077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780947+0000","last_change":"2024-04-25T16:52:25.821889+0000","last_active":"2024-04-25T16:52:27.780947+0000","last_peered":"2024-04-25T16:52:27.780947+0000","last_clean":"2024-04-25T16:52:27.780947+0000","last_became_active":"2024-04-25T16:52:25.821643+0000","last_became_peered":"2024-04-25T16:52:25.821643+0000","last_unstale":"2024-04-25T16:52:27.780947+0000","last_undegraded":"2024-04-25T16:52:27.780947+0000","last_fullsized":"2024-04-25T16:52:27.780947+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:27.786669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877414+0000","last_change":"2024-04-25T16:52:26.802864+0000","last_active":"2024-04-25T16:52:34.877414+0000","last_peered":"2024-04-25T16:52:34.877414+0000","last_clean":"2024-04-25T16:52:34.877414+0000","last_became_active":"2024-04-25T16:52:26.801913+0000","last_became_peered":"2024-04-25T16:52:26.801913+0000","last_unstale":"2024-04-25T16:52:34.877414+0000","last_undegraded":"2024-04-25T16:52:34.877414+0000","last_fullsized":"2024-04-25T16:52:34.877414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341042+0000","last_change":"2024-04-25T16:52:25.806124+0000","last_active":"2024-04-25T16:52:35.341042+0000","last_peered":"2024-04-25T16:52:35.341042+0000","last_clean":"2024-04-25T16:52:35.341042+0000","last_became_active":"2024-04-25T16:52:25.805306+0000","last_became_peered":"2024-04-25T16:52:25.805306+0000","last_unstale":"2024-04-25T16:52:35.341042+0000","last_undegraded":"2024-04-25T16:52:35.341042+0000","last_fullsized":"2024-04-25T16:52:35.341042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872688+0000","last_change":"2024-04-25T16:52:26.819372+0000","last_active":"2024-04-25T16:52:34.872688+0000","last_peered":"2024-04-25T16:52:34.872688+0000","last_clean":"2024-04-25T16:52:34.872688+0000","last_became_active":"2024-04-25T16:52:26.807182+0000","last_became_peered":"2024-04-25T16:52:26.807182+0000","last_unstale":"2024-04-25T16:52:34.872688+0000","last_undegraded":"2024-04-25T16:52:34.872688+0000","last_fullsized":"2024-04-25T16:52:34.872688+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:14.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780505+0000","last_change":"2024-04-25T16:52:25.788959+0000","last_active":"2024-04-25T16:52:27.780505+0000","last_peered":"2024-04-25T16:52:27.780505+0000","last_clean":"2024-04-25T16:52:27.780505+0000","last_became_active":"2024-04-25T16:52:25.788261+0000","last_became_peered":"2024-04-25T16:52:25.788261+0000","last_unstale":"2024-04-25T16:52:27.780505+0000","last_undegraded":"2024-04-25T16:52:27.780505+0000","last_fullsized":"2024-04-25T16:52:27.780505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:13.943131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831765+0000","last_change":"2024-04-25T16:52:26.819641+0000","last_active":"2024-04-25T16:52:27.831765+0000","last_peered":"2024-04-25T16:52:27.831765+0000","last_clean":"2024-04-25T16:52:27.831765+0000","last_became_active":"2024-04-25T16:52:26.819314+0000","last_became_peered":"2024-04-25T16:52:26.819314+0000","last_unstale":"2024-04-25T16:52:27.831765+0000","last_undegraded":"2024-04-25T16:52:27.831765+0000","last_fullsized":"2024-04-25T16:52:27.831765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878486+0000","last_change":"2024-04-25T16:52:25.806269+0000","last_active":"2024-04-25T16:52:34.878486+0000","last_peered":"2024-04-25T16:52:34.878486+0000","last_clean":"2024-04-25T16:52:34.878486+0000","last_became_active":"2024-04-25T16:52:25.804183+0000","last_became_peered":"2024-04-25T16:52:25.804183+0000","last_unstale":"2024-04-25T16:52:34.878486+0000","last_undegraded":"2024-04-25T16:52:34.878486+0000","last_fullsized":"2024-04-25T16:52:34.878486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:07.902516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341557+0000","last_change":"2024-04-25T16:52:26.841753+0000","last_active":"2024-04-25T16:52:35.341557+0000","last_peered":"2024-04-25T16:52:35.341557+0000","last_clean":"2024-04-25T16:52:35.341557+0000","last_became_active":"2024-04-25T16:52:26.841112+0000","last_became_peered":"2024-04-25T16:52:26.841112+0000","last_unstale":"2024-04-25T16:52:35.341557+0000","last_undegraded":"2024-04-25T16:52:35.341557+0000","last_fullsized":"2024-04-25T16:52:35.341557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781212+0000","last_change":"2024-04-25T16:52:25.803168+0000","last_active":"2024-04-25T16:52:27.781212+0000","last_peered":"2024-04-25T16:52:27.781212+0000","last_clean":"2024-04-25T16:52:27.781212+0000","last_became_active":"2024-04-25T16:52:25.802880+0000","last_became_peered":"2024-04-25T16:52:25.802880+0000","last_unstale":"2024-04-25T16:52:27.781212+0000","last_undegraded":"2024-04-25T16:52:27.781212+0000","last_fullsized":"2024-04-25T16:52:27.781212+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.210482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873797+0000","last_change":"2024-04-25T16:52:26.819781+0000","last_active":"2024-04-25T16:52:34.873797+0000","last_peered":"2024-04-25T16:52:34.873797+0000","last_clean":"2024-04-25T16:52:34.873797+0000","last_became_active":"2024-04-25T16:52:26.819467+0000","last_became_peered":"2024-04-25T16:52:26.819467+0000","last_unstale":"2024-04-25T16:52:34.873797+0000","last_undegraded":"2024-04-25T16:52:34.873797+0000","last_fullsized":"2024-04-25T16:52:34.873797+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878534+0000","last_change":"2024-04-25T16:52:25.799953+0000","last_active":"2024-04-25T16:52:34.878534+0000","last_peered":"2024-04-25T16:52:34.878534+0000","last_clean":"2024-04-25T16:52:34.878534+0000","last_became_active":"2024-04-25T16:52:25.799689+0000","last_became_peered":"2024-04-25T16:52:25.799689+0000","last_unstale":"2024-04-25T16:52:34.878534+0000","last_undegraded":"2024-04-25T16:52:34.878534+0000","last_fullsized":"2024-04-25T16:52:34.878534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877833+0000","last_change":"2024-04-25T16:52:26.799367+0000","last_active":"2024-04-25T16:52:34.877833+0000","last_peered":"2024-04-25T16:52:34.877833+0000","last_clean":"2024-04-25T16:52:34.877833+0000","last_became_active":"2024-04-25T16:52:26.797337+0000","last_became_peered":"2024-04-25T16:52:26.797337+0000","last_unstale":"2024-04-25T16:52:34.877833+0000","last_undegraded":"2024-04-25T16:52:34.877833+0000","last_fullsized":"2024-04-25T16:52:34.877833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878430+0000","last_change":"2024-04-25T16:52:25.806956+0000","last_active":"2024-04-25T16:52:34.878430+0000","last_peered":"2024-04-25T16:52:34.878430+0000","last_clean":"2024-04-25T16:52:34.878430+0000","last_became_active":"2024-04-25T16:52:25.806604+0000","last_became_peered":"2024-04-25T16:52:25.806604+0000","last_unstale":"2024-04-25T16:52:34.878430+0000","last_undegraded":"2024-04-25T16:52:34.878430+0000","last_fullsized":"2024-04-25T16:52:34.878430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831008+0000","last_change":"2024-04-25T16:52:26.820845+0000","last_active":"2024-04-25T16:52:27.831008+0000","last_peered":"2024-04-25T16:52:27.831008+0000","last_clean":"2024-04-25T16:52:27.831008+0000","last_became_active":"2024-04-25T16:52:26.820484+0000","last_became_peered":"2024-04-25T16:52:26.820484+0000","last_unstale":"2024-04-25T16:52:27.831008+0000","last_undegraded":"2024-04-25T16:52:27.831008+0000","last_fullsized":"2024-04-25T16:52:27.831008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:44.359144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874630+0000","last_change":"2024-04-25T16:52:25.785723+0000","last_active":"2024-04-25T16:52:34.874630+0000","last_peered":"2024-04-25T16:52:34.874630+0000","last_clean":"2024-04-25T16:52:34.874630+0000","last_became_active":"2024-04-25T16:52:25.784930+0000","last_became_peered":"2024-04-25T16:52:25.784930+0000","last_unstale":"2024-04-25T16:52:34.874630+0000","last_undegraded":"2024-04-25T16:52:34.874630+0000","last_fullsized":"2024-04-25T16:52:34.874630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872198+0000","last_change":"2024-04-25T16:52:26.799277+0000","last_active":"2024-04-25T16:52:34.872198+0000","last_peered":"2024-04-25T16:52:34.872198+0000","last_clean":"2024-04-25T16:52:34.872198+0000","last_became_active":"2024-04-25T16:52:26.797817+0000","last_became_peered":"2024-04-25T16:52:26.797817+0000","last_unstale":"2024-04-25T16:52:34.872198+0000","last_undegraded":"2024-04-25T16:52:34.872198+0000","last_fullsized":"2024-04-25T16:52:34.872198+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871616+0000","last_change":"2024-04-25T16:52:25.810628+0000","last_active":"2024-04-25T16:52:34.871616+0000","last_peered":"2024-04-25T16:52:34.871616+0000","last_clean":"2024-04-25T16:52:34.871616+0000","last_became_active":"2024-04-25T16:52:25.809669+0000","last_became_peered":"2024-04-25T16:52:25.809669+0000","last_unstale":"2024-04-25T16:52:34.871616+0000","last_undegraded":"2024-04-25T16:52:34.871616+0000","last_fullsized":"2024-04-25T16:52:34.871616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:36.638682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341549+0000","last_change":"2024-04-25T16:52:26.832124+0000","last_active":"2024-04-25T16:52:35.341549+0000","last_peered":"2024-04-25T16:52:35.341549+0000","last_clean":"2024-04-25T16:52:35.341549+0000","last_became_active":"2024-04-25T16:52:26.831437+0000","last_became_peered":"2024-04-25T16:52:26.831437+0000","last_unstale":"2024-04-25T16:52:35.341549+0000","last_undegraded":"2024-04-25T16:52:35.341549+0000","last_fullsized":"2024-04-25T16:52:35.341549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872502+0000","last_change":"2024-04-25T16:52:25.793944+0000","last_active":"2024-04-25T16:52:34.872502+0000","last_peered":"2024-04-25T16:52:34.872502+0000","last_clean":"2024-04-25T16:52:34.872502+0000","last_became_active":"2024-04-25T16:52:25.793522+0000","last_became_peered":"2024-04-25T16:52:25.793522+0000","last_unstale":"2024-04-25T16:52:34.872502+0000","last_undegraded":"2024-04-25T16:52:34.872502+0000","last_fullsized":"2024-04-25T16:52:34.872502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:41.542439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877722+0000","last_change":"2024-04-25T16:52:26.804643+0000","last_active":"2024-04-25T16:52:34.877722+0000","last_peered":"2024-04-25T16:52:34.877722+0000","last_clean":"2024-04-25T16:52:34.877722+0000","last_became_active":"2024-04-25T16:52:26.802297+0000","last_became_peered":"2024-04-25T16:52:26.802297+0000","last_unstale":"2024-04-25T16:52:34.877722+0000","last_undegraded":"2024-04-25T16:52:34.877722+0000","last_fullsized":"2024-04-25T16:52:34.877722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341255+0000","last_change":"2024-04-25T16:52:25.796075+0000","last_active":"2024-04-25T16:52:35.341255+0000","last_peered":"2024-04-25T16:52:35.341255+0000","last_clean":"2024-04-25T16:52:35.341255+0000","last_became_active":"2024-04-25T16:52:25.795226+0000","last_became_peered":"2024-04-25T16:52:25.795226+0000","last_unstale":"2024-04-25T16:52:35.341255+0000","last_undegraded":"2024-04-25T16:52:35.341255+0000","last_fullsized":"2024-04-25T16:52:35.341255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":145,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340402+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:52:35.340402+0000","last_peered":"2024-04-25T16:52:35.340402+0000","last_clean":"2024-04-25T16:52:35.340402+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:52:35.340402+0000","last_undegraded":"2024-04-25T16:52:35.340402+0000","last_fullsized":"2024-04-25T16:52:35.340402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":196969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":196499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":458752,"data_stored":196991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35796,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336044,"statfs":{"total":96636764160,"available":96600109056,"internally_reserved":0,"allocated":466944,"data_stored":198033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":190098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35776,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336064,"statfs":{"total":96636764160,"available":96600129536,"internally_reserved":0,"allocated":442368,"data_stored":190118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335476,"statfs":{"total":96636764160,"available":96599527424,"internally_reserved":0,"allocated":1048576,"data_stored":786789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:52:40.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-04-25T16:52:40.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 13, 'reported_epoch': 38, 'state': 'peering', 'last_fresh': '2024-04-25T16:52:30.709501+0000', 'last_change': '2024-04-25T16:52:34.873590+0000', 'last_active': '2024-04-25T16:52:34.873590+0000', 'last_peered': '2024-04-25T16:52:30.709501+0000', 'last_clean': '2024-04-25T16:52:30.709501+0000', 'last_became_active': '2024-04-25T16:52:26.820528+0000', 'last_became_peered': '2024-04-25T16:52:26.820528+0000', 'last_unstale': '2024-04-25T16:52:30.709501+0000', 'last_undegraded': '2024-04-25T16:52:30.709501+0000', 'last_fullsized': '2024-04-25T16:52:30.709501+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T16:52:25.752315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T16:52:25.752315+0000', 'last_clean_scrub_stamp': '2024-04-25T16:52:25.752315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T16:52:40.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T16:52:41.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.023+0000 7f1d673df640 1 Processor -- start 2024-04-25T16:52:41.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.023+0000 7f1d673df640 1 -- start start 2024-04-25T16:52:41.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d673df640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d600cdaa0 0x7f1d600d2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:41.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d673df640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1d600cdeb0 0x7f1d600d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:41.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d673df640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1d600b6910 0x7f1d600b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:41.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d673df640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f1d600cec90 con 0x7f1d600b6910 2024-04-25T16:52:41.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d673df640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1d600ce3f0 con 0x7f1d600cdeb0 2024-04-25T16:52:41.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d673df640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1d600ce6d0 con 0x7f1d600cdaa0 2024-04-25T16:52:41.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d6597c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1d600b6910 0x7f1d600b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:41.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d6597c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1d600b6910 0x7f1d600b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49782/0 (socket says 172.21.15.94:49782) 2024-04-25T16:52:41.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d6597c640 1 -- 172.21.15.94:0/559721360 learned_addr learned my addr 172.21.15.94:0/559721360 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:41.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d6497a640 1 --2- 172.21.15.94:0/559721360 >> v2:172.21.15.138:3300/0 conn(0x7f1d600cdeb0 0x7f1d600d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:41.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d6517b640 1 --2- 172.21.15.94:0/559721360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d600cdaa0 0x7f1d600d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:41.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d6597c640 1 -- 172.21.15.94:0/559721360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d600cdaa0 msgr2=0x7f1d600d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:41.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d6597c640 1 --2- 172.21.15.94:0/559721360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d600cdaa0 0x7f1d600d2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d6597c640 1 -- 172.21.15.94:0/559721360 >> v2:172.21.15.138:3300/0 conn(0x7f1d600cdeb0 msgr2=0x7f1d600d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:41.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d6597c640 1 --2- 172.21.15.94:0/559721360 >> v2:172.21.15.138:3300/0 conn(0x7f1d600cdeb0 0x7f1d600d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d6597c640 1 -- 172.21.15.94:0/559721360 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d600ce9b0 con 0x7f1d600b6910 2024-04-25T16:52:41.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.027+0000 7f1d6497a640 1 --2- 172.21.15.94:0/559721360 >> v2:172.21.15.138:3300/0 conn(0x7f1d600cdeb0 0x7f1d600d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:41.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d6597c640 1 --2- 172.21.15.94:0/559721360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d600b6910 0x7f1d600b6d20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1d6006dcd0 tx=0x7f1d5801d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d15bafa86f92bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:41.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d57fff640 1 -- 172.21.15.94:0/559721360 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d580a56f0 con 0x7f1d600b6910 2024-04-25T16:52:41.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d57fff640 1 -- 172.21.15.94:0/559721360 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d580a5890 con 0x7f1d600b6910 2024-04-25T16:52:41.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d57fff640 1 -- 172.21.15.94:0/559721360 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d580a5c60 con 0x7f1d600b6910 2024-04-25T16:52:41.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d673df640 1 -- 172.21.15.94:0/559721360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d600b6910 msgr2=0x7f1d600b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:41.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d673df640 1 --2- 172.21.15.94:0/559721360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d600b6910 0x7f1d600b6d20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1d6006dcd0 tx=0x7f1d5801d3c0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d673df640 1 -- 172.21.15.94:0/559721360 shutdown_connections 2024-04-25T16:52:41.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d673df640 1 --2- 172.21.15.94:0/559721360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d600b6910 0x7f1d600b6d20 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d673df640 1 --2- 172.21.15.94:0/559721360 >> v2:172.21.15.138:3300/0 conn(0x7f1d600cdeb0 0x7f1d600d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d673df640 1 --2- 172.21.15.94:0/559721360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d600cdaa0 0x7f1d600d2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d673df640 1 -- 172.21.15.94:0/559721360 >> 172.21.15.94:0/559721360 conn(0x7f1d600cfed0 msgr2=0x7f1d600d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:41.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d673df640 1 -- 172.21.15.94:0/559721360 shutdown_connections 2024-04-25T16:52:41.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d673df640 1 -- 172.21.15.94:0/559721360 wait complete. 2024-04-25T16:52:41.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d673df640 1 Processor -- start 2024-04-25T16:52:41.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.031+0000 7f1d673df640 1 -- start start 2024-04-25T16:52:41.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d673df640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d600b6910 0x7f1d60157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:41.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d673df640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d600cdaa0 0x7f1d60149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:41.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d673df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d600cdeb0 0x7f1d60152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:41.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d673df640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d600cf960 con 0x7f1d600cdeb0 2024-04-25T16:52:41.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d673df640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d600cf7e0 con 0x7f1d600b6910 2024-04-25T16:52:41.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d673df640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1d60169d40 con 0x7f1d600cdaa0 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d6597c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d600cdeb0 0x7f1d60152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d6597c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d600cdeb0 0x7f1d60152b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49784/0 (socket says 172.21.15.94:49784) 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d6597c640 1 -- 172.21.15.94:0/2920813698 learned_addr learned my addr 172.21.15.94:0/2920813698 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d6497a640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d600cdaa0 0x7f1d60149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d6517b640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d600b6910 0x7f1d60157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d6597c640 1 -- 172.21.15.94:0/2920813698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d600cdaa0 msgr2=0x7f1d60149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d6597c640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d600cdaa0 0x7f1d60149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d6597c640 1 -- 172.21.15.94:0/2920813698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d600b6910 msgr2=0x7f1d60157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d6597c640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d600b6910 0x7f1d60157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d6597c640 1 -- 172.21.15.94:0/2920813698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d601498a0 con 0x7f1d600cdeb0 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d6597c640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d600cdeb0 0x7f1d60152b30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1d580af040 tx=0x7f1d58019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d55ffb640 1 -- 172.21.15.94:0/2920813698 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d580bc030 con 0x7f1d600cdeb0 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d55ffb640 1 -- 172.21.15.94:0/2920813698 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d580b8470 con 0x7f1d600cdeb0 2024-04-25T16:52:41.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d673df640 1 -- 172.21.15.94:0/2920813698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6014f0a0 con 0x7f1d600cdeb0 2024-04-25T16:52:41.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d55ffb640 1 -- 172.21.15.94:0/2920813698 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d580b8820 con 0x7f1d600cdeb0 2024-04-25T16:52:41.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d673df640 1 -- 172.21.15.94:0/2920813698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6014f640 con 0x7f1d600cdeb0 2024-04-25T16:52:41.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d673df640 1 -- 172.21.15.94:0/2920813698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6006a730 con 0x7f1d600cdeb0 2024-04-25T16:52:41.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d55ffb640 1 -- 172.21.15.94:0/2920813698 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1d580bf070 con 0x7f1d600cdeb0 2024-04-25T16:52:41.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d55ffb640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d4406f430 0x7f1d44071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:41.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d55ffb640 1 -- 172.21.15.94:0/2920813698 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f1d5812a830 con 0x7f1d600cdeb0 2024-04-25T16:52:41.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.035+0000 7f1d6517b640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d4406f430 0x7f1d44071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:41.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.039+0000 7f1d6517b640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d4406f430 0x7f1d44071940 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1d60073b90 tx=0x7f1d5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:41.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.039+0000 7f1d55ffb640 1 -- 172.21.15.94:0/2920813698 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1d580165e0 con 0x7f1d600cdeb0 2024-04-25T16:52:41.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.255+0000 7f1d673df640 1 -- 172.21.15.94:0/2920813698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1d60153070 con 0x7f1d600cdeb0 2024-04-25T16:52:41.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.255+0000 7f1d55ffb640 1 -- 172.21.15.94:0/2920813698 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1026 (secure 0 0 0) 0x7f1d580f76e0 con 0x7f1d600cdeb0 2024-04-25T16:52:41.253 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:41.253 INFO:teuthology.orchestra.run.smithi094.stdout:{"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":132,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":40,"num_osds":8,"num_up_osds":8,"osd_up_since":1714063840,"num_in_osds":8,"osd_in_since":1714063831,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":96},{"state_name":"peering","count":1}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":264519680,"bytes_avail":772829593600,"bytes_total":773094113280,"inactive_pgs_ratio":0.010309278033673763},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5442}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-25T16:51:59.684427+0000","services":{}},"progress_events":{}} 2024-04-25T16:52:41.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.255+0000 7f1d673df640 1 -- 172.21.15.94:0/2920813698 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d4406f430 msgr2=0x7f1d44071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:41.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.255+0000 7f1d673df640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d4406f430 0x7f1d44071940 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1d60073b90 tx=0x7f1d5c008040 comp rx=0 tx=0).stop 2024-04-25T16:52:41.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.255+0000 7f1d673df640 1 -- 172.21.15.94:0/2920813698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d600cdeb0 msgr2=0x7f1d60152b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:41.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.255+0000 7f1d673df640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d600cdeb0 0x7f1d60152b30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1d580af040 tx=0x7f1d58019700 comp rx=0 tx=0).stop 2024-04-25T16:52:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.259+0000 7f1d673df640 1 -- 172.21.15.94:0/2920813698 shutdown_connections 2024-04-25T16:52:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.259+0000 7f1d673df640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d600cdeb0 0x7f1d60152b30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.259+0000 7f1d673df640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1d4406f430 0x7f1d44071940 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.259+0000 7f1d673df640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d600cdaa0 0x7f1d60149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.259+0000 7f1d673df640 1 --2- 172.21.15.94:0/2920813698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d600b6910 0x7f1d60157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.259+0000 7f1d673df640 1 -- 172.21.15.94:0/2920813698 >> 172.21.15.94:0/2920813698 conn(0x7f1d600cfed0 msgr2=0x7f1d60154440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.259+0000 7f1d673df640 1 -- 172.21.15.94:0/2920813698 shutdown_connections 2024-04-25T16:52:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.259+0000 7f1d673df640 1 -- 172.21.15.94:0/2920813698 wait complete. 2024-04-25T16:52:41.273 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T16:52:41.274 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:52:41.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.423+0000 7f8c0bd60640 1 Processor -- start 2024-04-25T16:52:41.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.423+0000 7f8c0bd60640 1 -- start start 2024-04-25T16:52:41.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c0bd60640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8c0415cf40 0x7f8c0406d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:41.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c0bd60640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c0406dc50 0x7f8c040af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:41.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c0bd60640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0406e060 con 0x7f8c04070180 2024-04-25T16:52:41.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c0bd60640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0415dec0 con 0x7f8c0415cf40 2024-04-25T16:52:41.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c0bd60640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8c04160760 con 0x7f8c0406dc50 2024-04-25T16:52:41.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c0a2fd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c0406dc50 0x7f8c040af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:41.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c092fb640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8c0415cf40 0x7f8c0406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:41.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c0a2fd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c0406dc50 0x7f8c040af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:33160/0 (socket says 172.21.15.94:33160) 2024-04-25T16:52:41.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c0a2fd640 1 -- 172.21.15.94:0/2814658212 learned_addr learned my addr 172.21.15.94:0/2814658212 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:41.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c0a2fd640 1 -- 172.21.15.94:0/2814658212 >> v2:172.21.15.138:3300/0 conn(0x7f8c0415cf40 msgr2=0x7f8c0406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:41.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c0a2fd640 1 --2- 172.21.15.94:0/2814658212 >> v2:172.21.15.138:3300/0 conn(0x7f8c0415cf40 0x7f8c0406d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c0a2fd640 1 -- 172.21.15.94:0/2814658212 >> v1:172.21.15.138:6789/0 conn(0x7f8c04070180 legacy=0x7f8c04160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:41.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c0a2fd640 1 -- 172.21.15.94:0/2814658212 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c04161300 con 0x7f8c0406dc50 2024-04-25T16:52:41.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.427+0000 7f8c092fb640 1 --2- 172.21.15.94:0/2814658212 >> v2:172.21.15.138:3300/0 conn(0x7f8c0415cf40 0x7f8c0406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:41.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.431+0000 7f8c0a2fd640 1 --2- 172.21.15.94:0/2814658212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c0406dc50 0x7f8c040af2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009e50 tx=0x7f8bfc01ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efd8f3035914d1ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:41.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.431+0000 7f8c08afa640 1 -- 172.21.15.94:0/2814658212 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc0a58e0 con 0x7f8c0406dc50 2024-04-25T16:52:41.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.431+0000 7f8c08afa640 1 -- 172.21.15.94:0/2814658212 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bfc0a5a80 con 0x7f8c0406dc50 2024-04-25T16:52:41.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.431+0000 7f8c08afa640 1 -- 172.21.15.94:0/2814658212 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc0a5e50 con 0x7f8c0406dc50 2024-04-25T16:52:41.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.431+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2814658212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c0406dc50 msgr2=0x7f8c040af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:41.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.431+0000 7f8c0bd60640 1 --2- 172.21.15.94:0/2814658212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c0406dc50 0x7f8c040af2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009e50 tx=0x7f8bfc01ca10 comp rx=0 tx=0).stop 2024-04-25T16:52:41.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.431+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2814658212 shutdown_connections 2024-04-25T16:52:41.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.431+0000 7f8c0bd60640 1 --2- 172.21.15.94:0/2814658212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c0406dc50 0x7f8c040af2c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.431+0000 7f8c0bd60640 1 --2- 172.21.15.94:0/2814658212 >> v2:172.21.15.138:3300/0 conn(0x7f8c0415cf40 0x7f8c0406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.431+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2814658212 >> 172.21.15.94:0/2814658212 conn(0x7f8c040b47f0 msgr2=0x7f8c040b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:41.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.431+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2814658212 shutdown_connections 2024-04-25T16:52:41.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.431+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2814658212 wait complete. 2024-04-25T16:52:41.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0bd60640 1 Processor -- start 2024-04-25T16:52:41.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0bd60640 1 -- start start 2024-04-25T16:52:41.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0bd60640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c0406dc50 0x7f8c041f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:41.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0bd60640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c04070180 0x7f8c0414a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:41.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0bd60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0415cf40 0x7f8c0414aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:41.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0bd60640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0406fea0 con 0x7f8c0415cf40 2024-04-25T16:52:41.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0bd60640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0415dbe0 con 0x7f8c04070180 2024-04-25T16:52:41.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0bd60640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8c040afc10 con 0x7f8c0406dc50 2024-04-25T16:52:41.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0a2fd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0415cf40 0x7f8c0414aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:41.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0a2fd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0415cf40 0x7f8c0414aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49800/0 (socket says 172.21.15.94:49800) 2024-04-25T16:52:41.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0a2fd640 1 -- 172.21.15.94:0/2882867797 learned_addr learned my addr 172.21.15.94:0/2882867797 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:41.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c092fb640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c04070180 0x7f8c0414a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:41.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c09afc640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c0406dc50 0x7f8c041f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:41.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0a2fd640 1 -- 172.21.15.94:0/2882867797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c0406dc50 msgr2=0x7f8c041f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:41.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0a2fd640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c0406dc50 0x7f8c041f9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0a2fd640 1 -- 172.21.15.94:0/2882867797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c04070180 msgr2=0x7f8c0414a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:41.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0a2fd640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c04070180 0x7f8c0414a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0a2fd640 1 -- 172.21.15.94:0/2882867797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c04147c80 con 0x7f8c0415cf40 2024-04-25T16:52:41.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c092fb640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c04070180 0x7f8c0414a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:41.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0a2fd640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0415cf40 0x7f8c0414aeb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0af040 tx=0x7f8bfc0197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:41.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8bfa7fc640 1 -- 172.21.15.94:0/2882867797 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc0a58e0 con 0x7f8c0415cf40 2024-04-25T16:52:41.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8bfa7fc640 1 -- 172.21.15.94:0/2882867797 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bfc007c50 con 0x7f8c0415cf40 2024-04-25T16:52:41.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2882867797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c04147070 con 0x7f8c0415cf40 2024-04-25T16:52:41.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8bfa7fc640 1 -- 172.21.15.94:0/2882867797 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc009510 con 0x7f8c0415cf40 2024-04-25T16:52:41.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2882867797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c04147630 con 0x7f8c0415cf40 2024-04-25T16:52:41.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.435+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2882867797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c04146de0 con 0x7f8c0415cf40 2024-04-25T16:52:41.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.439+0000 7f8bfa7fc640 1 -- 172.21.15.94:0/2882867797 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8bfc0c0030 con 0x7f8c0415cf40 2024-04-25T16:52:41.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.439+0000 7f8bfa7fc640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8be806f430 0x7f8be8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:41.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.439+0000 7f8bfa7fc640 1 -- 172.21.15.94:0/2882867797 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f8bfc12a2a0 con 0x7f8c0415cf40 2024-04-25T16:52:41.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.439+0000 7f8c09afc640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8be806f430 0x7f8be8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:41.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.439+0000 7f8c09afc640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8be806f430 0x7f8be8071940 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007fd0 tx=0x7f8c00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:41.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.443+0000 7f8bfa7fc640 1 -- 172.21.15.94:0/2882867797 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8bfc016600 con 0x7f8c0415cf40 2024-04-25T16:52:41.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.591+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2882867797 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c04076830 con 0x7f8be806f430 2024-04-25T16:52:41.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.607+0000 7f8bfa7fc640 1 -- 172.21.15.94:0/2882867797 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251872 (secure 0 0 0) 0x7f8c04076830 con 0x7f8be806f430 2024-04-25T16:52:41.609 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:41.610 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:52:41.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.611+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2882867797 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8be806f430 msgr2=0x7f8be8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:41.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.611+0000 7f8c0bd60640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8be806f430 0x7f8be8071940 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007fd0 tx=0x7f8c00009040 comp rx=0 tx=0).stop 2024-04-25T16:52:41.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.611+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2882867797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0415cf40 msgr2=0x7f8c0414aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:41.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.611+0000 7f8c0bd60640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0415cf40 0x7f8c0414aeb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0af040 tx=0x7f8bfc0197b0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.611+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2882867797 shutdown_connections 2024-04-25T16:52:41.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.611+0000 7f8c0bd60640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0415cf40 0x7f8c0414aeb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.611+0000 7f8c0bd60640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c04070180 0x7f8c0414a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.611+0000 7f8c0bd60640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8be806f430 0x7f8be8071940 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.611+0000 7f8c0bd60640 1 --2- 172.21.15.94:0/2882867797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c0406dc50 0x7f8c041f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:41.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.611+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2882867797 >> 172.21.15.94:0/2882867797 conn(0x7f8c040b47f0 msgr2=0x7f8c0415ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:41.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.611+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2882867797 shutdown_connections 2024-04-25T16:52:41.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:41.611+0000 7f8c0bd60640 1 -- 172.21.15.94:0/2882867797 wait complete. 2024-04-25T16:52:41.628 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-04-25T16:52:39.696569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258320,"kb_used_data":4708,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716400,"statfs":{"total":773094113280,"available":772829593600,"internally_reserved":0,"allocated":4820992,"data_stored":2737575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002542"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873724+0000","last_change":"2024-04-25T16:52:26.825748+0000","last_active":"2024-04-25T16:52:34.873724+0000","last_peered":"2024-04-25T16:52:34.873724+0000","last_clean":"2024-04-25T16:52:34.873724+0000","last_became_active":"2024-04-25T16:52:26.825333+0000","last_became_peered":"2024-04-25T16:52:26.825333+0000","last_unstale":"2024-04-25T16:52:34.873724+0000","last_undegraded":"2024-04-25T16:52:34.873724+0000","last_fullsized":"2024-04-25T16:52:34.873724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:19.084563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873506+0000","last_change":"2024-04-25T16:52:26.795964+0000","last_active":"2024-04-25T16:52:34.873506+0000","last_peered":"2024-04-25T16:52:34.873506+0000","last_clean":"2024-04-25T16:52:34.873506+0000","last_became_active":"2024-04-25T16:52:26.794884+0000","last_became_peered":"2024-04-25T16:52:26.794884+0000","last_unstale":"2024-04-25T16:52:34.873506+0000","last_undegraded":"2024-04-25T16:52:34.873506+0000","last_fullsized":"2024-04-25T16:52:34.873506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878233+0000","last_change":"2024-04-25T16:52:26.802075+0000","last_active":"2024-04-25T16:52:34.878233+0000","last_peered":"2024-04-25T16:52:34.878233+0000","last_clean":"2024-04-25T16:52:34.878233+0000","last_became_active":"2024-04-25T16:52:26.801203+0000","last_became_peered":"2024-04-25T16:52:26.801203+0000","last_unstale":"2024-04-25T16:52:34.878233+0000","last_undegraded":"2024-04-25T16:52:34.878233+0000","last_fullsized":"2024-04-25T16:52:34.878233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782213+0000","last_change":"2024-04-25T16:52:26.793541+0000","last_active":"2024-04-25T16:52:27.782213+0000","last_peered":"2024-04-25T16:52:27.782213+0000","last_clean":"2024-04-25T16:52:27.782213+0000","last_became_active":"2024-04-25T16:52:26.789101+0000","last_became_peered":"2024-04-25T16:52:26.789101+0000","last_unstale":"2024-04-25T16:52:27.782213+0000","last_undegraded":"2024-04-25T16:52:27.782213+0000","last_fullsized":"2024-04-25T16:52:27.782213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:38.013058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781321+0000","last_change":"2024-04-25T16:52:26.784467+0000","last_active":"2024-04-25T16:52:27.781321+0000","last_peered":"2024-04-25T16:52:27.781321+0000","last_clean":"2024-04-25T16:52:27.781321+0000","last_became_active":"2024-04-25T16:52:26.783673+0000","last_became_peered":"2024-04-25T16:52:26.783673+0000","last_unstale":"2024-04-25T16:52:27.781321+0000","last_undegraded":"2024-04-25T16:52:27.781321+0000","last_fullsized":"2024-04-25T16:52:27.781321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:36.583897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831274+0000","last_change":"2024-04-25T16:52:26.823413+0000","last_active":"2024-04-25T16:52:27.831274+0000","last_peered":"2024-04-25T16:52:27.831274+0000","last_clean":"2024-04-25T16:52:27.831274+0000","last_became_active":"2024-04-25T16:52:26.823059+0000","last_became_peered":"2024-04-25T16:52:26.823059+0000","last_unstale":"2024-04-25T16:52:27.831274+0000","last_undegraded":"2024-04-25T16:52:27.831274+0000","last_fullsized":"2024-04-25T16:52:27.831274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.876836+0000","last_change":"2024-04-25T16:52:26.808661+0000","last_active":"2024-04-25T16:52:34.876836+0000","last_peered":"2024-04-25T16:52:34.876836+0000","last_clean":"2024-04-25T16:52:34.876836+0000","last_became_active":"2024-04-25T16:52:26.805296+0000","last_became_peered":"2024-04-25T16:52:26.805296+0000","last_unstale":"2024-04-25T16:52:34.876836+0000","last_undegraded":"2024-04-25T16:52:34.876836+0000","last_fullsized":"2024-04-25T16:52:34.876836+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"peering","last_fresh":"2024-04-25T16:52:30.709501+0000","last_change":"2024-04-25T16:52:34.873590+0000","last_active":"2024-04-25T16:52:34.873590+0000","last_peered":"2024-04-25T16:52:30.709501+0000","last_clean":"2024-04-25T16:52:30.709501+0000","last_became_active":"2024-04-25T16:52:26.820528+0000","last_became_peered":"2024-04-25T16:52:26.820528+0000","last_unstale":"2024-04-25T16:52:30.709501+0000","last_undegraded":"2024-04-25T16:52:30.709501+0000","last_fullsized":"2024-04-25T16:52:30.709501+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.879080+0000","last_change":"2024-04-25T16:52:26.803363+0000","last_active":"2024-04-25T16:52:34.879080+0000","last_peered":"2024-04-25T16:52:34.879080+0000","last_clean":"2024-04-25T16:52:34.879080+0000","last_became_active":"2024-04-25T16:52:26.802539+0000","last_became_peered":"2024-04-25T16:52:26.802539+0000","last_unstale":"2024-04-25T16:52:34.879080+0000","last_undegraded":"2024-04-25T16:52:34.879080+0000","last_fullsized":"2024-04-25T16:52:34.879080+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877890+0000","last_change":"2024-04-25T16:52:26.842728+0000","last_active":"2024-04-25T16:52:34.877890+0000","last_peered":"2024-04-25T16:52:34.877890+0000","last_clean":"2024-04-25T16:52:34.877890+0000","last_became_active":"2024-04-25T16:52:26.842041+0000","last_became_peered":"2024-04-25T16:52:26.842041+0000","last_unstale":"2024-04-25T16:52:34.877890+0000","last_undegraded":"2024-04-25T16:52:34.877890+0000","last_fullsized":"2024-04-25T16:52:34.877890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:33.962706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877607+0000","last_change":"2024-04-25T16:52:26.819329+0000","last_active":"2024-04-25T16:52:34.877607+0000","last_peered":"2024-04-25T16:52:34.877607+0000","last_clean":"2024-04-25T16:52:34.877607+0000","last_became_active":"2024-04-25T16:52:26.818528+0000","last_became_peered":"2024-04-25T16:52:26.818528+0000","last_unstale":"2024-04-25T16:52:34.877607+0000","last_undegraded":"2024-04-25T16:52:34.877607+0000","last_fullsized":"2024-04-25T16:52:34.877607+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878111+0000","last_change":"2024-04-25T16:52:26.795942+0000","last_active":"2024-04-25T16:52:34.878111+0000","last_peered":"2024-04-25T16:52:34.878111+0000","last_clean":"2024-04-25T16:52:34.878111+0000","last_became_active":"2024-04-25T16:52:26.795124+0000","last_became_peered":"2024-04-25T16:52:26.795124+0000","last_unstale":"2024-04-25T16:52:34.878111+0000","last_undegraded":"2024-04-25T16:52:34.878111+0000","last_fullsized":"2024-04-25T16:52:34.878111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877026+0000","last_change":"2024-04-25T16:52:26.814749+0000","last_active":"2024-04-25T16:52:34.877026+0000","last_peered":"2024-04-25T16:52:34.877026+0000","last_clean":"2024-04-25T16:52:34.877026+0000","last_became_active":"2024-04-25T16:52:26.814020+0000","last_became_peered":"2024-04-25T16:52:26.814020+0000","last_unstale":"2024-04-25T16:52:34.877026+0000","last_undegraded":"2024-04-25T16:52:34.877026+0000","last_fullsized":"2024-04-25T16:52:34.877026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871890+0000","last_change":"2024-04-25T16:52:26.813118+0000","last_active":"2024-04-25T16:52:34.871890+0000","last_peered":"2024-04-25T16:52:34.871890+0000","last_clean":"2024-04-25T16:52:34.871890+0000","last_became_active":"2024-04-25T16:52:26.812400+0000","last_became_peered":"2024-04-25T16:52:26.812400+0000","last_unstale":"2024-04-25T16:52:34.871890+0000","last_undegraded":"2024-04-25T16:52:34.871890+0000","last_fullsized":"2024-04-25T16:52:34.871890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:16.169654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877027+0000","last_change":"2024-04-25T16:52:26.795864+0000","last_active":"2024-04-25T16:52:34.877027+0000","last_peered":"2024-04-25T16:52:34.877027+0000","last_clean":"2024-04-25T16:52:34.877027+0000","last_became_active":"2024-04-25T16:52:26.794738+0000","last_became_peered":"2024-04-25T16:52:26.794738+0000","last_unstale":"2024-04-25T16:52:34.877027+0000","last_undegraded":"2024-04-25T16:52:34.877027+0000","last_fullsized":"2024-04-25T16:52:34.877027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340722+0000","last_change":"2024-04-25T16:52:26.836640+0000","last_active":"2024-04-25T16:52:35.340722+0000","last_peered":"2024-04-25T16:52:35.340722+0000","last_clean":"2024-04-25T16:52:35.340722+0000","last_became_active":"2024-04-25T16:52:26.833071+0000","last_became_peered":"2024-04-25T16:52:26.833071+0000","last_unstale":"2024-04-25T16:52:35.340722+0000","last_undegraded":"2024-04-25T16:52:35.340722+0000","last_fullsized":"2024-04-25T16:52:35.340722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341191+0000","last_change":"2024-04-25T16:52:26.841207+0000","last_active":"2024-04-25T16:52:35.341191+0000","last_peered":"2024-04-25T16:52:35.341191+0000","last_clean":"2024-04-25T16:52:35.341191+0000","last_became_active":"2024-04-25T16:52:26.838750+0000","last_became_peered":"2024-04-25T16:52:26.838750+0000","last_unstale":"2024-04-25T16:52:35.341191+0000","last_undegraded":"2024-04-25T16:52:35.341191+0000","last_fullsized":"2024-04-25T16:52:35.341191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875046+0000","last_change":"2024-04-25T16:52:26.795939+0000","last_active":"2024-04-25T16:52:34.875046+0000","last_peered":"2024-04-25T16:52:34.875046+0000","last_clean":"2024-04-25T16:52:34.875046+0000","last_became_active":"2024-04-25T16:52:26.795084+0000","last_became_peered":"2024-04-25T16:52:26.795084+0000","last_unstale":"2024-04-25T16:52:34.875046+0000","last_undegraded":"2024-04-25T16:52:34.875046+0000","last_fullsized":"2024-04-25T16:52:34.875046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781139+0000","last_change":"2024-04-25T16:52:26.818025+0000","last_active":"2024-04-25T16:52:27.781139+0000","last_peered":"2024-04-25T16:52:27.781139+0000","last_clean":"2024-04-25T16:52:27.781139+0000","last_became_active":"2024-04-25T16:52:26.817652+0000","last_became_peered":"2024-04-25T16:52:26.817652+0000","last_unstale":"2024-04-25T16:52:27.781139+0000","last_undegraded":"2024-04-25T16:52:27.781139+0000","last_fullsized":"2024-04-25T16:52:27.781139+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:46.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872811+0000","last_change":"2024-04-25T16:52:26.796384+0000","last_active":"2024-04-25T16:52:34.872811+0000","last_peered":"2024-04-25T16:52:34.872811+0000","last_clean":"2024-04-25T16:52:34.872811+0000","last_became_active":"2024-04-25T16:52:26.795507+0000","last_became_peered":"2024-04-25T16:52:26.795507+0000","last_unstale":"2024-04-25T16:52:34.872811+0000","last_undegraded":"2024-04-25T16:52:34.872811+0000","last_fullsized":"2024-04-25T16:52:34.872811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877104+0000","last_change":"2024-04-25T16:52:26.809988+0000","last_active":"2024-04-25T16:52:34.877104+0000","last_peered":"2024-04-25T16:52:34.877104+0000","last_clean":"2024-04-25T16:52:34.877104+0000","last_became_active":"2024-04-25T16:52:26.809107+0000","last_became_peered":"2024-04-25T16:52:26.809107+0000","last_unstale":"2024-04-25T16:52:34.877104+0000","last_undegraded":"2024-04-25T16:52:34.877104+0000","last_fullsized":"2024-04-25T16:52:34.877104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872865+0000","last_change":"2024-04-25T16:52:26.784466+0000","last_active":"2024-04-25T16:52:34.872865+0000","last_peered":"2024-04-25T16:52:34.872865+0000","last_clean":"2024-04-25T16:52:34.872865+0000","last_became_active":"2024-04-25T16:52:26.783550+0000","last_became_peered":"2024-04-25T16:52:26.783550+0000","last_unstale":"2024-04-25T16:52:34.872865+0000","last_undegraded":"2024-04-25T16:52:34.872865+0000","last_fullsized":"2024-04-25T16:52:34.872865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340668+0000","last_change":"2024-04-25T16:52:26.832452+0000","last_active":"2024-04-25T16:52:35.340668+0000","last_peered":"2024-04-25T16:52:35.340668+0000","last_clean":"2024-04-25T16:52:35.340668+0000","last_became_active":"2024-04-25T16:52:26.831753+0000","last_became_peered":"2024-04-25T16:52:26.831753+0000","last_unstale":"2024-04-25T16:52:35.340668+0000","last_undegraded":"2024-04-25T16:52:35.340668+0000","last_fullsized":"2024-04-25T16:52:35.340668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780752+0000","last_change":"2024-04-25T16:52:26.805633+0000","last_active":"2024-04-25T16:52:27.780752+0000","last_peered":"2024-04-25T16:52:27.780752+0000","last_clean":"2024-04-25T16:52:27.780752+0000","last_became_active":"2024-04-25T16:52:26.804789+0000","last_became_peered":"2024-04-25T16:52:26.804789+0000","last_unstale":"2024-04-25T16:52:27.780752+0000","last_undegraded":"2024-04-25T16:52:27.780752+0000","last_fullsized":"2024-04-25T16:52:27.780752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:08.931465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875174+0000","last_change":"2024-04-25T16:52:26.798693+0000","last_active":"2024-04-25T16:52:34.875174+0000","last_peered":"2024-04-25T16:52:34.875174+0000","last_clean":"2024-04-25T16:52:34.875174+0000","last_became_active":"2024-04-25T16:52:26.797852+0000","last_became_peered":"2024-04-25T16:52:26.797852+0000","last_unstale":"2024-04-25T16:52:34.875174+0000","last_undegraded":"2024-04-25T16:52:34.875174+0000","last_fullsized":"2024-04-25T16:52:34.875174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831376+0000","last_change":"2024-04-25T16:52:26.819630+0000","last_active":"2024-04-25T16:52:27.831376+0000","last_peered":"2024-04-25T16:52:27.831376+0000","last_clean":"2024-04-25T16:52:27.831376+0000","last_became_active":"2024-04-25T16:52:26.819274+0000","last_became_peered":"2024-04-25T16:52:26.819274+0000","last_unstale":"2024-04-25T16:52:27.831376+0000","last_undegraded":"2024-04-25T16:52:27.831376+0000","last_fullsized":"2024-04-25T16:52:27.831376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873718+0000","last_change":"2024-04-25T16:52:26.791433+0000","last_active":"2024-04-25T16:52:34.873718+0000","last_peered":"2024-04-25T16:52:34.873718+0000","last_clean":"2024-04-25T16:52:34.873718+0000","last_became_active":"2024-04-25T16:52:26.790483+0000","last_became_peered":"2024-04-25T16:52:26.790483+0000","last_unstale":"2024-04-25T16:52:34.873718+0000","last_undegraded":"2024-04-25T16:52:34.873718+0000","last_fullsized":"2024-04-25T16:52:34.873718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877762+0000","last_change":"2024-04-25T16:52:26.804860+0000","last_active":"2024-04-25T16:52:34.877762+0000","last_peered":"2024-04-25T16:52:34.877762+0000","last_clean":"2024-04-25T16:52:34.877762+0000","last_became_active":"2024-04-25T16:52:26.804065+0000","last_became_peered":"2024-04-25T16:52:26.804065+0000","last_unstale":"2024-04-25T16:52:34.877762+0000","last_undegraded":"2024-04-25T16:52:34.877762+0000","last_fullsized":"2024-04-25T16:52:34.877762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873009+0000","last_change":"2024-04-25T16:52:26.806705+0000","last_active":"2024-04-25T16:52:34.873009+0000","last_peered":"2024-04-25T16:52:34.873009+0000","last_clean":"2024-04-25T16:52:34.873009+0000","last_became_active":"2024-04-25T16:52:26.805889+0000","last_became_peered":"2024-04-25T16:52:26.805889+0000","last_unstale":"2024-04-25T16:52:34.873009+0000","last_undegraded":"2024-04-25T16:52:34.873009+0000","last_fullsized":"2024-04-25T16:52:34.873009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874258+0000","last_change":"2024-04-25T16:52:26.798903+0000","last_active":"2024-04-25T16:52:34.874258+0000","last_peered":"2024-04-25T16:52:34.874258+0000","last_clean":"2024-04-25T16:52:34.874258+0000","last_became_active":"2024-04-25T16:52:26.798011+0000","last_became_peered":"2024-04-25T16:52:26.798011+0000","last_unstale":"2024-04-25T16:52:34.874258+0000","last_undegraded":"2024-04-25T16:52:34.874258+0000","last_fullsized":"2024-04-25T16:52:34.874258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340837+0000","last_change":"2024-04-25T16:52:26.841037+0000","last_active":"2024-04-25T16:52:35.340837+0000","last_peered":"2024-04-25T16:52:35.340837+0000","last_clean":"2024-04-25T16:52:35.340837+0000","last_became_active":"2024-04-25T16:52:26.838194+0000","last_became_peered":"2024-04-25T16:52:26.838194+0000","last_unstale":"2024-04-25T16:52:35.340837+0000","last_undegraded":"2024-04-25T16:52:35.340837+0000","last_fullsized":"2024-04-25T16:52:35.340837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830883+0000","last_change":"2024-04-25T16:52:26.813243+0000","last_active":"2024-04-25T16:52:27.830883+0000","last_peered":"2024-04-25T16:52:27.830883+0000","last_clean":"2024-04-25T16:52:27.830883+0000","last_became_active":"2024-04-25T16:52:26.811807+0000","last_became_peered":"2024-04-25T16:52:26.811807+0000","last_unstale":"2024-04-25T16:52:27.830883+0000","last_undegraded":"2024-04-25T16:52:27.830883+0000","last_fullsized":"2024-04-25T16:52:27.830883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:42.553053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341094+0000","last_change":"2024-04-25T16:52:26.841017+0000","last_active":"2024-04-25T16:52:35.341094+0000","last_peered":"2024-04-25T16:52:35.341094+0000","last_clean":"2024-04-25T16:52:35.341094+0000","last_became_active":"2024-04-25T16:52:26.837543+0000","last_became_peered":"2024-04-25T16:52:26.837543+0000","last_unstale":"2024-04-25T16:52:35.341094+0000","last_undegraded":"2024-04-25T16:52:35.341094+0000","last_fullsized":"2024-04-25T16:52:35.341094+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875394+0000","last_change":"2024-04-25T16:52:25.818069+0000","last_active":"2024-04-25T16:52:34.875394+0000","last_peered":"2024-04-25T16:52:34.875394+0000","last_clean":"2024-04-25T16:52:34.875394+0000","last_became_active":"2024-04-25T16:52:25.817362+0000","last_became_peered":"2024-04-25T16:52:25.817362+0000","last_unstale":"2024-04-25T16:52:34.875394+0000","last_undegraded":"2024-04-25T16:52:34.875394+0000","last_fullsized":"2024-04-25T16:52:34.875394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:08.488136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833310+0000","last_change":"2024-04-25T16:52:26.829008+0000","last_active":"2024-04-25T16:52:27.833310+0000","last_peered":"2024-04-25T16:52:27.833310+0000","last_clean":"2024-04-25T16:52:27.833310+0000","last_became_active":"2024-04-25T16:52:26.828706+0000","last_became_peered":"2024-04-25T16:52:26.828706+0000","last_unstale":"2024-04-25T16:52:27.833310+0000","last_undegraded":"2024-04-25T16:52:27.833310+0000","last_fullsized":"2024-04-25T16:52:27.833310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878876+0000","last_change":"2024-04-25T16:52:25.795967+0000","last_active":"2024-04-25T16:52:34.878876+0000","last_peered":"2024-04-25T16:52:34.878876+0000","last_clean":"2024-04-25T16:52:34.878876+0000","last_became_active":"2024-04-25T16:52:25.795695+0000","last_became_peered":"2024-04-25T16:52:25.795695+0000","last_unstale":"2024-04-25T16:52:34.878876+0000","last_undegraded":"2024-04-25T16:52:34.878876+0000","last_fullsized":"2024-04-25T16:52:34.878876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875243+0000","last_change":"2024-04-25T16:52:26.789430+0000","last_active":"2024-04-25T16:52:34.875243+0000","last_peered":"2024-04-25T16:52:34.875243+0000","last_clean":"2024-04-25T16:52:34.875243+0000","last_became_active":"2024-04-25T16:52:26.782243+0000","last_became_peered":"2024-04-25T16:52:26.782243+0000","last_unstale":"2024-04-25T16:52:34.875243+0000","last_undegraded":"2024-04-25T16:52:34.875243+0000","last_fullsized":"2024-04-25T16:52:34.875243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.833428+0000","last_change":"2024-04-25T16:52:25.819162+0000","last_active":"2024-04-25T16:52:27.833428+0000","last_peered":"2024-04-25T16:52:27.833428+0000","last_clean":"2024-04-25T16:52:27.833428+0000","last_became_active":"2024-04-25T16:52:25.818859+0000","last_became_peered":"2024-04-25T16:52:25.818859+0000","last_unstale":"2024-04-25T16:52:27.833428+0000","last_undegraded":"2024-04-25T16:52:27.833428+0000","last_fullsized":"2024-04-25T16:52:27.833428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874082+0000","last_change":"2024-04-25T16:52:26.805376+0000","last_active":"2024-04-25T16:52:34.874082+0000","last_peered":"2024-04-25T16:52:34.874082+0000","last_clean":"2024-04-25T16:52:34.874082+0000","last_became_active":"2024-04-25T16:52:26.804542+0000","last_became_peered":"2024-04-25T16:52:26.804542+0000","last_unstale":"2024-04-25T16:52:34.874082+0000","last_undegraded":"2024-04-25T16:52:34.874082+0000","last_fullsized":"2024-04-25T16:52:34.874082+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.620224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874004+0000","last_change":"2024-04-25T16:52:25.815751+0000","last_active":"2024-04-25T16:52:34.874004+0000","last_peered":"2024-04-25T16:52:34.874004+0000","last_clean":"2024-04-25T16:52:34.874004+0000","last_became_active":"2024-04-25T16:52:25.814927+0000","last_became_peered":"2024-04-25T16:52:25.814927+0000","last_unstale":"2024-04-25T16:52:34.874004+0000","last_undegraded":"2024-04-25T16:52:34.874004+0000","last_fullsized":"2024-04-25T16:52:34.874004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877461+0000","last_change":"2024-04-25T16:52:26.814640+0000","last_active":"2024-04-25T16:52:34.877461+0000","last_peered":"2024-04-25T16:52:34.877461+0000","last_clean":"2024-04-25T16:52:34.877461+0000","last_became_active":"2024-04-25T16:52:26.814016+0000","last_became_peered":"2024-04-25T16:52:26.814016+0000","last_unstale":"2024-04-25T16:52:34.877461+0000","last_undegraded":"2024-04-25T16:52:34.877461+0000","last_fullsized":"2024-04-25T16:52:34.877461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877431+0000","last_change":"2024-04-25T16:52:25.805323+0000","last_active":"2024-04-25T16:52:34.877431+0000","last_peered":"2024-04-25T16:52:34.877431+0000","last_clean":"2024-04-25T16:52:34.877431+0000","last_became_active":"2024-04-25T16:52:25.796444+0000","last_became_peered":"2024-04-25T16:52:25.796444+0000","last_unstale":"2024-04-25T16:52:34.877431+0000","last_undegraded":"2024-04-25T16:52:34.877431+0000","last_fullsized":"2024-04-25T16:52:34.877431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:28.996351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877181+0000","last_change":"2024-04-25T16:52:26.821648+0000","last_active":"2024-04-25T16:52:34.877181+0000","last_peered":"2024-04-25T16:52:34.877181+0000","last_clean":"2024-04-25T16:52:34.877181+0000","last_became_active":"2024-04-25T16:52:26.820835+0000","last_became_peered":"2024-04-25T16:52:26.820835+0000","last_unstale":"2024-04-25T16:52:34.877181+0000","last_undegraded":"2024-04-25T16:52:34.877181+0000","last_fullsized":"2024-04-25T16:52:34.877181+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340828+0000","last_change":"2024-04-25T16:52:25.816250+0000","last_active":"2024-04-25T16:52:35.340828+0000","last_peered":"2024-04-25T16:52:35.340828+0000","last_clean":"2024-04-25T16:52:35.340828+0000","last_became_active":"2024-04-25T16:52:25.815656+0000","last_became_peered":"2024-04-25T16:52:25.815656+0000","last_unstale":"2024-04-25T16:52:35.340828+0000","last_undegraded":"2024-04-25T16:52:35.340828+0000","last_fullsized":"2024-04-25T16:52:35.340828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872461+0000","last_change":"2024-04-25T16:52:26.813425+0000","last_active":"2024-04-25T16:52:34.872461+0000","last_peered":"2024-04-25T16:52:34.872461+0000","last_clean":"2024-04-25T16:52:34.872461+0000","last_became_active":"2024-04-25T16:52:26.812713+0000","last_became_peered":"2024-04-25T16:52:26.812713+0000","last_unstale":"2024-04-25T16:52:34.872461+0000","last_undegraded":"2024-04-25T16:52:34.872461+0000","last_fullsized":"2024-04-25T16:52:34.872461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831072+0000","last_change":"2024-04-25T16:52:25.803265+0000","last_active":"2024-04-25T16:52:27.831072+0000","last_peered":"2024-04-25T16:52:27.831072+0000","last_clean":"2024-04-25T16:52:27.831072+0000","last_became_active":"2024-04-25T16:52:25.802617+0000","last_became_peered":"2024-04-25T16:52:25.802617+0000","last_unstale":"2024-04-25T16:52:27.831072+0000","last_undegraded":"2024-04-25T16:52:27.831072+0000","last_fullsized":"2024-04-25T16:52:27.831072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872313+0000","last_change":"2024-04-25T16:52:26.819267+0000","last_active":"2024-04-25T16:52:34.872313+0000","last_peered":"2024-04-25T16:52:34.872313+0000","last_clean":"2024-04-25T16:52:34.872313+0000","last_became_active":"2024-04-25T16:52:26.818740+0000","last_became_peered":"2024-04-25T16:52:26.818740+0000","last_unstale":"2024-04-25T16:52:34.872313+0000","last_undegraded":"2024-04-25T16:52:34.872313+0000","last_fullsized":"2024-04-25T16:52:34.872313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871955+0000","last_change":"2024-04-25T16:52:25.816663+0000","last_active":"2024-04-25T16:52:34.871955+0000","last_peered":"2024-04-25T16:52:34.871955+0000","last_clean":"2024-04-25T16:52:34.871955+0000","last_became_active":"2024-04-25T16:52:25.815420+0000","last_became_peered":"2024-04-25T16:52:25.815420+0000","last_unstale":"2024-04-25T16:52:34.871955+0000","last_undegraded":"2024-04-25T16:52:34.871955+0000","last_fullsized":"2024-04-25T16:52:34.871955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:30.288538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.782313+0000","last_change":"2024-04-25T16:52:26.789927+0000","last_active":"2024-04-25T16:52:27.782313+0000","last_peered":"2024-04-25T16:52:27.782313+0000","last_clean":"2024-04-25T16:52:27.782313+0000","last_became_active":"2024-04-25T16:52:26.788014+0000","last_became_peered":"2024-04-25T16:52:26.788014+0000","last_unstale":"2024-04-25T16:52:27.782313+0000","last_undegraded":"2024-04-25T16:52:27.782313+0000","last_fullsized":"2024-04-25T16:52:27.782313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:47.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873885+0000","last_change":"2024-04-25T16:52:25.798974+0000","last_active":"2024-04-25T16:52:34.873885+0000","last_peered":"2024-04-25T16:52:34.873885+0000","last_clean":"2024-04-25T16:52:34.873885+0000","last_became_active":"2024-04-25T16:52:25.798314+0000","last_became_peered":"2024-04-25T16:52:25.798314+0000","last_unstale":"2024-04-25T16:52:34.873885+0000","last_undegraded":"2024-04-25T16:52:34.873885+0000","last_fullsized":"2024-04-25T16:52:34.873885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.348530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341450+0000","last_change":"2024-04-25T16:52:26.842485+0000","last_active":"2024-04-25T16:52:35.341450+0000","last_peered":"2024-04-25T16:52:35.341450+0000","last_clean":"2024-04-25T16:52:35.341450+0000","last_became_active":"2024-04-25T16:52:26.838813+0000","last_became_peered":"2024-04-25T16:52:26.838813+0000","last_unstale":"2024-04-25T16:52:35.341450+0000","last_undegraded":"2024-04-25T16:52:35.341450+0000","last_fullsized":"2024-04-25T16:52:35.341450+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878521+0000","last_change":"2024-04-25T16:52:25.803289+0000","last_active":"2024-04-25T16:52:34.878521+0000","last_peered":"2024-04-25T16:52:34.878521+0000","last_clean":"2024-04-25T16:52:34.878521+0000","last_became_active":"2024-04-25T16:52:25.801415+0000","last_became_peered":"2024-04-25T16:52:25.801415+0000","last_unstale":"2024-04-25T16:52:34.878521+0000","last_undegraded":"2024-04-25T16:52:34.878521+0000","last_fullsized":"2024-04-25T16:52:34.878521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:45.310089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878182+0000","last_change":"2024-04-25T16:52:26.840379+0000","last_active":"2024-04-25T16:52:34.878182+0000","last_peered":"2024-04-25T16:52:34.878182+0000","last_clean":"2024-04-25T16:52:34.878182+0000","last_became_active":"2024-04-25T16:52:26.840062+0000","last_became_peered":"2024-04-25T16:52:26.840062+0000","last_unstale":"2024-04-25T16:52:34.878182+0000","last_undegraded":"2024-04-25T16:52:34.878182+0000","last_fullsized":"2024-04-25T16:52:34.878182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878024+0000","last_change":"2024-04-25T16:52:25.809391+0000","last_active":"2024-04-25T16:52:34.878024+0000","last_peered":"2024-04-25T16:52:34.878024+0000","last_clean":"2024-04-25T16:52:34.878024+0000","last_became_active":"2024-04-25T16:52:25.808455+0000","last_became_peered":"2024-04-25T16:52:25.808455+0000","last_unstale":"2024-04-25T16:52:34.878024+0000","last_undegraded":"2024-04-25T16:52:34.878024+0000","last_fullsized":"2024-04-25T16:52:34.878024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872072+0000","last_change":"2024-04-25T16:52:26.782954+0000","last_active":"2024-04-25T16:52:34.872072+0000","last_peered":"2024-04-25T16:52:34.872072+0000","last_clean":"2024-04-25T16:52:34.872072+0000","last_became_active":"2024-04-25T16:52:26.782108+0000","last_became_peered":"2024-04-25T16:52:26.782108+0000","last_unstale":"2024-04-25T16:52:34.872072+0000","last_undegraded":"2024-04-25T16:52:34.872072+0000","last_fullsized":"2024-04-25T16:52:34.872072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873363+0000","last_change":"2024-04-25T16:52:25.795020+0000","last_active":"2024-04-25T16:52:34.873363+0000","last_peered":"2024-04-25T16:52:34.873363+0000","last_clean":"2024-04-25T16:52:34.873363+0000","last_became_active":"2024-04-25T16:52:25.794633+0000","last_became_peered":"2024-04-25T16:52:25.794633+0000","last_unstale":"2024-04-25T16:52:34.873363+0000","last_undegraded":"2024-04-25T16:52:34.873363+0000","last_fullsized":"2024-04-25T16:52:34.873363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:38.694859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877510+0000","last_change":"2024-04-25T16:52:26.821081+0000","last_active":"2024-04-25T16:52:34.877510+0000","last_peered":"2024-04-25T16:52:34.877510+0000","last_clean":"2024-04-25T16:52:34.877510+0000","last_became_active":"2024-04-25T16:52:26.819337+0000","last_became_peered":"2024-04-25T16:52:26.819337+0000","last_unstale":"2024-04-25T16:52:34.877510+0000","last_undegraded":"2024-04-25T16:52:34.877510+0000","last_fullsized":"2024-04-25T16:52:34.877510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872999+0000","last_change":"2024-04-25T16:52:25.793039+0000","last_active":"2024-04-25T16:52:34.872999+0000","last_peered":"2024-04-25T16:52:34.872999+0000","last_clean":"2024-04-25T16:52:34.872999+0000","last_became_active":"2024-04-25T16:52:25.792552+0000","last_became_peered":"2024-04-25T16:52:25.792552+0000","last_unstale":"2024-04-25T16:52:34.872999+0000","last_undegraded":"2024-04-25T16:52:34.872999+0000","last_fullsized":"2024-04-25T16:52:34.872999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:30.954700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872507+0000","last_change":"2024-04-25T16:52:26.823643+0000","last_active":"2024-04-25T16:52:34.872507+0000","last_peered":"2024-04-25T16:52:34.872507+0000","last_clean":"2024-04-25T16:52:34.872507+0000","last_became_active":"2024-04-25T16:52:26.823348+0000","last_became_peered":"2024-04-25T16:52:26.823348+0000","last_unstale":"2024-04-25T16:52:34.872507+0000","last_undegraded":"2024-04-25T16:52:34.872507+0000","last_fullsized":"2024-04-25T16:52:34.872507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.830926+0000","last_change":"2024-04-25T16:52:25.813706+0000","last_active":"2024-04-25T16:52:27.830926+0000","last_peered":"2024-04-25T16:52:27.830926+0000","last_clean":"2024-04-25T16:52:27.830926+0000","last_became_active":"2024-04-25T16:52:25.813387+0000","last_became_peered":"2024-04-25T16:52:25.813387+0000","last_unstale":"2024-04-25T16:52:27.830926+0000","last_undegraded":"2024-04-25T16:52:27.830926+0000","last_fullsized":"2024-04-25T16:52:27.830926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340755+0000","last_change":"2024-04-25T16:52:26.817332+0000","last_active":"2024-04-25T16:52:35.340755+0000","last_peered":"2024-04-25T16:52:35.340755+0000","last_clean":"2024-04-25T16:52:35.340755+0000","last_became_active":"2024-04-25T16:52:26.816020+0000","last_became_peered":"2024-04-25T16:52:26.816020+0000","last_unstale":"2024-04-25T16:52:35.340755+0000","last_undegraded":"2024-04-25T16:52:35.340755+0000","last_fullsized":"2024-04-25T16:52:35.340755+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872306+0000","last_change":"2024-04-25T16:52:25.815545+0000","last_active":"2024-04-25T16:52:34.872306+0000","last_peered":"2024-04-25T16:52:34.872306+0000","last_clean":"2024-04-25T16:52:34.872306+0000","last_became_active":"2024-04-25T16:52:25.814711+0000","last_became_peered":"2024-04-25T16:52:25.814711+0000","last_unstale":"2024-04-25T16:52:34.872306+0000","last_undegraded":"2024-04-25T16:52:34.872306+0000","last_fullsized":"2024-04-25T16:52:34.872306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872682+0000","last_change":"2024-04-25T16:52:26.826567+0000","last_active":"2024-04-25T16:52:34.872682+0000","last_peered":"2024-04-25T16:52:34.872682+0000","last_clean":"2024-04-25T16:52:34.872682+0000","last_became_active":"2024-04-25T16:52:26.826224+0000","last_became_peered":"2024-04-25T16:52:26.826224+0000","last_unstale":"2024-04-25T16:52:34.872682+0000","last_undegraded":"2024-04-25T16:52:34.872682+0000","last_fullsized":"2024-04-25T16:52:34.872682+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:06.823841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":24,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340795+0000","last_change":"2024-04-25T16:52:25.815153+0000","last_active":"2024-04-25T16:52:35.340795+0000","last_peered":"2024-04-25T16:52:35.340795+0000","last_clean":"2024-04-25T16:52:35.340795+0000","last_became_active":"2024-04-25T16:52:25.814360+0000","last_became_peered":"2024-04-25T16:52:25.814360+0000","last_unstale":"2024-04-25T16:52:35.340795+0000","last_undegraded":"2024-04-25T16:52:35.340795+0000","last_fullsized":"2024-04-25T16:52:35.340795+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873169+0000","last_change":"2024-04-25T16:52:26.804704+0000","last_active":"2024-04-25T16:52:34.873169+0000","last_peered":"2024-04-25T16:52:34.873169+0000","last_clean":"2024-04-25T16:52:34.873169+0000","last_became_active":"2024-04-25T16:52:26.803888+0000","last_became_peered":"2024-04-25T16:52:26.803888+0000","last_unstale":"2024-04-25T16:52:34.873169+0000","last_undegraded":"2024-04-25T16:52:34.873169+0000","last_fullsized":"2024-04-25T16:52:34.873169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.640204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878107+0000","last_change":"2024-04-25T16:52:25.818880+0000","last_active":"2024-04-25T16:52:34.878107+0000","last_peered":"2024-04-25T16:52:34.878107+0000","last_clean":"2024-04-25T16:52:34.878107+0000","last_became_active":"2024-04-25T16:52:25.817841+0000","last_became_peered":"2024-04-25T16:52:25.817841+0000","last_unstale":"2024-04-25T16:52:34.878107+0000","last_undegraded":"2024-04-25T16:52:34.878107+0000","last_fullsized":"2024-04-25T16:52:34.878107+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:26.817110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877704+0000","last_change":"2024-04-25T16:52:26.809554+0000","last_active":"2024-04-25T16:52:34.877704+0000","last_peered":"2024-04-25T16:52:34.877704+0000","last_clean":"2024-04-25T16:52:34.877704+0000","last_became_active":"2024-04-25T16:52:26.808709+0000","last_became_peered":"2024-04-25T16:52:26.808709+0000","last_unstale":"2024-04-25T16:52:34.877704+0000","last_undegraded":"2024-04-25T16:52:34.877704+0000","last_fullsized":"2024-04-25T16:52:34.877704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.875584+0000","last_change":"2024-04-25T16:52:25.812269+0000","last_active":"2024-04-25T16:52:34.875584+0000","last_peered":"2024-04-25T16:52:34.875584+0000","last_clean":"2024-04-25T16:52:34.875584+0000","last_became_active":"2024-04-25T16:52:25.810831+0000","last_became_peered":"2024-04-25T16:52:25.810831+0000","last_unstale":"2024-04-25T16:52:34.875584+0000","last_undegraded":"2024-04-25T16:52:34.875584+0000","last_fullsized":"2024-04-25T16:52:34.875584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874980+0000","last_change":"2024-04-25T16:52:26.806939+0000","last_active":"2024-04-25T16:52:34.874980+0000","last_peered":"2024-04-25T16:52:34.874980+0000","last_clean":"2024-04-25T16:52:34.874980+0000","last_became_active":"2024-04-25T16:52:26.806053+0000","last_became_peered":"2024-04-25T16:52:26.806053+0000","last_unstale":"2024-04-25T16:52:34.874980+0000","last_undegraded":"2024-04-25T16:52:34.874980+0000","last_fullsized":"2024-04-25T16:52:34.874980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781168+0000","last_change":"2024-04-25T16:52:25.783759+0000","last_active":"2024-04-25T16:52:27.781168+0000","last_peered":"2024-04-25T16:52:27.781168+0000","last_clean":"2024-04-25T16:52:27.781168+0000","last_became_active":"2024-04-25T16:52:25.783353+0000","last_became_peered":"2024-04-25T16:52:25.783353+0000","last_unstale":"2024-04-25T16:52:27.781168+0000","last_undegraded":"2024-04-25T16:52:27.781168+0000","last_fullsized":"2024-04-25T16:52:27.781168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:34.281502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780945+0000","last_change":"2024-04-25T16:52:26.798434+0000","last_active":"2024-04-25T16:52:27.780945+0000","last_peered":"2024-04-25T16:52:27.780945+0000","last_clean":"2024-04-25T16:52:27.780945+0000","last_became_active":"2024-04-25T16:52:26.797631+0000","last_became_peered":"2024-04-25T16:52:26.797631+0000","last_unstale":"2024-04-25T16:52:27.780945+0000","last_undegraded":"2024-04-25T16:52:27.780945+0000","last_fullsized":"2024-04-25T16:52:27.780945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:16.248387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831612+0000","last_change":"2024-04-25T16:52:25.819136+0000","last_active":"2024-04-25T16:52:27.831612+0000","last_peered":"2024-04-25T16:52:27.831612+0000","last_clean":"2024-04-25T16:52:27.831612+0000","last_became_active":"2024-04-25T16:52:25.818809+0000","last_became_peered":"2024-04-25T16:52:25.818809+0000","last_unstale":"2024-04-25T16:52:27.831612+0000","last_undegraded":"2024-04-25T16:52:27.831612+0000","last_fullsized":"2024-04-25T16:52:27.831612+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341292+0000","last_change":"2024-04-25T16:52:26.819612+0000","last_active":"2024-04-25T16:52:35.341292+0000","last_peered":"2024-04-25T16:52:35.341292+0000","last_clean":"2024-04-25T16:52:35.341292+0000","last_became_active":"2024-04-25T16:52:26.818874+0000","last_became_peered":"2024-04-25T16:52:26.818874+0000","last_unstale":"2024-04-25T16:52:35.341292+0000","last_undegraded":"2024-04-25T16:52:35.341292+0000","last_fullsized":"2024-04-25T16:52:35.341292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781486+0000","last_change":"2024-04-25T16:52:25.780874+0000","last_active":"2024-04-25T16:52:27.781486+0000","last_peered":"2024-04-25T16:52:27.781486+0000","last_clean":"2024-04-25T16:52:27.781486+0000","last_became_active":"2024-04-25T16:52:25.780529+0000","last_became_peered":"2024-04-25T16:52:25.780529+0000","last_unstale":"2024-04-25T16:52:27.781486+0000","last_undegraded":"2024-04-25T16:52:27.781486+0000","last_fullsized":"2024-04-25T16:52:27.781486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:57.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878077+0000","last_change":"2024-04-25T16:52:26.845027+0000","last_active":"2024-04-25T16:52:34.878077+0000","last_peered":"2024-04-25T16:52:34.878077+0000","last_clean":"2024-04-25T16:52:34.878077+0000","last_became_active":"2024-04-25T16:52:26.844620+0000","last_became_peered":"2024-04-25T16:52:26.844620+0000","last_unstale":"2024-04-25T16:52:34.878077+0000","last_undegraded":"2024-04-25T16:52:34.878077+0000","last_fullsized":"2024-04-25T16:52:34.878077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780947+0000","last_change":"2024-04-25T16:52:25.821889+0000","last_active":"2024-04-25T16:52:27.780947+0000","last_peered":"2024-04-25T16:52:27.780947+0000","last_clean":"2024-04-25T16:52:27.780947+0000","last_became_active":"2024-04-25T16:52:25.821643+0000","last_became_peered":"2024-04-25T16:52:25.821643+0000","last_unstale":"2024-04-25T16:52:27.780947+0000","last_undegraded":"2024-04-25T16:52:27.780947+0000","last_fullsized":"2024-04-25T16:52:27.780947+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:27.786669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877414+0000","last_change":"2024-04-25T16:52:26.802864+0000","last_active":"2024-04-25T16:52:34.877414+0000","last_peered":"2024-04-25T16:52:34.877414+0000","last_clean":"2024-04-25T16:52:34.877414+0000","last_became_active":"2024-04-25T16:52:26.801913+0000","last_became_peered":"2024-04-25T16:52:26.801913+0000","last_unstale":"2024-04-25T16:52:34.877414+0000","last_undegraded":"2024-04-25T16:52:34.877414+0000","last_fullsized":"2024-04-25T16:52:34.877414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341042+0000","last_change":"2024-04-25T16:52:25.806124+0000","last_active":"2024-04-25T16:52:35.341042+0000","last_peered":"2024-04-25T16:52:35.341042+0000","last_clean":"2024-04-25T16:52:35.341042+0000","last_became_active":"2024-04-25T16:52:25.805306+0000","last_became_peered":"2024-04-25T16:52:25.805306+0000","last_unstale":"2024-04-25T16:52:35.341042+0000","last_undegraded":"2024-04-25T16:52:35.341042+0000","last_fullsized":"2024-04-25T16:52:35.341042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872688+0000","last_change":"2024-04-25T16:52:26.819372+0000","last_active":"2024-04-25T16:52:34.872688+0000","last_peered":"2024-04-25T16:52:34.872688+0000","last_clean":"2024-04-25T16:52:34.872688+0000","last_became_active":"2024-04-25T16:52:26.807182+0000","last_became_peered":"2024-04-25T16:52:26.807182+0000","last_unstale":"2024-04-25T16:52:34.872688+0000","last_undegraded":"2024-04-25T16:52:34.872688+0000","last_fullsized":"2024-04-25T16:52:34.872688+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:14.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.780505+0000","last_change":"2024-04-25T16:52:25.788959+0000","last_active":"2024-04-25T16:52:27.780505+0000","last_peered":"2024-04-25T16:52:27.780505+0000","last_clean":"2024-04-25T16:52:27.780505+0000","last_became_active":"2024-04-25T16:52:25.788261+0000","last_became_peered":"2024-04-25T16:52:25.788261+0000","last_unstale":"2024-04-25T16:52:27.780505+0000","last_undegraded":"2024-04-25T16:52:27.780505+0000","last_fullsized":"2024-04-25T16:52:27.780505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:13.943131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831765+0000","last_change":"2024-04-25T16:52:26.819641+0000","last_active":"2024-04-25T16:52:27.831765+0000","last_peered":"2024-04-25T16:52:27.831765+0000","last_clean":"2024-04-25T16:52:27.831765+0000","last_became_active":"2024-04-25T16:52:26.819314+0000","last_became_peered":"2024-04-25T16:52:26.819314+0000","last_unstale":"2024-04-25T16:52:27.831765+0000","last_undegraded":"2024-04-25T16:52:27.831765+0000","last_fullsized":"2024-04-25T16:52:27.831765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878486+0000","last_change":"2024-04-25T16:52:25.806269+0000","last_active":"2024-04-25T16:52:34.878486+0000","last_peered":"2024-04-25T16:52:34.878486+0000","last_clean":"2024-04-25T16:52:34.878486+0000","last_became_active":"2024-04-25T16:52:25.804183+0000","last_became_peered":"2024-04-25T16:52:25.804183+0000","last_unstale":"2024-04-25T16:52:34.878486+0000","last_undegraded":"2024-04-25T16:52:34.878486+0000","last_fullsized":"2024-04-25T16:52:34.878486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:07.902516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341557+0000","last_change":"2024-04-25T16:52:26.841753+0000","last_active":"2024-04-25T16:52:35.341557+0000","last_peered":"2024-04-25T16:52:35.341557+0000","last_clean":"2024-04-25T16:52:35.341557+0000","last_became_active":"2024-04-25T16:52:26.841112+0000","last_became_peered":"2024-04-25T16:52:26.841112+0000","last_unstale":"2024-04-25T16:52:35.341557+0000","last_undegraded":"2024-04-25T16:52:35.341557+0000","last_fullsized":"2024-04-25T16:52:35.341557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.781212+0000","last_change":"2024-04-25T16:52:25.803168+0000","last_active":"2024-04-25T16:52:27.781212+0000","last_peered":"2024-04-25T16:52:27.781212+0000","last_clean":"2024-04-25T16:52:27.781212+0000","last_became_active":"2024-04-25T16:52:25.802880+0000","last_became_peered":"2024-04-25T16:52:25.802880+0000","last_unstale":"2024-04-25T16:52:27.781212+0000","last_undegraded":"2024-04-25T16:52:27.781212+0000","last_fullsized":"2024-04-25T16:52:27.781212+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.210482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.873797+0000","last_change":"2024-04-25T16:52:26.819781+0000","last_active":"2024-04-25T16:52:34.873797+0000","last_peered":"2024-04-25T16:52:34.873797+0000","last_clean":"2024-04-25T16:52:34.873797+0000","last_became_active":"2024-04-25T16:52:26.819467+0000","last_became_peered":"2024-04-25T16:52:26.819467+0000","last_unstale":"2024-04-25T16:52:34.873797+0000","last_undegraded":"2024-04-25T16:52:34.873797+0000","last_fullsized":"2024-04-25T16:52:34.873797+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878534+0000","last_change":"2024-04-25T16:52:25.799953+0000","last_active":"2024-04-25T16:52:34.878534+0000","last_peered":"2024-04-25T16:52:34.878534+0000","last_clean":"2024-04-25T16:52:34.878534+0000","last_became_active":"2024-04-25T16:52:25.799689+0000","last_became_peered":"2024-04-25T16:52:25.799689+0000","last_unstale":"2024-04-25T16:52:34.878534+0000","last_undegraded":"2024-04-25T16:52:34.878534+0000","last_fullsized":"2024-04-25T16:52:34.878534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877833+0000","last_change":"2024-04-25T16:52:26.799367+0000","last_active":"2024-04-25T16:52:34.877833+0000","last_peered":"2024-04-25T16:52:34.877833+0000","last_clean":"2024-04-25T16:52:34.877833+0000","last_became_active":"2024-04-25T16:52:26.797337+0000","last_became_peered":"2024-04-25T16:52:26.797337+0000","last_unstale":"2024-04-25T16:52:34.877833+0000","last_undegraded":"2024-04-25T16:52:34.877833+0000","last_fullsized":"2024-04-25T16:52:34.877833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.878430+0000","last_change":"2024-04-25T16:52:25.806956+0000","last_active":"2024-04-25T16:52:34.878430+0000","last_peered":"2024-04-25T16:52:34.878430+0000","last_clean":"2024-04-25T16:52:34.878430+0000","last_became_active":"2024-04-25T16:52:25.806604+0000","last_became_peered":"2024-04-25T16:52:25.806604+0000","last_unstale":"2024-04-25T16:52:34.878430+0000","last_undegraded":"2024-04-25T16:52:34.878430+0000","last_fullsized":"2024-04-25T16:52:34.878430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T16:52:27.831008+0000","last_change":"2024-04-25T16:52:26.820845+0000","last_active":"2024-04-25T16:52:27.831008+0000","last_peered":"2024-04-25T16:52:27.831008+0000","last_clean":"2024-04-25T16:52:27.831008+0000","last_became_active":"2024-04-25T16:52:26.820484+0000","last_became_peered":"2024-04-25T16:52:26.820484+0000","last_unstale":"2024-04-25T16:52:27.831008+0000","last_undegraded":"2024-04-25T16:52:27.831008+0000","last_fullsized":"2024-04-25T16:52:27.831008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:44.359144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.874630+0000","last_change":"2024-04-25T16:52:25.785723+0000","last_active":"2024-04-25T16:52:34.874630+0000","last_peered":"2024-04-25T16:52:34.874630+0000","last_clean":"2024-04-25T16:52:34.874630+0000","last_became_active":"2024-04-25T16:52:25.784930+0000","last_became_peered":"2024-04-25T16:52:25.784930+0000","last_unstale":"2024-04-25T16:52:34.874630+0000","last_undegraded":"2024-04-25T16:52:34.874630+0000","last_fullsized":"2024-04-25T16:52:34.874630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872198+0000","last_change":"2024-04-25T16:52:26.799277+0000","last_active":"2024-04-25T16:52:34.872198+0000","last_peered":"2024-04-25T16:52:34.872198+0000","last_clean":"2024-04-25T16:52:34.872198+0000","last_became_active":"2024-04-25T16:52:26.797817+0000","last_became_peered":"2024-04-25T16:52:26.797817+0000","last_unstale":"2024-04-25T16:52:34.872198+0000","last_undegraded":"2024-04-25T16:52:34.872198+0000","last_fullsized":"2024-04-25T16:52:34.872198+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.871616+0000","last_change":"2024-04-25T16:52:25.810628+0000","last_active":"2024-04-25T16:52:34.871616+0000","last_peered":"2024-04-25T16:52:34.871616+0000","last_clean":"2024-04-25T16:52:34.871616+0000","last_became_active":"2024-04-25T16:52:25.809669+0000","last_became_peered":"2024-04-25T16:52:25.809669+0000","last_unstale":"2024-04-25T16:52:34.871616+0000","last_undegraded":"2024-04-25T16:52:34.871616+0000","last_fullsized":"2024-04-25T16:52:34.871616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:36.638682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341549+0000","last_change":"2024-04-25T16:52:26.832124+0000","last_active":"2024-04-25T16:52:35.341549+0000","last_peered":"2024-04-25T16:52:35.341549+0000","last_clean":"2024-04-25T16:52:35.341549+0000","last_became_active":"2024-04-25T16:52:26.831437+0000","last_became_peered":"2024-04-25T16:52:26.831437+0000","last_unstale":"2024-04-25T16:52:35.341549+0000","last_undegraded":"2024-04-25T16:52:35.341549+0000","last_fullsized":"2024-04-25T16:52:35.341549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.872502+0000","last_change":"2024-04-25T16:52:25.793944+0000","last_active":"2024-04-25T16:52:34.872502+0000","last_peered":"2024-04-25T16:52:34.872502+0000","last_clean":"2024-04-25T16:52:34.872502+0000","last_became_active":"2024-04-25T16:52:25.793522+0000","last_became_peered":"2024-04-25T16:52:25.793522+0000","last_unstale":"2024-04-25T16:52:34.872502+0000","last_undegraded":"2024-04-25T16:52:34.872502+0000","last_fullsized":"2024-04-25T16:52:34.872502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:41.542439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:34.877722+0000","last_change":"2024-04-25T16:52:26.804643+0000","last_active":"2024-04-25T16:52:34.877722+0000","last_peered":"2024-04-25T16:52:34.877722+0000","last_clean":"2024-04-25T16:52:34.877722+0000","last_became_active":"2024-04-25T16:52:26.802297+0000","last_became_peered":"2024-04-25T16:52:26.802297+0000","last_unstale":"2024-04-25T16:52:34.877722+0000","last_undegraded":"2024-04-25T16:52:34.877722+0000","last_fullsized":"2024-04-25T16:52:34.877722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.341255+0000","last_change":"2024-04-25T16:52:25.796075+0000","last_active":"2024-04-25T16:52:35.341255+0000","last_peered":"2024-04-25T16:52:35.341255+0000","last_clean":"2024-04-25T16:52:35.341255+0000","last_became_active":"2024-04-25T16:52:25.795226+0000","last_became_peered":"2024-04-25T16:52:25.795226+0000","last_unstale":"2024-04-25T16:52:35.341255+0000","last_undegraded":"2024-04-25T16:52:35.341255+0000","last_fullsized":"2024-04-25T16:52:35.341255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":145,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.340402+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:52:35.340402+0000","last_peered":"2024-04-25T16:52:35.340402+0000","last_clean":"2024-04-25T16:52:35.340402+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:52:35.340402+0000","last_undegraded":"2024-04-25T16:52:35.340402+0000","last_fullsized":"2024-04-25T16:52:35.340402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":196969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":196499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":458752,"data_stored":196991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35796,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336044,"statfs":{"total":96636764160,"available":96600109056,"internally_reserved":0,"allocated":466944,"data_stored":198033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":190098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35776,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336064,"statfs":{"total":96636764160,"available":96600129536,"internally_reserved":0,"allocated":442368,"data_stored":190118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335476,"statfs":{"total":96636764160,"available":96599527424,"internally_reserved":0,"allocated":1048576,"data_stored":786789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:52:44.518 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:52:44.521 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:52:44.634 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:52:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.795+0000 7f3c394e8640 1 Processor -- start 2024-04-25T16:52:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.795+0000 7f3c394e8640 1 -- start start 2024-04-25T16:52:44.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.795+0000 7f3c394e8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3c340d1580 0x7f3c34144c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:44.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.795+0000 7f3c394e8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c340d1990 con 0x7f3c34070510 2024-04-25T16:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.795+0000 7f3c394e8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c3406d0e0 con 0x7f3c340d1170 2024-04-25T16:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.795+0000 7f3c394e8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3c3406ce90 con 0x7f3c340d1580 2024-04-25T16:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.795+0000 7f3c327fc640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f3c34070510 0x7f3c34144530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:33598/0 (socket says 172.21.15.94:33598) 2024-04-25T16:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.795+0000 7f3c327fc640 1 -- 172.21.15.94:0/4220889577 learned_addr learned my addr 172.21.15.94:0/4220889577 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.795+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 511505142 0 0) 0x7f3c28001120 con 0x7f3c34070510 2024-04-25T16:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.795+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c1c003530 con 0x7f3c34070510 2024-04-25T16:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.795+0000 7f3c337fe640 1 --2- 172.21.15.94:0/4220889577 >> v2:172.21.15.138:3300/0 conn(0x7f3c340d1580 0x7f3c34144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1334896280 0 0) 0x7f3c1c003530 con 0x7f3c34070510 2024-04-25T16:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 >> v2:172.21.15.138:3300/0 conn(0x7f3c340d1580 msgr2=0x7f3c34144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 --2- 172.21.15.94:0/4220889577 >> v2:172.21.15.138:3300/0 conn(0x7f3c340d1580 0x7f3c34144c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 >> v1:172.21.15.138:6789/0 conn(0x7f3c340d1170 legacy=0x7f3c3406fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3406d3c0 con 0x7f3c34070510 2024-04-25T16:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f3c28001850 con 0x7f3c34070510 2024-04-25T16:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c337fe640 1 --2- 172.21.15.94:0/4220889577 >> v2:172.21.15.138:3300/0 conn(0x7f3c340d1580 0x7f3c34144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 >> v1:172.21.15.94:6789/0 conn(0x7f3c34070510 legacy=0x7f3c34144530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 --2- 172.21.15.94:0/4220889577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c1c0039c0 0x7f3c1c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 --2- 172.21.15.94:0/4220889577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c1c004810 0x7f3c1c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 --2- 172.21.15.94:0/4220889577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c1c005280 0x7f3c1c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3406ce90 con 0x7f3c1c004810 2024-04-25T16:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c00c210 con 0x7f3c1c0039c0 2024-04-25T16:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1c00c410 con 0x7f3c1c005280 2024-04-25T16:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c327fc640 1 --2- 172.21.15.94:0/4220889577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c1c004810 0x7f3c1c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c32ffd640 1 --2- 172.21.15.94:0/4220889577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c1c0039c0 0x7f3c1c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c337fe640 1 --2- 172.21.15.94:0/4220889577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c1c005280 0x7f3c1c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c327fc640 1 -- 172.21.15.94:0/4220889577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c1c005280 msgr2=0x7f3c1c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c327fc640 1 --2- 172.21.15.94:0/4220889577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c1c005280 0x7f3c1c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c327fc640 1 -- 172.21.15.94:0/4220889577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c1c0039c0 msgr2=0x7f3c1c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c327fc640 1 --2- 172.21.15.94:0/4220889577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c1c0039c0 0x7f3c1c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c327fc640 1 -- 172.21.15.94:0/4220889577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c003530 con 0x7f3c1c004810 2024-04-25T16:52:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c337fe640 1 -- 172.21.15.94:0/4220889577 reap_dead start 2024-04-25T16:52:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c32ffd640 1 --2- 172.21.15.94:0/4220889577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c1c0039c0 0x7f3c1c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c327fc640 1 --2- 172.21.15.94:0/4220889577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c1c004810 0x7f3c1c004c20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800c4c0 tx=0x7f3c2801f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a05b16acfa8858 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800c0e0 con 0x7f3c1c004810 2024-04-25T16:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c28005b00 con 0x7f3c1c004810 2024-04-25T16:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c31ffb640 1 -- 172.21.15.94:0/4220889577 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28007070 con 0x7f3c1c004810 2024-04-25T16:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c394e8640 1 -- 172.21.15.94:0/4220889577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c1c004810 msgr2=0x7f3c1c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c394e8640 1 --2- 172.21.15.94:0/4220889577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c1c004810 0x7f3c1c004c20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800c4c0 tx=0x7f3c2801f350 comp rx=0 tx=0).stop 2024-04-25T16:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c394e8640 1 -- 172.21.15.94:0/4220889577 shutdown_connections 2024-04-25T16:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c394e8640 1 --2- 172.21.15.94:0/4220889577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c1c004810 0x7f3c1c004c20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.799+0000 7f3c394e8640 1 -- 172.21.15.94:0/4220889577 >> 172.21.15.94:0/4220889577 conn(0x7f3c34070f40 msgr2=0x7f3c34071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c394e8640 1 -- 172.21.15.94:0/4220889577 shutdown_connections 2024-04-25T16:52:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c394e8640 1 -- 172.21.15.94:0/4220889577 wait complete. 2024-04-25T16:52:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c394e8640 1 Processor -- start 2024-04-25T16:52:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c394e8640 1 -- start start 2024-04-25T16:52:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c394e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c1c004810 0x7f3c340b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c394e8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c340b66a0 0x7f3c341e9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c394e8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c340b6ab0 0x7f3c3412f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c394e8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34147230 con 0x7f3c1c004810 2024-04-25T16:52:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c394e8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3406d980 con 0x7f3c340b66a0 2024-04-25T16:52:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c394e8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3c3406fe00 con 0x7f3c340b6ab0 2024-04-25T16:52:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c32ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c1c004810 0x7f3c340b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c32ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c1c004810 0x7f3c340b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49820/0 (socket says 172.21.15.94:49820) 2024-04-25T16:52:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c32ffd640 1 -- 172.21.15.94:0/1032887727 learned_addr learned my addr 172.21.15.94:0/1032887727 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c327fc640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c340b66a0 0x7f3c341e9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c337fe640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c340b6ab0 0x7f3c3412f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c32ffd640 1 -- 172.21.15.94:0/1032887727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c340b6ab0 msgr2=0x7f3c3412f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c32ffd640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c340b6ab0 0x7f3c3412f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c32ffd640 1 -- 172.21.15.94:0/1032887727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c340b66a0 msgr2=0x7f3c341e9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c32ffd640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c340b66a0 0x7f3c341e9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.803+0000 7f3c32ffd640 1 -- 172.21.15.94:0/1032887727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c340b1470 con 0x7f3c1c004810 2024-04-25T16:52:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.807+0000 7f3c32ffd640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c1c004810 0x7f3c340b0f30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400e440 tx=0x7f3c2400e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.807+0000 7f3c1b7fe640 1 -- 172.21.15.94:0/1032887727 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24019070 con 0x7f3c1c004810 2024-04-25T16:52:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.807+0000 7f3c1b7fe640 1 -- 172.21.15.94:0/1032887727 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c240052e0 con 0x7f3c1c004810 2024-04-25T16:52:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.807+0000 7f3c394e8640 1 -- 172.21.15.94:0/1032887727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c340b5600 con 0x7f3c1c004810 2024-04-25T16:52:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.807+0000 7f3c1b7fe640 1 -- 172.21.15.94:0/1032887727 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24003840 con 0x7f3c1c004810 2024-04-25T16:52:44.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.807+0000 7f3c394e8640 1 -- 172.21.15.94:0/1032887727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c340b5bc0 con 0x7f3c1c004810 2024-04-25T16:52:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.807+0000 7f3c394e8640 1 -- 172.21.15.94:0/1032887727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c34076830 con 0x7f3c1c004810 2024-04-25T16:52:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.807+0000 7f3c1b7fe640 1 -- 172.21.15.94:0/1032887727 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3c2401c070 con 0x7f3c1c004810 2024-04-25T16:52:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.807+0000 7f3c1b7fe640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c2006f400 0x7f3c20071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.807+0000 7f3c327fc640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c2006f400 0x7f3c20071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.807+0000 7f3c1b7fe640 1 -- 172.21.15.94:0/1032887727 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f3c2408e980 con 0x7f3c1c004810 2024-04-25T16:52:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.807+0000 7f3c327fc640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c2006f400 0x7f3c20071910 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3c280af000 tx=0x7f3c280aa000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.811+0000 7f3c1b7fe640 1 -- 172.21.15.94:0/1032887727 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3c2405b070 con 0x7f3c1c004810 2024-04-25T16:52:44.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.971+0000 7f3c394e8640 1 -- 172.21.15.94:0/1032887727 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c341413e0 con 0x7f3c2006f400 2024-04-25T16:52:44.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.987+0000 7f3c1b7fe640 1 -- 172.21.15.94:0/1032887727 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251927 (secure 0 0 0) 0x7f3c280fc830 con 0x7f3c2006f400 2024-04-25T16:52:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:44.990 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T16:52:44.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.991+0000 7f3c394e8640 1 -- 172.21.15.94:0/1032887727 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c2006f400 msgr2=0x7f3c20071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:44.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.991+0000 7f3c394e8640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c2006f400 0x7f3c20071910 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3c280af000 tx=0x7f3c280aa000 comp rx=0 tx=0).stop 2024-04-25T16:52:44.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.991+0000 7f3c394e8640 1 -- 172.21.15.94:0/1032887727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c1c004810 msgr2=0x7f3c340b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:44.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.991+0000 7f3c394e8640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c1c004810 0x7f3c340b0f30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400e440 tx=0x7f3c2400e900 comp rx=0 tx=0).stop 2024-04-25T16:52:44.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.991+0000 7f3c394e8640 1 -- 172.21.15.94:0/1032887727 shutdown_connections 2024-04-25T16:52:44.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.991+0000 7f3c394e8640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c2006f400 0x7f3c20071910 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:44.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.991+0000 7f3c394e8640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c340b6ab0 0x7f3c3412f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:44.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.991+0000 7f3c394e8640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c340b66a0 0x7f3c341e9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:44.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.991+0000 7f3c394e8640 1 --2- 172.21.15.94:0/1032887727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c1c004810 0x7f3c340b0f30 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:44.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.991+0000 7f3c394e8640 1 -- 172.21.15.94:0/1032887727 >> 172.21.15.94:0/1032887727 conn(0x7f3c34070f40 msgr2=0x7f3c3415cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:44.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.991+0000 7f3c394e8640 1 -- 172.21.15.94:0/1032887727 shutdown_connections 2024-04-25T16:52:44.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:44.995+0000 7f3c394e8640 1 -- 172.21.15.94:0/1032887727 wait complete. 2024-04-25T16:52:45.011 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-04-25T16:52:43.698304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258452,"kb_used_data":4840,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754716268,"statfs":{"total":773094113280,"available":772829458432,"internally_reserved":0,"allocated":4956160,"data_stored":2807356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.828802"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881044+0000","last_change":"2024-04-25T16:52:26.825748+0000","last_active":"2024-04-25T16:52:35.881044+0000","last_peered":"2024-04-25T16:52:35.881044+0000","last_clean":"2024-04-25T16:52:35.881044+0000","last_became_active":"2024-04-25T16:52:26.825333+0000","last_became_peered":"2024-04-25T16:52:26.825333+0000","last_unstale":"2024-04-25T16:52:35.881044+0000","last_undegraded":"2024-04-25T16:52:35.881044+0000","last_fullsized":"2024-04-25T16:52:35.881044+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:19.084563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.880830+0000","last_change":"2024-04-25T16:52:26.795964+0000","last_active":"2024-04-25T16:52:35.880830+0000","last_peered":"2024-04-25T16:52:35.880830+0000","last_clean":"2024-04-25T16:52:35.880830+0000","last_became_active":"2024-04-25T16:52:26.794884+0000","last_became_peered":"2024-04-25T16:52:26.794884+0000","last_unstale":"2024-04-25T16:52:35.880830+0000","last_undegraded":"2024-04-25T16:52:35.880830+0000","last_fullsized":"2024-04-25T16:52:35.880830+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.050794+0000","last_change":"2024-04-25T16:52:26.802075+0000","last_active":"2024-04-25T16:52:36.050794+0000","last_peered":"2024-04-25T16:52:36.050794+0000","last_clean":"2024-04-25T16:52:36.050794+0000","last_became_active":"2024-04-25T16:52:26.801203+0000","last_became_peered":"2024-04-25T16:52:26.801203+0000","last_unstale":"2024-04-25T16:52:36.050794+0000","last_undegraded":"2024-04-25T16:52:36.050794+0000","last_fullsized":"2024-04-25T16:52:36.050794+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.325530+0000","last_change":"2024-04-25T16:52:26.793541+0000","last_active":"2024-04-25T16:52:36.325530+0000","last_peered":"2024-04-25T16:52:36.325530+0000","last_clean":"2024-04-25T16:52:36.325530+0000","last_became_active":"2024-04-25T16:52:26.789101+0000","last_became_peered":"2024-04-25T16:52:26.789101+0000","last_unstale":"2024-04-25T16:52:36.325530+0000","last_undegraded":"2024-04-25T16:52:36.325530+0000","last_fullsized":"2024-04-25T16:52:36.325530+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:38.013058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.325025+0000","last_change":"2024-04-25T16:52:26.784467+0000","last_active":"2024-04-25T16:52:36.325025+0000","last_peered":"2024-04-25T16:52:36.325025+0000","last_clean":"2024-04-25T16:52:36.325025+0000","last_became_active":"2024-04-25T16:52:26.783673+0000","last_became_peered":"2024-04-25T16:52:26.783673+0000","last_unstale":"2024-04-25T16:52:36.325025+0000","last_undegraded":"2024-04-25T16:52:36.325025+0000","last_fullsized":"2024-04-25T16:52:36.325025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:36.583897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324128+0000","last_change":"2024-04-25T16:52:26.823413+0000","last_active":"2024-04-25T16:52:36.324128+0000","last_peered":"2024-04-25T16:52:36.324128+0000","last_clean":"2024-04-25T16:52:36.324128+0000","last_became_active":"2024-04-25T16:52:26.823059+0000","last_became_peered":"2024-04-25T16:52:26.823059+0000","last_unstale":"2024-04-25T16:52:36.324128+0000","last_undegraded":"2024-04-25T16:52:36.324128+0000","last_fullsized":"2024-04-25T16:52:36.324128+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.031351+0000","last_change":"2024-04-25T16:52:26.808661+0000","last_active":"2024-04-25T16:52:36.031351+0000","last_peered":"2024-04-25T16:52:36.031351+0000","last_clean":"2024-04-25T16:52:36.031351+0000","last_became_active":"2024-04-25T16:52:26.805296+0000","last_became_peered":"2024-04-25T16:52:26.805296+0000","last_unstale":"2024-04-25T16:52:36.031351+0000","last_undegraded":"2024-04-25T16:52:36.031351+0000","last_fullsized":"2024-04-25T16:52:36.031351+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.053284+0000","last_change":"2024-04-25T16:52:36.053284+0000","last_active":"2024-04-25T16:52:36.053284+0000","last_peered":"2024-04-25T16:52:36.053284+0000","last_clean":"2024-04-25T16:52:36.053284+0000","last_became_active":"2024-04-25T16:52:36.052514+0000","last_became_peered":"2024-04-25T16:52:36.052514+0000","last_unstale":"2024-04-25T16:52:36.053284+0000","last_undegraded":"2024-04-25T16:52:36.053284+0000","last_fullsized":"2024-04-25T16:52:36.053284+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:33.801020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.051222+0000","last_change":"2024-04-25T16:52:26.803363+0000","last_active":"2024-04-25T16:52:36.051222+0000","last_peered":"2024-04-25T16:52:36.051222+0000","last_clean":"2024-04-25T16:52:36.051222+0000","last_became_active":"2024-04-25T16:52:26.802539+0000","last_became_peered":"2024-04-25T16:52:26.802539+0000","last_unstale":"2024-04-25T16:52:36.051222+0000","last_undegraded":"2024-04-25T16:52:36.051222+0000","last_fullsized":"2024-04-25T16:52:36.051222+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.032387+0000","last_change":"2024-04-25T16:52:26.842728+0000","last_active":"2024-04-25T16:52:36.032387+0000","last_peered":"2024-04-25T16:52:36.032387+0000","last_clean":"2024-04-25T16:52:36.032387+0000","last_became_active":"2024-04-25T16:52:26.842041+0000","last_became_peered":"2024-04-25T16:52:26.842041+0000","last_unstale":"2024-04-25T16:52:36.032387+0000","last_undegraded":"2024-04-25T16:52:36.032387+0000","last_fullsized":"2024-04-25T16:52:36.032387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:51:33.962706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.032157+0000","last_change":"2024-04-25T16:52:26.819329+0000","last_active":"2024-04-25T16:52:36.032157+0000","last_peered":"2024-04-25T16:52:36.032157+0000","last_clean":"2024-04-25T16:52:36.032157+0000","last_became_active":"2024-04-25T16:52:26.818528+0000","last_became_peered":"2024-04-25T16:52:26.818528+0000","last_unstale":"2024-04-25T16:52:36.032157+0000","last_undegraded":"2024-04-25T16:52:36.032157+0000","last_fullsized":"2024-04-25T16:52:36.032157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.050673+0000","last_change":"2024-04-25T16:52:26.795942+0000","last_active":"2024-04-25T16:52:36.050673+0000","last_peered":"2024-04-25T16:52:36.050673+0000","last_clean":"2024-04-25T16:52:36.050673+0000","last_became_active":"2024-04-25T16:52:26.795124+0000","last_became_peered":"2024-04-25T16:52:26.795124+0000","last_unstale":"2024-04-25T16:52:36.050673+0000","last_undegraded":"2024-04-25T16:52:36.050673+0000","last_fullsized":"2024-04-25T16:52:36.050673+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.031738+0000","last_change":"2024-04-25T16:52:26.814749+0000","last_active":"2024-04-25T16:52:36.031738+0000","last_peered":"2024-04-25T16:52:36.031738+0000","last_clean":"2024-04-25T16:52:36.031738+0000","last_became_active":"2024-04-25T16:52:26.814020+0000","last_became_peered":"2024-04-25T16:52:26.814020+0000","last_unstale":"2024-04-25T16:52:36.031738+0000","last_undegraded":"2024-04-25T16:52:36.031738+0000","last_fullsized":"2024-04-25T16:52:36.031738+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.880649+0000","last_change":"2024-04-25T16:52:26.813118+0000","last_active":"2024-04-25T16:52:35.880649+0000","last_peered":"2024-04-25T16:52:35.880649+0000","last_clean":"2024-04-25T16:52:35.880649+0000","last_became_active":"2024-04-25T16:52:26.812400+0000","last_became_peered":"2024-04-25T16:52:26.812400+0000","last_unstale":"2024-04-25T16:52:35.880649+0000","last_undegraded":"2024-04-25T16:52:35.880649+0000","last_fullsized":"2024-04-25T16:52:35.880649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:16.169654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.031554+0000","last_change":"2024-04-25T16:52:26.795864+0000","last_active":"2024-04-25T16:52:36.031554+0000","last_peered":"2024-04-25T16:52:36.031554+0000","last_clean":"2024-04-25T16:52:36.031554+0000","last_became_active":"2024-04-25T16:52:26.794738+0000","last_became_peered":"2024-04-25T16:52:26.794738+0000","last_unstale":"2024-04-25T16:52:36.031554+0000","last_undegraded":"2024-04-25T16:52:36.031554+0000","last_fullsized":"2024-04-25T16:52:36.031554+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324216+0000","last_change":"2024-04-25T16:52:26.836640+0000","last_active":"2024-04-25T16:52:36.324216+0000","last_peered":"2024-04-25T16:52:36.324216+0000","last_clean":"2024-04-25T16:52:36.324216+0000","last_became_active":"2024-04-25T16:52:26.833071+0000","last_became_peered":"2024-04-25T16:52:26.833071+0000","last_unstale":"2024-04-25T16:52:36.324216+0000","last_undegraded":"2024-04-25T16:52:36.324216+0000","last_fullsized":"2024-04-25T16:52:36.324216+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.325460+0000","last_change":"2024-04-25T16:52:26.841207+0000","last_active":"2024-04-25T16:52:36.325460+0000","last_peered":"2024-04-25T16:52:36.325460+0000","last_clean":"2024-04-25T16:52:36.325460+0000","last_became_active":"2024-04-25T16:52:26.838750+0000","last_became_peered":"2024-04-25T16:52:26.838750+0000","last_unstale":"2024-04-25T16:52:36.325460+0000","last_undegraded":"2024-04-25T16:52:36.325460+0000","last_fullsized":"2024-04-25T16:52:36.325460+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.882090+0000","last_change":"2024-04-25T16:52:26.795939+0000","last_active":"2024-04-25T16:52:35.882090+0000","last_peered":"2024-04-25T16:52:35.882090+0000","last_clean":"2024-04-25T16:52:35.882090+0000","last_became_active":"2024-04-25T16:52:26.795084+0000","last_became_peered":"2024-04-25T16:52:26.795084+0000","last_unstale":"2024-04-25T16:52:35.882090+0000","last_undegraded":"2024-04-25T16:52:35.882090+0000","last_fullsized":"2024-04-25T16:52:35.882090+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.325533+0000","last_change":"2024-04-25T16:52:26.818025+0000","last_active":"2024-04-25T16:52:36.325533+0000","last_peered":"2024-04-25T16:52:36.325533+0000","last_clean":"2024-04-25T16:52:36.325533+0000","last_became_active":"2024-04-25T16:52:26.817652+0000","last_became_peered":"2024-04-25T16:52:26.817652+0000","last_unstale":"2024-04-25T16:52:36.325533+0000","last_undegraded":"2024-04-25T16:52:36.325533+0000","last_fullsized":"2024-04-25T16:52:36.325533+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:46.412359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881647+0000","last_change":"2024-04-25T16:52:26.796384+0000","last_active":"2024-04-25T16:52:35.881647+0000","last_peered":"2024-04-25T16:52:35.881647+0000","last_clean":"2024-04-25T16:52:35.881647+0000","last_became_active":"2024-04-25T16:52:26.795507+0000","last_became_peered":"2024-04-25T16:52:26.795507+0000","last_unstale":"2024-04-25T16:52:35.881647+0000","last_undegraded":"2024-04-25T16:52:35.881647+0000","last_fullsized":"2024-04-25T16:52:35.881647+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.031772+0000","last_change":"2024-04-25T16:52:26.809988+0000","last_active":"2024-04-25T16:52:36.031772+0000","last_peered":"2024-04-25T16:52:36.031772+0000","last_clean":"2024-04-25T16:52:36.031772+0000","last_became_active":"2024-04-25T16:52:26.809107+0000","last_became_peered":"2024-04-25T16:52:26.809107+0000","last_unstale":"2024-04-25T16:52:36.031772+0000","last_undegraded":"2024-04-25T16:52:36.031772+0000","last_fullsized":"2024-04-25T16:52:36.031772+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881714+0000","last_change":"2024-04-25T16:52:26.784466+0000","last_active":"2024-04-25T16:52:35.881714+0000","last_peered":"2024-04-25T16:52:35.881714+0000","last_clean":"2024-04-25T16:52:35.881714+0000","last_became_active":"2024-04-25T16:52:26.783550+0000","last_became_peered":"2024-04-25T16:52:26.783550+0000","last_unstale":"2024-04-25T16:52:35.881714+0000","last_undegraded":"2024-04-25T16:52:35.881714+0000","last_fullsized":"2024-04-25T16:52:35.881714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:16.356202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324404+0000","last_change":"2024-04-25T16:52:26.832452+0000","last_active":"2024-04-25T16:52:36.324404+0000","last_peered":"2024-04-25T16:52:36.324404+0000","last_clean":"2024-04-25T16:52:36.324404+0000","last_became_active":"2024-04-25T16:52:26.831753+0000","last_became_peered":"2024-04-25T16:52:26.831753+0000","last_unstale":"2024-04-25T16:52:36.324404+0000","last_undegraded":"2024-04-25T16:52:36.324404+0000","last_fullsized":"2024-04-25T16:52:36.324404+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:35.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324531+0000","last_change":"2024-04-25T16:52:26.805633+0000","last_active":"2024-04-25T16:52:36.324531+0000","last_peered":"2024-04-25T16:52:36.324531+0000","last_clean":"2024-04-25T16:52:36.324531+0000","last_became_active":"2024-04-25T16:52:26.804789+0000","last_became_peered":"2024-04-25T16:52:26.804789+0000","last_unstale":"2024-04-25T16:52:36.324531+0000","last_undegraded":"2024-04-25T16:52:36.324531+0000","last_fullsized":"2024-04-25T16:52:36.324531+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:08.931465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881791+0000","last_change":"2024-04-25T16:52:26.798693+0000","last_active":"2024-04-25T16:52:35.881791+0000","last_peered":"2024-04-25T16:52:35.881791+0000","last_clean":"2024-04-25T16:52:35.881791+0000","last_became_active":"2024-04-25T16:52:26.797852+0000","last_became_peered":"2024-04-25T16:52:26.797852+0000","last_unstale":"2024-04-25T16:52:35.881791+0000","last_undegraded":"2024-04-25T16:52:35.881791+0000","last_fullsized":"2024-04-25T16:52:35.881791+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324911+0000","last_change":"2024-04-25T16:52:26.819630+0000","last_active":"2024-04-25T16:52:36.324911+0000","last_peered":"2024-04-25T16:52:36.324911+0000","last_clean":"2024-04-25T16:52:36.324911+0000","last_became_active":"2024-04-25T16:52:26.819274+0000","last_became_peered":"2024-04-25T16:52:26.819274+0000","last_unstale":"2024-04-25T16:52:36.324911+0000","last_undegraded":"2024-04-25T16:52:36.324911+0000","last_fullsized":"2024-04-25T16:52:36.324911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.883062+0000","last_change":"2024-04-25T16:52:26.791433+0000","last_active":"2024-04-25T16:52:35.883062+0000","last_peered":"2024-04-25T16:52:35.883062+0000","last_clean":"2024-04-25T16:52:35.883062+0000","last_became_active":"2024-04-25T16:52:26.790483+0000","last_became_peered":"2024-04-25T16:52:26.790483+0000","last_unstale":"2024-04-25T16:52:35.883062+0000","last_undegraded":"2024-04-25T16:52:35.883062+0000","last_fullsized":"2024-04-25T16:52:35.883062+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.032089+0000","last_change":"2024-04-25T16:52:26.804860+0000","last_active":"2024-04-25T16:52:36.032089+0000","last_peered":"2024-04-25T16:52:36.032089+0000","last_clean":"2024-04-25T16:52:36.032089+0000","last_became_active":"2024-04-25T16:52:26.804065+0000","last_became_peered":"2024-04-25T16:52:26.804065+0000","last_unstale":"2024-04-25T16:52:36.032089+0000","last_undegraded":"2024-04-25T16:52:36.032089+0000","last_fullsized":"2024-04-25T16:52:36.032089+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:20.302373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.882333+0000","last_change":"2024-04-25T16:52:26.806705+0000","last_active":"2024-04-25T16:52:35.882333+0000","last_peered":"2024-04-25T16:52:35.882333+0000","last_clean":"2024-04-25T16:52:35.882333+0000","last_became_active":"2024-04-25T16:52:26.805889+0000","last_became_peered":"2024-04-25T16:52:26.805889+0000","last_unstale":"2024-04-25T16:52:35.882333+0000","last_undegraded":"2024-04-25T16:52:35.882333+0000","last_fullsized":"2024-04-25T16:52:35.882333+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881416+0000","last_change":"2024-04-25T16:52:26.798903+0000","last_active":"2024-04-25T16:52:35.881416+0000","last_peered":"2024-04-25T16:52:35.881416+0000","last_clean":"2024-04-25T16:52:35.881416+0000","last_became_active":"2024-04-25T16:52:26.798011+0000","last_became_peered":"2024-04-25T16:52:26.798011+0000","last_unstale":"2024-04-25T16:52:35.881416+0000","last_undegraded":"2024-04-25T16:52:35.881416+0000","last_fullsized":"2024-04-25T16:52:35.881416+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324542+0000","last_change":"2024-04-25T16:52:26.841037+0000","last_active":"2024-04-25T16:52:36.324542+0000","last_peered":"2024-04-25T16:52:36.324542+0000","last_clean":"2024-04-25T16:52:36.324542+0000","last_became_active":"2024-04-25T16:52:26.838194+0000","last_became_peered":"2024-04-25T16:52:26.838194+0000","last_unstale":"2024-04-25T16:52:36.324542+0000","last_undegraded":"2024-04-25T16:52:36.324542+0000","last_fullsized":"2024-04-25T16:52:36.324542+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:11.144551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.323964+0000","last_change":"2024-04-25T16:52:26.813243+0000","last_active":"2024-04-25T16:52:36.323964+0000","last_peered":"2024-04-25T16:52:36.323964+0000","last_clean":"2024-04-25T16:52:36.323964+0000","last_became_active":"2024-04-25T16:52:26.811807+0000","last_became_peered":"2024-04-25T16:52:26.811807+0000","last_unstale":"2024-04-25T16:52:36.323964+0000","last_undegraded":"2024-04-25T16:52:36.323964+0000","last_fullsized":"2024-04-25T16:52:36.323964+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:42.553053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.325131+0000","last_change":"2024-04-25T16:52:26.841017+0000","last_active":"2024-04-25T16:52:36.325131+0000","last_peered":"2024-04-25T16:52:36.325131+0000","last_clean":"2024-04-25T16:52:36.325131+0000","last_became_active":"2024-04-25T16:52:26.837543+0000","last_became_peered":"2024-04-25T16:52:26.837543+0000","last_unstale":"2024-04-25T16:52:36.325131+0000","last_undegraded":"2024-04-25T16:52:36.325131+0000","last_fullsized":"2024-04-25T16:52:36.325131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:52.479044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.882183+0000","last_change":"2024-04-25T16:52:25.818069+0000","last_active":"2024-04-25T16:52:35.882183+0000","last_peered":"2024-04-25T16:52:35.882183+0000","last_clean":"2024-04-25T16:52:35.882183+0000","last_became_active":"2024-04-25T16:52:25.817362+0000","last_became_peered":"2024-04-25T16:52:25.817362+0000","last_unstale":"2024-04-25T16:52:35.882183+0000","last_undegraded":"2024-04-25T16:52:35.882183+0000","last_fullsized":"2024-04-25T16:52:35.882183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:08.488136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324714+0000","last_change":"2024-04-25T16:52:26.829008+0000","last_active":"2024-04-25T16:52:36.324714+0000","last_peered":"2024-04-25T16:52:36.324714+0000","last_clean":"2024-04-25T16:52:36.324714+0000","last_became_active":"2024-04-25T16:52:26.828706+0000","last_became_peered":"2024-04-25T16:52:26.828706+0000","last_unstale":"2024-04-25T16:52:36.324714+0000","last_undegraded":"2024-04-25T16:52:36.324714+0000","last_fullsized":"2024-04-25T16:52:36.324714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.050766+0000","last_change":"2024-04-25T16:52:25.795967+0000","last_active":"2024-04-25T16:52:36.050766+0000","last_peered":"2024-04-25T16:52:36.050766+0000","last_clean":"2024-04-25T16:52:36.050766+0000","last_became_active":"2024-04-25T16:52:25.795695+0000","last_became_peered":"2024-04-25T16:52:25.795695+0000","last_unstale":"2024-04-25T16:52:36.050766+0000","last_undegraded":"2024-04-25T16:52:36.050766+0000","last_fullsized":"2024-04-25T16:52:36.050766+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881478+0000","last_change":"2024-04-25T16:52:26.789430+0000","last_active":"2024-04-25T16:52:35.881478+0000","last_peered":"2024-04-25T16:52:35.881478+0000","last_clean":"2024-04-25T16:52:35.881478+0000","last_became_active":"2024-04-25T16:52:26.782243+0000","last_became_peered":"2024-04-25T16:52:26.782243+0000","last_unstale":"2024-04-25T16:52:35.881478+0000","last_undegraded":"2024-04-25T16:52:35.881478+0000","last_fullsized":"2024-04-25T16:52:35.881478+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:30.006966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324896+0000","last_change":"2024-04-25T16:52:25.819162+0000","last_active":"2024-04-25T16:52:36.324896+0000","last_peered":"2024-04-25T16:52:36.324896+0000","last_clean":"2024-04-25T16:52:36.324896+0000","last_became_active":"2024-04-25T16:52:25.818859+0000","last_became_peered":"2024-04-25T16:52:25.818859+0000","last_unstale":"2024-04-25T16:52:36.324896+0000","last_undegraded":"2024-04-25T16:52:36.324896+0000","last_fullsized":"2024-04-25T16:52:36.324896+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.880943+0000","last_change":"2024-04-25T16:52:26.805376+0000","last_active":"2024-04-25T16:52:35.880943+0000","last_peered":"2024-04-25T16:52:35.880943+0000","last_clean":"2024-04-25T16:52:35.880943+0000","last_became_active":"2024-04-25T16:52:26.804542+0000","last_became_peered":"2024-04-25T16:52:26.804542+0000","last_unstale":"2024-04-25T16:52:35.880943+0000","last_undegraded":"2024-04-25T16:52:35.880943+0000","last_fullsized":"2024-04-25T16:52:35.880943+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.620224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.883054+0000","last_change":"2024-04-25T16:52:25.815751+0000","last_active":"2024-04-25T16:52:35.883054+0000","last_peered":"2024-04-25T16:52:35.883054+0000","last_clean":"2024-04-25T16:52:35.883054+0000","last_became_active":"2024-04-25T16:52:25.814927+0000","last_became_peered":"2024-04-25T16:52:25.814927+0000","last_unstale":"2024-04-25T16:52:35.883054+0000","last_undegraded":"2024-04-25T16:52:35.883054+0000","last_fullsized":"2024-04-25T16:52:35.883054+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.051072+0000","last_change":"2024-04-25T16:52:26.814640+0000","last_active":"2024-04-25T16:52:36.051072+0000","last_peered":"2024-04-25T16:52:36.051072+0000","last_clean":"2024-04-25T16:52:36.051072+0000","last_became_active":"2024-04-25T16:52:26.814016+0000","last_became_peered":"2024-04-25T16:52:26.814016+0000","last_unstale":"2024-04-25T16:52:36.051072+0000","last_undegraded":"2024-04-25T16:52:36.051072+0000","last_fullsized":"2024-04-25T16:52:36.051072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.705474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.032098+0000","last_change":"2024-04-25T16:52:25.805323+0000","last_active":"2024-04-25T16:52:36.032098+0000","last_peered":"2024-04-25T16:52:36.032098+0000","last_clean":"2024-04-25T16:52:36.032098+0000","last_became_active":"2024-04-25T16:52:25.796444+0000","last_became_peered":"2024-04-25T16:52:25.796444+0000","last_unstale":"2024-04-25T16:52:36.032098+0000","last_undegraded":"2024-04-25T16:52:36.032098+0000","last_fullsized":"2024-04-25T16:52:36.032098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:28.996351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.031673+0000","last_change":"2024-04-25T16:52:26.821648+0000","last_active":"2024-04-25T16:52:36.031673+0000","last_peered":"2024-04-25T16:52:36.031673+0000","last_clean":"2024-04-25T16:52:36.031673+0000","last_became_active":"2024-04-25T16:52:26.820835+0000","last_became_peered":"2024-04-25T16:52:26.820835+0000","last_unstale":"2024-04-25T16:52:36.031673+0000","last_undegraded":"2024-04-25T16:52:36.031673+0000","last_fullsized":"2024-04-25T16:52:36.031673+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324556+0000","last_change":"2024-04-25T16:52:25.816250+0000","last_active":"2024-04-25T16:52:36.324556+0000","last_peered":"2024-04-25T16:52:36.324556+0000","last_clean":"2024-04-25T16:52:36.324556+0000","last_became_active":"2024-04-25T16:52:25.815656+0000","last_became_peered":"2024-04-25T16:52:25.815656+0000","last_unstale":"2024-04-25T16:52:36.324556+0000","last_undegraded":"2024-04-25T16:52:36.324556+0000","last_fullsized":"2024-04-25T16:52:36.324556+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881226+0000","last_change":"2024-04-25T16:52:26.813425+0000","last_active":"2024-04-25T16:52:35.881226+0000","last_peered":"2024-04-25T16:52:35.881226+0000","last_clean":"2024-04-25T16:52:35.881226+0000","last_became_active":"2024-04-25T16:52:26.812713+0000","last_became_peered":"2024-04-25T16:52:26.812713+0000","last_unstale":"2024-04-25T16:52:35.881226+0000","last_undegraded":"2024-04-25T16:52:35.881226+0000","last_fullsized":"2024-04-25T16:52:35.881226+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:27.827725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324115+0000","last_change":"2024-04-25T16:52:25.803265+0000","last_active":"2024-04-25T16:52:36.324115+0000","last_peered":"2024-04-25T16:52:36.324115+0000","last_clean":"2024-04-25T16:52:36.324115+0000","last_became_active":"2024-04-25T16:52:25.802617+0000","last_became_peered":"2024-04-25T16:52:25.802617+0000","last_unstale":"2024-04-25T16:52:36.324115+0000","last_undegraded":"2024-04-25T16:52:36.324115+0000","last_fullsized":"2024-04-25T16:52:36.324115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:45.401744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.880949+0000","last_change":"2024-04-25T16:52:26.819267+0000","last_active":"2024-04-25T16:52:35.880949+0000","last_peered":"2024-04-25T16:52:35.880949+0000","last_clean":"2024-04-25T16:52:35.880949+0000","last_became_active":"2024-04-25T16:52:26.818740+0000","last_became_peered":"2024-04-25T16:52:26.818740+0000","last_unstale":"2024-04-25T16:52:35.880949+0000","last_undegraded":"2024-04-25T16:52:35.880949+0000","last_fullsized":"2024-04-25T16:52:35.880949+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.880518+0000","last_change":"2024-04-25T16:52:25.816663+0000","last_active":"2024-04-25T16:52:35.880518+0000","last_peered":"2024-04-25T16:52:35.880518+0000","last_clean":"2024-04-25T16:52:35.880518+0000","last_became_active":"2024-04-25T16:52:25.815420+0000","last_became_peered":"2024-04-25T16:52:25.815420+0000","last_unstale":"2024-04-25T16:52:35.880518+0000","last_undegraded":"2024-04-25T16:52:35.880518+0000","last_fullsized":"2024-04-25T16:52:35.880518+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:30.288538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.325718+0000","last_change":"2024-04-25T16:52:26.789927+0000","last_active":"2024-04-25T16:52:36.325718+0000","last_peered":"2024-04-25T16:52:36.325718+0000","last_clean":"2024-04-25T16:52:36.325718+0000","last_became_active":"2024-04-25T16:52:26.788014+0000","last_became_peered":"2024-04-25T16:52:26.788014+0000","last_unstale":"2024-04-25T16:52:36.325718+0000","last_undegraded":"2024-04-25T16:52:36.325718+0000","last_fullsized":"2024-04-25T16:52:36.325718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:47.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881158+0000","last_change":"2024-04-25T16:52:25.798974+0000","last_active":"2024-04-25T16:52:35.881158+0000","last_peered":"2024-04-25T16:52:35.881158+0000","last_clean":"2024-04-25T16:52:35.881158+0000","last_became_active":"2024-04-25T16:52:25.798314+0000","last_became_peered":"2024-04-25T16:52:25.798314+0000","last_unstale":"2024-04-25T16:52:35.881158+0000","last_undegraded":"2024-04-25T16:52:35.881158+0000","last_fullsized":"2024-04-25T16:52:35.881158+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.348530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324883+0000","last_change":"2024-04-25T16:52:26.842485+0000","last_active":"2024-04-25T16:52:36.324883+0000","last_peered":"2024-04-25T16:52:36.324883+0000","last_clean":"2024-04-25T16:52:36.324883+0000","last_became_active":"2024-04-25T16:52:26.838813+0000","last_became_peered":"2024-04-25T16:52:26.838813+0000","last_unstale":"2024-04-25T16:52:36.324883+0000","last_undegraded":"2024-04-25T16:52:36.324883+0000","last_fullsized":"2024-04-25T16:52:36.324883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:20.684352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.032823+0000","last_change":"2024-04-25T16:52:25.803289+0000","last_active":"2024-04-25T16:52:36.032823+0000","last_peered":"2024-04-25T16:52:36.032823+0000","last_clean":"2024-04-25T16:52:36.032823+0000","last_became_active":"2024-04-25T16:52:25.801415+0000","last_became_peered":"2024-04-25T16:52:25.801415+0000","last_unstale":"2024-04-25T16:52:36.032823+0000","last_undegraded":"2024-04-25T16:52:36.032823+0000","last_fullsized":"2024-04-25T16:52:36.032823+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:45.310089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.050614+0000","last_change":"2024-04-25T16:52:26.840379+0000","last_active":"2024-04-25T16:52:36.050614+0000","last_peered":"2024-04-25T16:52:36.050614+0000","last_clean":"2024-04-25T16:52:36.050614+0000","last_became_active":"2024-04-25T16:52:26.840062+0000","last_became_peered":"2024-04-25T16:52:26.840062+0000","last_unstale":"2024-04-25T16:52:36.050614+0000","last_undegraded":"2024-04-25T16:52:36.050614+0000","last_fullsized":"2024-04-25T16:52:36.050614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.032405+0000","last_change":"2024-04-25T16:52:25.809391+0000","last_active":"2024-04-25T16:52:36.032405+0000","last_peered":"2024-04-25T16:52:36.032405+0000","last_clean":"2024-04-25T16:52:36.032405+0000","last_became_active":"2024-04-25T16:52:25.808455+0000","last_became_peered":"2024-04-25T16:52:25.808455+0000","last_unstale":"2024-04-25T16:52:36.032405+0000","last_undegraded":"2024-04-25T16:52:36.032405+0000","last_fullsized":"2024-04-25T16:52:36.032405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.882243+0000","last_change":"2024-04-25T16:52:26.782954+0000","last_active":"2024-04-25T16:52:35.882243+0000","last_peered":"2024-04-25T16:52:35.882243+0000","last_clean":"2024-04-25T16:52:35.882243+0000","last_became_active":"2024-04-25T16:52:26.782108+0000","last_became_peered":"2024-04-25T16:52:26.782108+0000","last_unstale":"2024-04-25T16:52:35.882243+0000","last_undegraded":"2024-04-25T16:52:35.882243+0000","last_fullsized":"2024-04-25T16:52:35.882243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:17.378653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":24,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.882133+0000","last_change":"2024-04-25T16:52:25.795020+0000","last_active":"2024-04-25T16:52:35.882133+0000","last_peered":"2024-04-25T16:52:35.882133+0000","last_clean":"2024-04-25T16:52:35.882133+0000","last_became_active":"2024-04-25T16:52:25.794633+0000","last_became_peered":"2024-04-25T16:52:25.794633+0000","last_unstale":"2024-04-25T16:52:35.882133+0000","last_undegraded":"2024-04-25T16:52:35.882133+0000","last_fullsized":"2024-04-25T16:52:35.882133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:38.694859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.050964+0000","last_change":"2024-04-25T16:52:26.821081+0000","last_active":"2024-04-25T16:52:36.050964+0000","last_peered":"2024-04-25T16:52:36.050964+0000","last_clean":"2024-04-25T16:52:36.050964+0000","last_became_active":"2024-04-25T16:52:26.819337+0000","last_became_peered":"2024-04-25T16:52:26.819337+0000","last_unstale":"2024-04-25T16:52:36.050964+0000","last_undegraded":"2024-04-25T16:52:36.050964+0000","last_fullsized":"2024-04-25T16:52:36.050964+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:31.965314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881855+0000","last_change":"2024-04-25T16:52:25.793039+0000","last_active":"2024-04-25T16:52:35.881855+0000","last_peered":"2024-04-25T16:52:35.881855+0000","last_clean":"2024-04-25T16:52:35.881855+0000","last_became_active":"2024-04-25T16:52:25.792552+0000","last_became_peered":"2024-04-25T16:52:25.792552+0000","last_unstale":"2024-04-25T16:52:35.881855+0000","last_undegraded":"2024-04-25T16:52:35.881855+0000","last_fullsized":"2024-04-25T16:52:35.881855+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:30.954700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.880544+0000","last_change":"2024-04-25T16:52:26.823643+0000","last_active":"2024-04-25T16:52:35.880544+0000","last_peered":"2024-04-25T16:52:35.880544+0000","last_clean":"2024-04-25T16:52:35.880544+0000","last_became_active":"2024-04-25T16:52:26.823348+0000","last_became_peered":"2024-04-25T16:52:26.823348+0000","last_unstale":"2024-04-25T16:52:35.880544+0000","last_undegraded":"2024-04-25T16:52:35.880544+0000","last_fullsized":"2024-04-25T16:52:35.880544+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324259+0000","last_change":"2024-04-25T16:52:25.813706+0000","last_active":"2024-04-25T16:52:36.324259+0000","last_peered":"2024-04-25T16:52:36.324259+0000","last_clean":"2024-04-25T16:52:36.324259+0000","last_became_active":"2024-04-25T16:52:25.813387+0000","last_became_peered":"2024-04-25T16:52:25.813387+0000","last_unstale":"2024-04-25T16:52:36.324259+0000","last_undegraded":"2024-04-25T16:52:36.324259+0000","last_fullsized":"2024-04-25T16:52:36.324259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324535+0000","last_change":"2024-04-25T16:52:26.817332+0000","last_active":"2024-04-25T16:52:36.324535+0000","last_peered":"2024-04-25T16:52:36.324535+0000","last_clean":"2024-04-25T16:52:36.324535+0000","last_became_active":"2024-04-25T16:52:26.816020+0000","last_became_peered":"2024-04-25T16:52:26.816020+0000","last_unstale":"2024-04-25T16:52:36.324535+0000","last_undegraded":"2024-04-25T16:52:36.324535+0000","last_fullsized":"2024-04-25T16:52:36.324535+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881263+0000","last_change":"2024-04-25T16:52:25.815545+0000","last_active":"2024-04-25T16:52:35.881263+0000","last_peered":"2024-04-25T16:52:35.881263+0000","last_clean":"2024-04-25T16:52:35.881263+0000","last_became_active":"2024-04-25T16:52:25.814711+0000","last_became_peered":"2024-04-25T16:52:25.814711+0000","last_unstale":"2024-04-25T16:52:35.881263+0000","last_undegraded":"2024-04-25T16:52:35.881263+0000","last_fullsized":"2024-04-25T16:52:35.881263+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881267+0000","last_change":"2024-04-25T16:52:26.826567+0000","last_active":"2024-04-25T16:52:35.881267+0000","last_peered":"2024-04-25T16:52:35.881267+0000","last_clean":"2024-04-25T16:52:35.881267+0000","last_became_active":"2024-04-25T16:52:26.826224+0000","last_became_peered":"2024-04-25T16:52:26.826224+0000","last_unstale":"2024-04-25T16:52:35.881267+0000","last_undegraded":"2024-04-25T16:52:35.881267+0000","last_fullsized":"2024-04-25T16:52:35.881267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:06.823841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":26,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324431+0000","last_change":"2024-04-25T16:52:25.815153+0000","last_active":"2024-04-25T16:52:36.324431+0000","last_peered":"2024-04-25T16:52:36.324431+0000","last_clean":"2024-04-25T16:52:36.324431+0000","last_became_active":"2024-04-25T16:52:25.814360+0000","last_became_peered":"2024-04-25T16:52:25.814360+0000","last_unstale":"2024-04-25T16:52:36.324431+0000","last_undegraded":"2024-04-25T16:52:36.324431+0000","last_fullsized":"2024-04-25T16:52:36.324431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881871+0000","last_change":"2024-04-25T16:52:26.804704+0000","last_active":"2024-04-25T16:52:35.881871+0000","last_peered":"2024-04-25T16:52:35.881871+0000","last_clean":"2024-04-25T16:52:35.881871+0000","last_became_active":"2024-04-25T16:52:26.803888+0000","last_became_peered":"2024-04-25T16:52:26.803888+0000","last_unstale":"2024-04-25T16:52:35.881871+0000","last_undegraded":"2024-04-25T16:52:35.881871+0000","last_fullsized":"2024-04-25T16:52:35.881871+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.640204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.032760+0000","last_change":"2024-04-25T16:52:25.818880+0000","last_active":"2024-04-25T16:52:36.032760+0000","last_peered":"2024-04-25T16:52:36.032760+0000","last_clean":"2024-04-25T16:52:36.032760+0000","last_became_active":"2024-04-25T16:52:25.817841+0000","last_became_peered":"2024-04-25T16:52:25.817841+0000","last_unstale":"2024-04-25T16:52:36.032760+0000","last_undegraded":"2024-04-25T16:52:36.032760+0000","last_fullsized":"2024-04-25T16:52:36.032760+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:26.817110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.032267+0000","last_change":"2024-04-25T16:52:26.809554+0000","last_active":"2024-04-25T16:52:36.032267+0000","last_peered":"2024-04-25T16:52:36.032267+0000","last_clean":"2024-04-25T16:52:36.032267+0000","last_became_active":"2024-04-25T16:52:26.808709+0000","last_became_peered":"2024-04-25T16:52:26.808709+0000","last_unstale":"2024-04-25T16:52:36.032267+0000","last_undegraded":"2024-04-25T16:52:36.032267+0000","last_fullsized":"2024-04-25T16:52:36.032267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:03.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881738+0000","last_change":"2024-04-25T16:52:25.812269+0000","last_active":"2024-04-25T16:52:35.881738+0000","last_peered":"2024-04-25T16:52:35.881738+0000","last_clean":"2024-04-25T16:52:35.881738+0000","last_became_active":"2024-04-25T16:52:25.810831+0000","last_became_peered":"2024-04-25T16:52:25.810831+0000","last_unstale":"2024-04-25T16:52:35.881738+0000","last_undegraded":"2024-04-25T16:52:35.881738+0000","last_fullsized":"2024-04-25T16:52:35.881738+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:17.420411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881211+0000","last_change":"2024-04-25T16:52:26.806939+0000","last_active":"2024-04-25T16:52:35.881211+0000","last_peered":"2024-04-25T16:52:35.881211+0000","last_clean":"2024-04-25T16:52:35.881211+0000","last_became_active":"2024-04-25T16:52:26.806053+0000","last_became_peered":"2024-04-25T16:52:26.806053+0000","last_unstale":"2024-04-25T16:52:35.881211+0000","last_undegraded":"2024-04-25T16:52:35.881211+0000","last_fullsized":"2024-04-25T16:52:35.881211+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:08.913131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.325577+0000","last_change":"2024-04-25T16:52:25.783759+0000","last_active":"2024-04-25T16:52:36.325577+0000","last_peered":"2024-04-25T16:52:36.325577+0000","last_clean":"2024-04-25T16:52:36.325577+0000","last_became_active":"2024-04-25T16:52:25.783353+0000","last_became_peered":"2024-04-25T16:52:25.783353+0000","last_unstale":"2024-04-25T16:52:36.325577+0000","last_undegraded":"2024-04-25T16:52:36.325577+0000","last_fullsized":"2024-04-25T16:52:36.325577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:34.281502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.325236+0000","last_change":"2024-04-25T16:52:26.798434+0000","last_active":"2024-04-25T16:52:36.325236+0000","last_peered":"2024-04-25T16:52:36.325236+0000","last_clean":"2024-04-25T16:52:36.325236+0000","last_became_active":"2024-04-25T16:52:26.797631+0000","last_became_peered":"2024-04-25T16:52:26.797631+0000","last_unstale":"2024-04-25T16:52:36.325236+0000","last_undegraded":"2024-04-25T16:52:36.325236+0000","last_fullsized":"2024-04-25T16:52:36.325236+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:16.248387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324389+0000","last_change":"2024-04-25T16:52:25.819136+0000","last_active":"2024-04-25T16:52:36.324389+0000","last_peered":"2024-04-25T16:52:36.324389+0000","last_clean":"2024-04-25T16:52:36.324389+0000","last_became_active":"2024-04-25T16:52:25.818809+0000","last_became_peered":"2024-04-25T16:52:25.818809+0000","last_unstale":"2024-04-25T16:52:36.324389+0000","last_undegraded":"2024-04-25T16:52:36.324389+0000","last_fullsized":"2024-04-25T16:52:36.324389+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324532+0000","last_change":"2024-04-25T16:52:26.819612+0000","last_active":"2024-04-25T16:52:36.324532+0000","last_peered":"2024-04-25T16:52:36.324532+0000","last_clean":"2024-04-25T16:52:36.324532+0000","last_became_active":"2024-04-25T16:52:26.818874+0000","last_became_peered":"2024-04-25T16:52:26.818874+0000","last_unstale":"2024-04-25T16:52:36.324532+0000","last_undegraded":"2024-04-25T16:52:36.324532+0000","last_fullsized":"2024-04-25T16:52:36.324532+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:41.283996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.325144+0000","last_change":"2024-04-25T16:52:25.780874+0000","last_active":"2024-04-25T16:52:36.325144+0000","last_peered":"2024-04-25T16:52:36.325144+0000","last_clean":"2024-04-25T16:52:36.325144+0000","last_became_active":"2024-04-25T16:52:25.780529+0000","last_became_peered":"2024-04-25T16:52:25.780529+0000","last_unstale":"2024-04-25T16:52:36.325144+0000","last_undegraded":"2024-04-25T16:52:36.325144+0000","last_fullsized":"2024-04-25T16:52:36.325144+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:57.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.050499+0000","last_change":"2024-04-25T16:52:26.845027+0000","last_active":"2024-04-25T16:52:36.050499+0000","last_peered":"2024-04-25T16:52:36.050499+0000","last_clean":"2024-04-25T16:52:36.050499+0000","last_became_active":"2024-04-25T16:52:26.844620+0000","last_became_peered":"2024-04-25T16:52:26.844620+0000","last_unstale":"2024-04-25T16:52:36.050499+0000","last_undegraded":"2024-04-25T16:52:36.050499+0000","last_fullsized":"2024-04-25T16:52:36.050499+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:25.355294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.325168+0000","last_change":"2024-04-25T16:52:25.821889+0000","last_active":"2024-04-25T16:52:36.325168+0000","last_peered":"2024-04-25T16:52:36.325168+0000","last_clean":"2024-04-25T16:52:36.325168+0000","last_became_active":"2024-04-25T16:52:25.821643+0000","last_became_peered":"2024-04-25T16:52:25.821643+0000","last_unstale":"2024-04-25T16:52:36.325168+0000","last_undegraded":"2024-04-25T16:52:36.325168+0000","last_fullsized":"2024-04-25T16:52:36.325168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:27.786669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.031760+0000","last_change":"2024-04-25T16:52:26.802864+0000","last_active":"2024-04-25T16:52:36.031760+0000","last_peered":"2024-04-25T16:52:36.031760+0000","last_clean":"2024-04-25T16:52:36.031760+0000","last_became_active":"2024-04-25T16:52:26.801913+0000","last_became_peered":"2024-04-25T16:52:26.801913+0000","last_unstale":"2024-04-25T16:52:36.031760+0000","last_undegraded":"2024-04-25T16:52:36.031760+0000","last_fullsized":"2024-04-25T16:52:36.031760+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:16.327769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324762+0000","last_change":"2024-04-25T16:52:25.806124+0000","last_active":"2024-04-25T16:52:36.324762+0000","last_peered":"2024-04-25T16:52:36.324762+0000","last_clean":"2024-04-25T16:52:36.324762+0000","last_became_active":"2024-04-25T16:52:25.805306+0000","last_became_peered":"2024-04-25T16:52:25.805306+0000","last_unstale":"2024-04-25T16:52:36.324762+0000","last_undegraded":"2024-04-25T16:52:36.324762+0000","last_fullsized":"2024-04-25T16:52:36.324762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:46.761681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881562+0000","last_change":"2024-04-25T16:52:26.819372+0000","last_active":"2024-04-25T16:52:35.881562+0000","last_peered":"2024-04-25T16:52:35.881562+0000","last_clean":"2024-04-25T16:52:35.881562+0000","last_became_active":"2024-04-25T16:52:26.807182+0000","last_became_peered":"2024-04-25T16:52:26.807182+0000","last_unstale":"2024-04-25T16:52:35.881562+0000","last_undegraded":"2024-04-25T16:52:35.881562+0000","last_fullsized":"2024-04-25T16:52:35.881562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:14.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324632+0000","last_change":"2024-04-25T16:52:25.788959+0000","last_active":"2024-04-25T16:52:36.324632+0000","last_peered":"2024-04-25T16:52:36.324632+0000","last_clean":"2024-04-25T16:52:36.324632+0000","last_became_active":"2024-04-25T16:52:25.788261+0000","last_became_peered":"2024-04-25T16:52:25.788261+0000","last_unstale":"2024-04-25T16:52:36.324632+0000","last_undegraded":"2024-04-25T16:52:36.324632+0000","last_fullsized":"2024-04-25T16:52:36.324632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:13.943131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324246+0000","last_change":"2024-04-25T16:52:26.819641+0000","last_active":"2024-04-25T16:52:36.324246+0000","last_peered":"2024-04-25T16:52:36.324246+0000","last_clean":"2024-04-25T16:52:36.324246+0000","last_became_active":"2024-04-25T16:52:26.819314+0000","last_became_peered":"2024-04-25T16:52:26.819314+0000","last_unstale":"2024-04-25T16:52:36.324246+0000","last_undegraded":"2024-04-25T16:52:36.324246+0000","last_fullsized":"2024-04-25T16:52:36.324246+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.032993+0000","last_change":"2024-04-25T16:52:25.806269+0000","last_active":"2024-04-25T16:52:36.032993+0000","last_peered":"2024-04-25T16:52:36.032993+0000","last_clean":"2024-04-25T16:52:36.032993+0000","last_became_active":"2024-04-25T16:52:25.804183+0000","last_became_peered":"2024-04-25T16:52:25.804183+0000","last_unstale":"2024-04-25T16:52:36.032993+0000","last_undegraded":"2024-04-25T16:52:36.032993+0000","last_fullsized":"2024-04-25T16:52:36.032993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:07.902516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.325111+0000","last_change":"2024-04-25T16:52:26.841753+0000","last_active":"2024-04-25T16:52:36.325111+0000","last_peered":"2024-04-25T16:52:36.325111+0000","last_clean":"2024-04-25T16:52:36.325111+0000","last_became_active":"2024-04-25T16:52:26.841112+0000","last_became_peered":"2024-04-25T16:52:26.841112+0000","last_unstale":"2024-04-25T16:52:36.325111+0000","last_undegraded":"2024-04-25T16:52:36.325111+0000","last_fullsized":"2024-04-25T16:52:36.325111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:58.025881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.325536+0000","last_change":"2024-04-25T16:52:25.803168+0000","last_active":"2024-04-25T16:52:36.325536+0000","last_peered":"2024-04-25T16:52:36.325536+0000","last_clean":"2024-04-25T16:52:36.325536+0000","last_became_active":"2024-04-25T16:52:25.802880+0000","last_became_peered":"2024-04-25T16:52:25.802880+0000","last_unstale":"2024-04-25T16:52:36.325536+0000","last_undegraded":"2024-04-25T16:52:36.325536+0000","last_fullsized":"2024-04-25T16:52:36.325536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:42.210482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881165+0000","last_change":"2024-04-25T16:52:26.819781+0000","last_active":"2024-04-25T16:52:35.881165+0000","last_peered":"2024-04-25T16:52:35.881165+0000","last_clean":"2024-04-25T16:52:35.881165+0000","last_became_active":"2024-04-25T16:52:26.819467+0000","last_became_peered":"2024-04-25T16:52:26.819467+0000","last_unstale":"2024-04-25T16:52:35.881165+0000","last_undegraded":"2024-04-25T16:52:35.881165+0000","last_fullsized":"2024-04-25T16:52:35.881165+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.960201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.050916+0000","last_change":"2024-04-25T16:52:25.799953+0000","last_active":"2024-04-25T16:52:36.050916+0000","last_peered":"2024-04-25T16:52:36.050916+0000","last_clean":"2024-04-25T16:52:36.050916+0000","last_became_active":"2024-04-25T16:52:25.799689+0000","last_became_peered":"2024-04-25T16:52:25.799689+0000","last_unstale":"2024-04-25T16:52:36.050916+0000","last_undegraded":"2024-04-25T16:52:36.050916+0000","last_fullsized":"2024-04-25T16:52:36.050916+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.051100+0000","last_change":"2024-04-25T16:52:26.799367+0000","last_active":"2024-04-25T16:52:36.051100+0000","last_peered":"2024-04-25T16:52:36.051100+0000","last_clean":"2024-04-25T16:52:36.051100+0000","last_became_active":"2024-04-25T16:52:26.797337+0000","last_became_peered":"2024-04-25T16:52:26.797337+0000","last_unstale":"2024-04-25T16:52:36.051100+0000","last_undegraded":"2024-04-25T16:52:36.051100+0000","last_fullsized":"2024-04-25T16:52:36.051100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:21.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.050936+0000","last_change":"2024-04-25T16:52:25.806956+0000","last_active":"2024-04-25T16:52:36.050936+0000","last_peered":"2024-04-25T16:52:36.050936+0000","last_clean":"2024-04-25T16:52:36.050936+0000","last_became_active":"2024-04-25T16:52:25.806604+0000","last_became_peered":"2024-04-25T16:52:25.806604+0000","last_unstale":"2024-04-25T16:52:36.050936+0000","last_undegraded":"2024-04-25T16:52:36.050936+0000","last_fullsized":"2024-04-25T16:52:36.050936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.573282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324064+0000","last_change":"2024-04-25T16:52:26.820845+0000","last_active":"2024-04-25T16:52:36.324064+0000","last_peered":"2024-04-25T16:52:36.324064+0000","last_clean":"2024-04-25T16:52:36.324064+0000","last_became_active":"2024-04-25T16:52:26.820484+0000","last_became_peered":"2024-04-25T16:52:26.820484+0000","last_unstale":"2024-04-25T16:52:36.324064+0000","last_undegraded":"2024-04-25T16:52:36.324064+0000","last_fullsized":"2024-04-25T16:52:36.324064+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:44.359144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.881712+0000","last_change":"2024-04-25T16:52:25.785723+0000","last_active":"2024-04-25T16:52:35.881712+0000","last_peered":"2024-04-25T16:52:35.881712+0000","last_clean":"2024-04-25T16:52:35.881712+0000","last_became_active":"2024-04-25T16:52:25.784930+0000","last_became_peered":"2024-04-25T16:52:25.784930+0000","last_unstale":"2024-04-25T16:52:35.881712+0000","last_undegraded":"2024-04-25T16:52:35.881712+0000","last_fullsized":"2024-04-25T16:52:35.881712+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:47.018173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.880928+0000","last_change":"2024-04-25T16:52:26.799277+0000","last_active":"2024-04-25T16:52:35.880928+0000","last_peered":"2024-04-25T16:52:35.880928+0000","last_clean":"2024-04-25T16:52:35.880928+0000","last_became_active":"2024-04-25T16:52:26.797817+0000","last_became_peered":"2024-04-25T16:52:26.797817+0000","last_unstale":"2024-04-25T16:52:35.880928+0000","last_undegraded":"2024-04-25T16:52:35.880928+0000","last_fullsized":"2024-04-25T16:52:35.880928+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.320703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.880544+0000","last_change":"2024-04-25T16:52:25.810628+0000","last_active":"2024-04-25T16:52:35.880544+0000","last_peered":"2024-04-25T16:52:35.880544+0000","last_clean":"2024-04-25T16:52:35.880544+0000","last_became_active":"2024-04-25T16:52:25.809669+0000","last_became_peered":"2024-04-25T16:52:25.809669+0000","last_unstale":"2024-04-25T16:52:35.880544+0000","last_undegraded":"2024-04-25T16:52:35.880544+0000","last_fullsized":"2024-04-25T16:52:35.880544+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:36.638682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324933+0000","last_change":"2024-04-25T16:52:26.832124+0000","last_active":"2024-04-25T16:52:36.324933+0000","last_peered":"2024-04-25T16:52:36.324933+0000","last_clean":"2024-04-25T16:52:36.324933+0000","last_became_active":"2024-04-25T16:52:26.831437+0000","last_became_peered":"2024-04-25T16:52:26.831437+0000","last_unstale":"2024-04-25T16:52:36.324933+0000","last_undegraded":"2024-04-25T16:52:36.324933+0000","last_fullsized":"2024-04-25T16:52:36.324933+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:58.305711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:35.880696+0000","last_change":"2024-04-25T16:52:25.793944+0000","last_active":"2024-04-25T16:52:35.880696+0000","last_peered":"2024-04-25T16:52:35.880696+0000","last_clean":"2024-04-25T16:52:35.880696+0000","last_became_active":"2024-04-25T16:52:25.793522+0000","last_became_peered":"2024-04-25T16:52:25.793522+0000","last_unstale":"2024-04-25T16:52:35.880696+0000","last_undegraded":"2024-04-25T16:52:35.880696+0000","last_fullsized":"2024-04-25T16:52:35.880696+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:41.542439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.032548+0000","last_change":"2024-04-25T16:52:26.804643+0000","last_active":"2024-04-25T16:52:36.032548+0000","last_peered":"2024-04-25T16:52:36.032548+0000","last_clean":"2024-04-25T16:52:36.032548+0000","last_became_active":"2024-04-25T16:52:26.802297+0000","last_became_peered":"2024-04-25T16:52:26.802297+0000","last_unstale":"2024-04-25T16:52:36.032548+0000","last_undegraded":"2024-04-25T16:52:36.032548+0000","last_fullsized":"2024-04-25T16:52:36.032548+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:25.752315+0000","last_clean_scrub_stamp":"2024-04-25T16:52:25.752315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:32.581354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.325396+0000","last_change":"2024-04-25T16:52:25.796075+0000","last_active":"2024-04-25T16:52:36.325396+0000","last_peered":"2024-04-25T16:52:36.325396+0000","last_clean":"2024-04-25T16:52:36.325396+0000","last_became_active":"2024-04-25T16:52:25.795226+0000","last_became_peered":"2024-04-25T16:52:25.795226+0000","last_unstale":"2024-04-25T16:52:36.325396+0000","last_undegraded":"2024-04-25T16:52:36.325396+0000","last_fullsized":"2024-04-25T16:52:36.325396+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:52:24.741701+0000","last_clean_scrub_stamp":"2024-04-25T16:52:24.741701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:37.002443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":147,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T16:52:36.324771+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-25T16:52:36.324771+0000","last_peered":"2024-04-25T16:52:36.324771+0000","last_clean":"2024-04-25T16:52:36.324771+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-25T16:52:36.324771+0000","last_undegraded":"2024-04-25T16:52:36.324771+0000","last_fullsized":"2024-04-25T16:52:36.324771+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1073152,"data_stored":794775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":466944,"data_stored":203307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":471040,"data_stored":202837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":471040,"data_stored":203329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35808,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336032,"statfs":{"total":96636764160,"available":96600096768,"internally_reserved":0,"allocated":479232,"data_stored":204371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":202795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35800,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336040,"statfs":{"total":96636764160,"available":96600104960,"internally_reserved":0,"allocated":466944,"data_stored":202815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335464,"statfs":{"total":96636764160,"available":96599515136,"internally_reserved":0,"allocated":1060864,"data_stored":793127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:52:45.015 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T16:52:45.015 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T16:52:45.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.215+0000 7fc231815640 1 Processor -- start 2024-04-25T16:52:45.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.215+0000 7fc231815640 1 -- start start 2024-04-25T16:52:45.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc231815640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc22c15c7e0 0x7fc22c0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:45.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc231815640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc22c06de20 0x7fc22c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:45.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc231815640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22c074380 con 0x7fc22c0b0330 2024-04-25T16:52:45.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc231815640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fc22c0727e0 con 0x7fc22c06de20 2024-04-25T16:52:45.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc231815640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc22c072ac0 con 0x7fc22c15c7e0 2024-04-25T16:52:45.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc22a7fc640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc22c0b0330 0x7fc22c06d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:33604/0 (socket says 172.21.15.94:33604) 2024-04-25T16:52:45.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc22a7fc640 1 -- 172.21.15.94:0/820987109 learned_addr learned my addr 172.21.15.94:0/820987109 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:45.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc22b7fe640 1 --2- 172.21.15.94:0/820987109 >> v2:172.21.15.94:3300/0 conn(0x7fc22c06de20 0x7fc22c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:45.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc22affd640 1 --2- 172.21.15.94:0/820987109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc22c15c7e0 0x7fc22c0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:45.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc22b7fe640 1 -- 172.21.15.94:0/820987109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc22c15c7e0 msgr2=0x7fc22c0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:45.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc22b7fe640 1 --2- 172.21.15.94:0/820987109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc22c15c7e0 0x7fc22c0afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc22b7fe640 1 -- 172.21.15.94:0/820987109 >> v1:172.21.15.94:6789/0 conn(0x7fc22c0b0330 legacy=0x7fc22c06d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:45.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc22b7fe640 1 -- 172.21.15.94:0/820987109 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c072da0 con 0x7fc22c06de20 2024-04-25T16:52:45.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.219+0000 7fc229ffb640 1 -- 172.21.15.94:0/820987109 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2147390154 0 0) 0x7fc21c001120 con 0x7fc22c0b0330 2024-04-25T16:52:45.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc22b7fe640 1 --2- 172.21.15.94:0/820987109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc22c06de20 0x7fc22c06e7a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc224009d90 tx=0x7fc22401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef2f944e7cc16ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:45.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc229ffb640 1 -- 172.21.15.94:0/820987109 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2240b2070 con 0x7fc22c06de20 2024-04-25T16:52:45.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc229ffb640 1 -- 172.21.15.94:0/820987109 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2240a5a00 con 0x7fc22c06de20 2024-04-25T16:52:45.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc229ffb640 1 -- 172.21.15.94:0/820987109 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2240a5dd0 con 0x7fc22c06de20 2024-04-25T16:52:45.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc231815640 1 -- 172.21.15.94:0/820987109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc22c06de20 msgr2=0x7fc22c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:45.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc231815640 1 --2- 172.21.15.94:0/820987109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc22c06de20 0x7fc22c06e7a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc224009d90 tx=0x7fc22401c940 comp rx=0 tx=0).stop 2024-04-25T16:52:45.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc231815640 1 -- 172.21.15.94:0/820987109 shutdown_connections 2024-04-25T16:52:45.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc231815640 1 --2- 172.21.15.94:0/820987109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc22c06de20 0x7fc22c06e7a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc231815640 1 --2- 172.21.15.94:0/820987109 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc22c15c7e0 0x7fc22c0afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc231815640 1 -- 172.21.15.94:0/820987109 >> 172.21.15.94:0/820987109 conn(0x7fc22c0b47f0 msgr2=0x7fc22c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:45.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc231815640 1 -- 172.21.15.94:0/820987109 shutdown_connections 2024-04-25T16:52:45.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc231815640 1 -- 172.21.15.94:0/820987109 wait complete. 2024-04-25T16:52:45.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc231815640 1 Processor -- start 2024-04-25T16:52:45.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.223+0000 7fc231815640 1 -- start start 2024-04-25T16:52:45.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc231815640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc22c0b0330 0x7fc22c09e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:45.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc231815640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc22c15c7e0 0x7fc22c09b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:45.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc231815640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc22c09b720 0x7fc22c1659a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:45.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc231815640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c1695d0 con 0x7fc22c09b720 2024-04-25T16:52:45.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc231815640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c15cbf0 con 0x7fc22c0b0330 2024-04-25T16:52:45.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc231815640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc22c169750 con 0x7fc22c15c7e0 2024-04-25T16:52:45.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc22b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc22c09b720 0x7fc22c1659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:45.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc22b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc22c09b720 0x7fc22c1659a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49828/0 (socket says 172.21.15.94:49828) 2024-04-25T16:52:45.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc22b7fe640 1 -- 172.21.15.94:0/3037898534 learned_addr learned my addr 172.21.15.94:0/3037898534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:45.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc22a7fc640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc22c15c7e0 0x7fc22c09b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:45.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc22b7fe640 1 -- 172.21.15.94:0/3037898534 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc22c15c7e0 msgr2=0x7fc22c09b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:45.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc22affd640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc22c0b0330 0x7fc22c09e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:45.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc22b7fe640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc22c15c7e0 0x7fc22c09b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc22b7fe640 1 -- 172.21.15.94:0/3037898534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc22c0b0330 msgr2=0x7fc22c09e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:45.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc22b7fe640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc22c0b0330 0x7fc22c09e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc22b7fe640 1 -- 172.21.15.94:0/3037898534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c15eb40 con 0x7fc22c09b720 2024-04-25T16:52:45.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc22b7fe640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc22c09b720 0x7fc22c1659a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc22401aff0 tx=0x7fc2240196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:45.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc213fff640 1 -- 172.21.15.94:0/3037898534 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2240b2050 con 0x7fc22c09b720 2024-04-25T16:52:45.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc213fff640 1 -- 172.21.15.94:0/3037898534 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2240be070 con 0x7fc22c09b720 2024-04-25T16:52:45.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc231815640 1 -- 172.21.15.94:0/3037898534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c15ee20 con 0x7fc22c09b720 2024-04-25T16:52:45.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc213fff640 1 -- 172.21.15.94:0/3037898534 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2240adad0 con 0x7fc22c09b720 2024-04-25T16:52:45.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc231815640 1 -- 172.21.15.94:0/3037898534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c15f3e0 con 0x7fc22c09b720 2024-04-25T16:52:45.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc231815640 1 -- 172.21.15.94:0/3037898534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc22c076830 con 0x7fc22c09b720 2024-04-25T16:52:45.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc213fff640 1 -- 172.21.15.94:0/3037898534 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc2240adc70 con 0x7fc22c09b720 2024-04-25T16:52:45.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc213fff640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc21806f430 0x7fc218071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:45.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc213fff640 1 -- 172.21.15.94:0/3037898534 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fc22412a190 con 0x7fc22c09b720 2024-04-25T16:52:45.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.227+0000 7fc22affd640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc21806f430 0x7fc218071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:45.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.231+0000 7fc22affd640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc21806f430 0x7fc218071940 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc2200096f0 tx=0x7fc220008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:45.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.235+0000 7fc213fff640 1 -- 172.21.15.94:0/3037898534 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc2240c7020 con 0x7fc22c09b720 2024-04-25T16:52:45.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.451+0000 7fc231815640 1 -- 172.21.15.94:0/3037898534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc22c071100 con 0x7fc22c09b720 2024-04-25T16:52:45.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.455+0000 7fc213fff640 1 -- 172.21.15.94:0/3037898534 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fc224016560 con 0x7fc22c09b720 2024-04-25T16:52:45.453 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-04-25T16:52:45.453 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCqiSpmzcOULhAAGKJvQYLzCNAQecQi22z2GQ== 2024-04-25T16:52:45.454 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow" 2024-04-25T16:52:45.454 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow r" 2024-04-25T16:52:45.454 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T16:52:45.454 INFO:teuthology.orchestra.run.smithi094.stderr:updated caps for client.0 2024-04-25T16:52:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.459+0000 7fc231815640 1 -- 172.21.15.94:0/3037898534 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc21806f430 msgr2=0x7fc218071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.459+0000 7fc231815640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc21806f430 0x7fc218071940 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc2200096f0 tx=0x7fc220008040 comp rx=0 tx=0).stop 2024-04-25T16:52:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.459+0000 7fc231815640 1 -- 172.21.15.94:0/3037898534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc22c09b720 msgr2=0x7fc22c1659a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.459+0000 7fc231815640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc22c09b720 0x7fc22c1659a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc22401aff0 tx=0x7fc2240196e0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.459+0000 7fc231815640 1 -- 172.21.15.94:0/3037898534 shutdown_connections 2024-04-25T16:52:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.459+0000 7fc231815640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc22c09b720 0x7fc22c1659a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.459+0000 7fc231815640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc21806f430 0x7fc218071940 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.459+0000 7fc231815640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc22c15c7e0 0x7fc22c09b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.459+0000 7fc231815640 1 --2- 172.21.15.94:0/3037898534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc22c0b0330 0x7fc22c09e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.459+0000 7fc231815640 1 -- 172.21.15.94:0/3037898534 >> 172.21.15.94:0/3037898534 conn(0x7fc22c0b47f0 msgr2=0x7fc22c14f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.459+0000 7fc231815640 1 -- 172.21.15.94:0/3037898534 shutdown_connections 2024-04-25T16:52:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.459+0000 7fc231815640 1 -- 172.21.15.94:0/3037898534 wait complete. 2024-04-25T16:52:45.475 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:52:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f6d47640 1 Processor -- start 2024-04-25T16:52:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f6d47640 1 -- start start 2024-04-25T16:52:45.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f6d47640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc0f015cf40 0x7fc0f006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:45.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f6d47640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc0f006dc50 0x7fc0f00af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:45.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f6d47640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0f006e060 con 0x7fc0f0070180 2024-04-25T16:52:45.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f6d47640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fc0f015dec0 con 0x7fc0f006dc50 2024-04-25T16:52:45.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f6d47640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc0f0160760 con 0x7fc0f015cf40 2024-04-25T16:52:45.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f52e4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc0f006dc50 0x7fc0f00af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:45.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f52e4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc0f006dc50 0x7fc0f00af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49836/0 (socket says 172.21.15.94:49836) 2024-04-25T16:52:45.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f52e4640 1 -- 172.21.15.94:0/2327706149 learned_addr learned my addr 172.21.15.94:0/2327706149 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:45.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0effff640 1 --2- 172.21.15.94:0/2327706149 >> v2:172.21.15.138:3300/0 conn(0x7fc0f015cf40 0x7fc0f006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:45.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f52e4640 1 -- 172.21.15.94:0/2327706149 >> v2:172.21.15.138:3300/0 conn(0x7fc0f015cf40 msgr2=0x7fc0f006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:45.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f52e4640 1 --2- 172.21.15.94:0/2327706149 >> v2:172.21.15.138:3300/0 conn(0x7fc0f015cf40 0x7fc0f006d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f52e4640 1 -- 172.21.15.94:0/2327706149 >> v1:172.21.15.138:6789/0 conn(0x7fc0f0070180 legacy=0x7fc0f0160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:45.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.623+0000 7fc0f52e4640 1 -- 172.21.15.94:0/2327706149 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f0161300 con 0x7fc0f006dc50 2024-04-25T16:52:45.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.627+0000 7fc0f52e4640 1 --2- 172.21.15.94:0/2327706149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f006dc50 0x7fc0f00af2c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc0f009f3b0 tx=0x7fc0e801e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c03fa36c230e7b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:45.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.627+0000 7fc0ef7fe640 1 -- 172.21.15.94:0/2327706149 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e800b2d0 con 0x7fc0f006dc50 2024-04-25T16:52:45.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.627+0000 7fc0ef7fe640 1 -- 172.21.15.94:0/2327706149 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e800b470 con 0x7fc0f006dc50 2024-04-25T16:52:45.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.627+0000 7fc0ef7fe640 1 -- 172.21.15.94:0/2327706149 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80a79a0 con 0x7fc0f006dc50 2024-04-25T16:52:45.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.627+0000 7fc0f6d47640 1 -- 172.21.15.94:0/2327706149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f006dc50 msgr2=0x7fc0f00af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:45.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.627+0000 7fc0f6d47640 1 --2- 172.21.15.94:0/2327706149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f006dc50 0x7fc0f00af2c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc0f009f3b0 tx=0x7fc0e801e9b0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.627+0000 7fc0f6d47640 1 -- 172.21.15.94:0/2327706149 shutdown_connections 2024-04-25T16:52:45.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.627+0000 7fc0f6d47640 1 --2- 172.21.15.94:0/2327706149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f006dc50 0x7fc0f00af2c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.627+0000 7fc0f6d47640 1 --2- 172.21.15.94:0/2327706149 >> v2:172.21.15.138:3300/0 conn(0x7fc0f015cf40 0x7fc0f006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.627+0000 7fc0f6d47640 1 -- 172.21.15.94:0/2327706149 >> 172.21.15.94:0/2327706149 conn(0x7fc0f00b47f0 msgr2=0x7fc0f00b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:45.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.627+0000 7fc0f6d47640 1 -- 172.21.15.94:0/2327706149 shutdown_connections 2024-04-25T16:52:45.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.627+0000 7fc0f6d47640 1 -- 172.21.15.94:0/2327706149 wait complete. 2024-04-25T16:52:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f6d47640 1 Processor -- start 2024-04-25T16:52:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f6d47640 1 -- start start 2024-04-25T16:52:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f6d47640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f006dc50 0x7fc0f0153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f6d47640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0f0070180 0x7fc0f014ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f6d47640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f015cf40 0x7fc0f014f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f6d47640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f006fea0 con 0x7fc0f015cf40 2024-04-25T16:52:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f6d47640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f015dbe0 con 0x7fc0f006dc50 2024-04-25T16:52:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f6d47640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f00afc10 con 0x7fc0f0070180 2024-04-25T16:52:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f52e4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f015cf40 0x7fc0f014f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f52e4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f015cf40 0x7fc0f014f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49850/0 (socket says 172.21.15.94:49850) 2024-04-25T16:52:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f52e4640 1 -- 172.21.15.94:0/385031754 learned_addr learned my addr 172.21.15.94:0/385031754 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f4ae3640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f006dc50 0x7fc0f0153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0effff640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0f0070180 0x7fc0f014ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f52e4640 1 -- 172.21.15.94:0/385031754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0f0070180 msgr2=0x7fc0f014ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f52e4640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0f0070180 0x7fc0f014ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f52e4640 1 -- 172.21.15.94:0/385031754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f006dc50 msgr2=0x7fc0f0153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f52e4640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f006dc50 0x7fc0f0153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f52e4640 1 -- 172.21.15.94:0/385031754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f01f4fa0 con 0x7fc0f015cf40 2024-04-25T16:52:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0effff640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0f0070180 0x7fc0f014ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f4ae3640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f006dc50 0x7fc0f0153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f52e4640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f015cf40 0x7fc0f014f8f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80b1040 tx=0x7fc0e801b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0ed7fa640 1 -- 172.21.15.94:0/385031754 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e800b720 con 0x7fc0f015cf40 2024-04-25T16:52:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0ed7fa640 1 -- 172.21.15.94:0/385031754 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e80c0070 con 0x7fc0f015cf40 2024-04-25T16:52:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f6d47640 1 -- 172.21.15.94:0/385031754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f01f54e0 con 0x7fc0f015cf40 2024-04-25T16:52:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0ed7fa640 1 -- 172.21.15.94:0/385031754 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8009a20 con 0x7fc0f015cf40 2024-04-25T16:52:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f6d47640 1 -- 172.21.15.94:0/385031754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f01f5aa0 con 0x7fc0f015cf40 2024-04-25T16:52:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.631+0000 7fc0f6d47640 1 -- 172.21.15.94:0/385031754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f01f5250 con 0x7fc0f015cf40 2024-04-25T16:52:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.635+0000 7fc0ed7fa640 1 -- 172.21.15.94:0/385031754 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc0e800b2e0 con 0x7fc0f015cf40 2024-04-25T16:52:45.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.635+0000 7fc0ed7fa640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc0d006f400 0x7fc0d0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:45.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.635+0000 7fc0f4ae3640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc0d006f400 0x7fc0d0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:45.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.635+0000 7fc0ed7fa640 1 -- 172.21.15.94:0/385031754 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fc0e812c510 con 0x7fc0f015cf40 2024-04-25T16:52:45.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.635+0000 7fc0f4ae3640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc0d006f400 0x7fc0d0071910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0071c30 tx=0x7fc0e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:45.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.639+0000 7fc0ed7fa640 1 -- 172.21.15.94:0/385031754 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc0f006e060 con 0x7fc0f015cf40 2024-04-25T16:52:45.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.831+0000 7fc0f6d47640 1 -- 172.21.15.94:0/385031754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0f0160260 con 0x7fc0f015cf40 2024-04-25T16:52:45.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.831+0000 7fc0ed7fa640 1 -- 172.21.15.94:0/385031754 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5184 (secure 0 0 0) 0x7fc0e80f92e0 con 0x7fc0f015cf40 2024-04-25T16:52:45.830 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:52:45.830 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/478421533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":478421533},{"type":"v1","addr":"172.21.15.138:6833","nonce":478421533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2461866660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2461866660},{"type":"v1","addr":"172.21.15.138:6835","nonce":2461866660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5439,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2566493915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2566493915},{"type":"v1","addr":"172.21.15.94:6835","nonce":2566493915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:52:26.175197+0000","modified":"2024-04-25T16:52:29.233625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5442":{"gid":5442,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/2235039329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2235039329},{"type":"v1","addr":"172.21.15.94:6837","nonce":2235039329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5442,"qdb_cluster":[5442]},"id":2}]} 2024-04-25T16:52:45.831 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-04-25T16:52:45.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.835+0000 7fc0f6d47640 1 -- 172.21.15.94:0/385031754 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc0d006f400 msgr2=0x7fc0d0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:45.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.835+0000 7fc0f6d47640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc0d006f400 0x7fc0d0071910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0071c30 tx=0x7fc0e0008040 comp rx=0 tx=0).stop 2024-04-25T16:52:45.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.835+0000 7fc0f6d47640 1 -- 172.21.15.94:0/385031754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f015cf40 msgr2=0x7fc0f014f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:45.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.835+0000 7fc0f6d47640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f015cf40 0x7fc0f014f8f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80b1040 tx=0x7fc0e801b750 comp rx=0 tx=0).stop 2024-04-25T16:52:45.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.835+0000 7fc0f6d47640 1 -- 172.21.15.94:0/385031754 shutdown_connections 2024-04-25T16:52:45.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.835+0000 7fc0f6d47640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f015cf40 0x7fc0f014f8f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.835+0000 7fc0f6d47640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fc0d006f400 0x7fc0d0071910 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.835+0000 7fc0f6d47640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0f0070180 0x7fc0f014ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.835+0000 7fc0f6d47640 1 --2- 172.21.15.94:0/385031754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f006dc50 0x7fc0f0153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:45.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.835+0000 7fc0f6d47640 1 -- 172.21.15.94:0/385031754 >> 172.21.15.94:0/385031754 conn(0x7fc0f00b47f0 msgr2=0x7fc0f00b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:45.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.835+0000 7fc0f6d47640 1 -- 172.21.15.94:0/385031754 shutdown_connections 2024-04-25T16:52:45.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:45.835+0000 7fc0f6d47640 1 -- 172.21.15.94:0/385031754 wait complete. 2024-04-25T16:52:46.255 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T16:52:26.175197+0000', 'modified': '2024-04-25T16:52:29.233625+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5442}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5442': {'gid': 5442, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6837/2235039329', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6836', 'nonce': 2235039329}, {'type': 'v1', 'addr': '172.21.15.94:6837', 'nonce': 2235039329}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5442, 'qdb_cluster': [5442]} 2024-04-25T16:52:46.255 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T16:52:46.255 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T16:52:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c9144640 1 Processor -- start 2024-04-25T16:52:46.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c9144640 1 -- start start 2024-04-25T16:52:46.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c9144640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97c41666d0 0x7f97c4168af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c9144640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97c4070630 con 0x7f97c406f470 2024-04-25T16:52:46.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c9144640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97c40ae170 con 0x7f97c40b0420 2024-04-25T16:52:46.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c9144640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f97c40aeaa0 con 0x7f97c41666d0 2024-04-25T16:52:46.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c259c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f97c406f470 0x7f97c415dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:33618/0 (socket says 172.21.15.94:33618) 2024-04-25T16:52:46.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c259c640 1 -- 172.21.15.94:0/4068795752 learned_addr learned my addr 172.21.15.94:0/4068795752 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:46.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c359e640 1 --2- 172.21.15.94:0/4068795752 >> v2:172.21.15.138:3300/0 conn(0x7f97c41666d0 0x7f97c4168af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1752083448 0 0) 0x7f97c4070630 con 0x7f97c406f470 2024-04-25T16:52:46.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97ac003530 con 0x7f97c406f470 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2336876737 0 0) 0x7f97ac003530 con 0x7f97c406f470 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 >> v2:172.21.15.138:3300/0 conn(0x7f97c41666d0 msgr2=0x7f97c4168af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c1d9b640 1 --2- 172.21.15.94:0/4068795752 >> v2:172.21.15.138:3300/0 conn(0x7f97c41666d0 0x7f97c4168af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 >> v1:172.21.15.138:6789/0 conn(0x7f97c40b0420 legacy=0x7f97c406ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c40ae450 con 0x7f97c406f470 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.419+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f97b8001680 con 0x7f97c406f470 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 >> v1:172.21.15.94:6789/0 conn(0x7f97c406f470 legacy=0x7f97c415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c1d9b640 1 --2- 172.21.15.94:0/4068795752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97ac0039c0 0x7f97ac003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c1d9b640 1 --2- 172.21.15.94:0/4068795752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97ac004810 0x7f97ac004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c1d9b640 1 --2- 172.21.15.94:0/4068795752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97ac0051d0 0x7f97ac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97c40aeaa0 con 0x7f97ac0039c0 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac00c170 con 0x7f97ac004810 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f97ac00c370 con 0x7f97ac0051d0 2024-04-25T16:52:46.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/4068795752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97ac0039c0 0x7f97ac003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c259c640 1 --2- 172.21.15.94:0/4068795752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97ac004810 0x7f97ac004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c359e640 1 --2- 172.21.15.94:0/4068795752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97ac0051d0 0x7f97ac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c2d9d640 1 -- 172.21.15.94:0/4068795752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97ac0051d0 msgr2=0x7f97ac00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:46.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/4068795752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97ac0051d0 0x7f97ac00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c2d9d640 1 -- 172.21.15.94:0/4068795752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97ac004810 msgr2=0x7f97ac004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:46.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/4068795752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97ac004810 0x7f97ac004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c2d9d640 1 -- 172.21.15.94:0/4068795752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ac003530 con 0x7f97ac0039c0 2024-04-25T16:52:46.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c359e640 1 -- 172.21.15.94:0/4068795752 reap_dead start 2024-04-25T16:52:46.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c259c640 1 --2- 172.21.15.94:0/4068795752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97ac004810 0x7f97ac004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:46.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/4068795752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97ac0039c0 0x7f97ac003e30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f97b4001750 tx=0x7f97b401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e4a36e87b975a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:46.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40013a0 con 0x7f97ac0039c0 2024-04-25T16:52:46.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b40052f0 con 0x7f97ac0039c0 2024-04-25T16:52:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c1d9b640 1 -- 172.21.15.94:0/4068795752 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4003840 con 0x7f97ac0039c0 2024-04-25T16:52:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c9144640 1 -- 172.21.15.94:0/4068795752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97ac0039c0 msgr2=0x7f97ac003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c9144640 1 --2- 172.21.15.94:0/4068795752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97ac0039c0 0x7f97ac003e30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f97b4001750 tx=0x7f97b401c300 comp rx=0 tx=0).stop 2024-04-25T16:52:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c9144640 1 -- 172.21.15.94:0/4068795752 shutdown_connections 2024-04-25T16:52:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c9144640 1 --2- 172.21.15.94:0/4068795752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97ac0039c0 0x7f97ac003e30 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c9144640 1 -- 172.21.15.94:0/4068795752 >> 172.21.15.94:0/4068795752 conn(0x7f97c40b47f0 msgr2=0x7f97c40b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c9144640 1 -- 172.21.15.94:0/4068795752 shutdown_connections 2024-04-25T16:52:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.423+0000 7f97c9144640 1 -- 172.21.15.94:0/4068795752 wait complete. 2024-04-25T16:52:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c9144640 1 Processor -- start 2024-04-25T16:52:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c9144640 1 -- start start 2024-04-25T16:52:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c9144640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97ac0039c0 0x7f97c414e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c9144640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97c41f77a0 0x7f97c41f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c9144640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c4151480 0x7f97c4151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c9144640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97c406c600 con 0x7f97c41f77a0 2024-04-25T16:52:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c9144640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97c40b0a40 con 0x7f97c4151480 2024-04-25T16:52:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c9144640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f97c406c780 con 0x7f97ac0039c0 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97c41f77a0 0x7f97c41f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97c41f77a0 0x7f97c41f7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49872/0 (socket says 172.21.15.94:49872) 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c259c640 1 -- 172.21.15.94:0/1293022632 learned_addr learned my addr 172.21.15.94:0/1293022632 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97ac0039c0 0x7f97c414e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c259c640 1 -- 172.21.15.94:0/1293022632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97ac0039c0 msgr2=0x7f97c414e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c259c640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97ac0039c0 0x7f97c414e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c259c640 1 -- 172.21.15.94:0/1293022632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c4151480 msgr2=0x7f97c4151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c359e640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c4151480 0x7f97c4151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c259c640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c4151480 0x7f97c4151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c259c640 1 -- 172.21.15.94:0/1293022632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c4151dd0 con 0x7f97c41f77a0 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97ac0039c0 0x7f97c414e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c259c640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97c41f77a0 0x7f97c41f7bb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f97b800d7d0 tx=0x7f97b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97ab7fe640 1 -- 172.21.15.94:0/1293022632 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8019070 con 0x7f97c41f77a0 2024-04-25T16:52:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97ab7fe640 1 -- 172.21.15.94:0/1293022632 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b80072d0 con 0x7f97c41f77a0 2024-04-25T16:52:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c9144640 1 -- 172.21.15.94:0/1293022632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c4153ce0 con 0x7f97c41f77a0 2024-04-25T16:52:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97ab7fe640 1 -- 172.21.15.94:0/1293022632 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b80047f0 con 0x7f97c41f77a0 2024-04-25T16:52:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.427+0000 7f97c9144640 1 -- 172.21.15.94:0/1293022632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c41542a0 con 0x7f97c41f77a0 2024-04-25T16:52:46.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.431+0000 7f97ab7fe640 1 -- 172.21.15.94:0/1293022632 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f97b801c070 con 0x7f97c41f77a0 2024-04-25T16:52:46.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.431+0000 7f97ab7fe640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f97b006f400 0x7f97b0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.431+0000 7f97ab7fe640 1 -- 172.21.15.94:0/1293022632 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f97b808ed90 con 0x7f97c41f77a0 2024-04-25T16:52:46.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.431+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f97b006f400 0x7f97b0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.431+0000 7f97d04a1000 1 Processor -- start 2024-04-25T16:52:46.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.431+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f97b006f400 0x7f97b0071910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97c40cd2f0 tx=0x7f97b40011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:46.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.431+0000 7f97d04a1000 1 -- start start 2024-04-25T16:52:46.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.431+0000 7f97d04a1000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c84c20 0x564106de2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.431+0000 7f97d04a1000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 0x564106de3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.431+0000 7f97d04a1000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106869280 0x564106de3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.431+0000 7f97d04a1000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564106c52160 con 0x564106c70ea0 2024-04-25T16:52:46.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.431+0000 7f97d04a1000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564106be7e20 con 0x564106c84c20 2024-04-25T16:52:46.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564106ba0a20 con 0x564106869280 2024-04-25T16:52:46.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97c259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 0x564106de3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97c259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 0x564106de3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49886/0 (socket says 172.21.15.94:49886) 2024-04-25T16:52:46.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97c259c640 1 -- 172.21.15.94:0/3238038829 learned_addr learned my addr 172.21.15.94:0/3238038829 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:46.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97c359e640 1 --2- 172.21.15.94:0/3238038829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106869280 0x564106de3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/3238038829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c84c20 0x564106de2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97c259c640 1 -- 172.21.15.94:0/3238038829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106869280 msgr2=0x564106de3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:46.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97c259c640 1 --2- 172.21.15.94:0/3238038829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106869280 0x564106de3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97c259c640 1 -- 172.21.15.94:0/3238038829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c84c20 msgr2=0x564106de2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:46.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97c259c640 1 --2- 172.21.15.94:0/3238038829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c84c20 0x564106de2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97c259c640 1 -- 172.21.15.94:0/3238038829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564106ba5100 con 0x564106c70ea0 2024-04-25T16:52:46.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97c259c640 1 --2- 172.21.15.94:0/3238038829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 0x564106de3420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f97b805b2c0 tx=0x7f97b8057320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d1d1df750fcb15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:46.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97a8ff9640 1 -- 172.21.15.94:0/3238038829 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b805f6e0 con 0x564106c70ea0 2024-04-25T16:52:46.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97a8ff9640 1 -- 172.21.15.94:0/3238038829 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b8064190 con 0x564106c70ea0 2024-04-25T16:52:46.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97a8ff9640 1 -- 172.21.15.94:0/3238038829 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b805a790 con 0x564106c70ea0 2024-04-25T16:52:46.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 -- 172.21.15.94:0/3238038829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 msgr2=0x564106de3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 --2- 172.21.15.94:0/3238038829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 0x564106de3420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f97b805b2c0 tx=0x7f97b8057320 comp rx=0 tx=0).stop 2024-04-25T16:52:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 -- 172.21.15.94:0/3238038829 shutdown_connections 2024-04-25T16:52:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 --2- 172.21.15.94:0/3238038829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106869280 0x564106de3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 --2- 172.21.15.94:0/3238038829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 0x564106de3420 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 --2- 172.21.15.94:0/3238038829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c84c20 0x564106de2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 -- 172.21.15.94:0/3238038829 >> 172.21.15.94:0/3238038829 conn(0x5641068a8ff0 msgr2=0x564106d984c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 -- 172.21.15.94:0/3238038829 shutdown_connections 2024-04-25T16:52:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 -- 172.21.15.94:0/3238038829 wait complete. 2024-04-25T16:52:46.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 -- start start 2024-04-25T16:52:46.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 Processor -- start 2024-04-25T16:52:46.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106869280 0x564106de9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 0x564106dea3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c84c20 0x564106da50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.435+0000 7f97d04a1000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564106bf5220 con 0x564106c70ea0 2024-04-25T16:52:46.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564106b8bea0 con 0x564106869280 2024-04-25T16:52:46.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564106be2b00 con 0x564106c84c20 2024-04-25T16:52:46.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 10 client.0 _ll_register_callbacks cb 0x564106bf4dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:52:46.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97c259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 0x564106dea3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97c259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 0x564106dea3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49898/0 (socket says 172.21.15.94:49898) 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97c259c640 1 -- 172.21.15.94:0/2861507518 learned_addr learned my addr 172.21.15.94:0/2861507518 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/2861507518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106869280 0x564106de9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97c359e640 1 --2- 172.21.15.94:0/2861507518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c84c20 0x564106da50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97c259c640 1 -- 172.21.15.94:0/2861507518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c84c20 msgr2=0x564106da50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97c259c640 1 --2- 172.21.15.94:0/2861507518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c84c20 0x564106da50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97c259c640 1 -- 172.21.15.94:0/2861507518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106869280 msgr2=0x564106de9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97c259c640 1 --2- 172.21.15.94:0/2861507518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106869280 0x564106de9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97c259c640 1 -- 172.21.15.94:0/2861507518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564106b87500 con 0x564106c70ea0 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97c259c640 1 --2- 172.21.15.94:0/2861507518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 0x564106dea3a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f97b805b2c0 tx=0x7f97b8020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed23aad7ccbfafd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97a1ffb640 1 -- 172.21.15.94:0/2861507518 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b805f6e0 con 0x564106c70ea0 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97a1ffb640 1 -- 172.21.15.94:0/2861507518 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f97b808bc70 con 0x564106c70ea0 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 -- 172.21.15.94:0/2861507518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b8064190 con 0x564106c70ea0 2024-04-25T16:52:46.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97a1ffb640 1 -- 172.21.15.94:0/2861507518 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f97b808be90 con 0x564106c70ea0 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 10 client.5700 fetch_fsmap learned FSMap version 18 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 -- 172.21.15.94:0/2861507518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x564106b6dea0 con 0x564106c70ea0 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97a1ffb640 1 -- 172.21.15.94:0/2861507518 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b8064190 con 0x564106c70ea0 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97a1ffb640 1 -- 172.21.15.94:0/2861507518 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f97b8074b20 con 0x564106c70ea0 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 10 client.5700 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 10 client.5700 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 20 client.5700 populate_metadata read hostname 'smithi094' 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 --2- 172.21.15.94:0/2861507518 >> [v2:172.21.15.94:6836/2235039329,v1:172.21.15.94:6837/2235039329] conn(0x56410688d7d0 0x564106da5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 4 client.5700 mds_command: new command op to 5442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 -- 172.21.15.94:0/2861507518 --> [v2:172.21.15.94:6836/2235039329,v1:172.21.15.94:6837/2235039329] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564106b80be0 con 0x56410688d7d0 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/2861507518 >> [v2:172.21.15.94:6836/2235039329,v1:172.21.15.94:6837/2235039329] conn(0x56410688d7d0 0x564106da5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97a1ffb640 10 client.5700 ms_handle_connect on v2:172.21.15.94:6836/2235039329 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/2861507518 >> [v2:172.21.15.94:6836/2235039329,v1:172.21.15.94:6837/2235039329] conn(0x56410688d7d0 0x564106da5e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8828a5202aeee9 server_cookie=d73d8c4436950693 in_seq=0 out_seq=0 2024-04-25T16:52:46.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97a1ffb640 1 -- 172.21.15.94:0/2861507518 <== mds.0 v2:172.21.15.94:6836/2235039329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564106b80be0 con 0x56410688d7d0 2024-04-25T16:52:46.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97a1ffb640 10 client.5700 handle_command_reply: tid=0 2024-04-25T16:52:46.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 client.5700 shutdown 2024-04-25T16:52:46.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 -- 172.21.15.94:0/2861507518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 msgr2=0x564106dea3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 --2- 172.21.15.94:0/2861507518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 0x564106dea3a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f97b805b2c0 tx=0x7f97b8020040 comp rx=0 tx=0).stop 2024-04-25T16:52:46.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 -- 172.21.15.94:0/2861507518 shutdown_connections 2024-04-25T16:52:46.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 --2- 172.21.15.94:0/2861507518 >> [v2:172.21.15.94:6836/2235039329,v1:172.21.15.94:6837/2235039329] conn(0x56410688d7d0 0x564106da5e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 --2- 172.21.15.94:0/2861507518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c84c20 0x564106da50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 --2- 172.21.15.94:0/2861507518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106c70ea0 0x564106dea3a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97a1ffb640 0 client.5700 ms_handle_reset on v2:172.21.15.94:6836/2235039329 2024-04-25T16:52:46.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 --2- 172.21.15.94:0/2861507518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106869280 0x564106de9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 -- 172.21.15.94:0/2861507518 >> 172.21.15.94:0/2861507518 conn(0x5641068a8ff0 msgr2=0x564106d98c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:46.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 -- 172.21.15.94:0/2861507518 shutdown_connections 2024-04-25T16:52:46.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 1 -- 172.21.15.94:0/2861507518 wait complete. 2024-04-25T16:52:46.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.439+0000 7f97d04a1000 20 client.5700 trim_cache size 0 max 16384 2024-04-25T16:52:46.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 Processor -- start 2024-04-25T16:52:46.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 -- start start 2024-04-25T16:52:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 0x564106deeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c70ea0 0x564106def3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c84c20 0x564106df8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4203480 con 0x564106869280 2024-04-25T16:52:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4152f00 con 0x564106c84c20 2024-04-25T16:52:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f97c4202fd0 con 0x564106c70ea0 2024-04-25T16:52:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97c2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 0x564106deeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97c2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 0x564106deeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49906/0 (socket says 172.21.15.94:49906) 2024-04-25T16:52:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97c2d9d640 1 -- 172.21.15.94:0/26230854 learned_addr learned my addr 172.21.15.94:0/26230854 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97c2d9d640 1 -- 172.21.15.94:0/26230854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c70ea0 msgr2=0x564106def3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:46.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/26230854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c70ea0 0x564106def3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97c2d9d640 1 -- 172.21.15.94:0/26230854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c84c20 msgr2=0x564106df8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:46.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/26230854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c84c20 0x564106df8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97c2d9d640 1 -- 172.21.15.94:0/26230854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564106d9c840 con 0x564106869280 2024-04-25T16:52:46.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/26230854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 0x564106deeea0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f97b4000c10 tx=0x7f97b4001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502f1136436b361a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:46.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97a3fff640 1 -- 172.21.15.94:0/26230854 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40067e0 con 0x564106869280 2024-04-25T16:52:46.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97a3fff640 1 -- 172.21.15.94:0/26230854 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b4006980 con 0x564106869280 2024-04-25T16:52:46.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97a3fff640 1 -- 172.21.15.94:0/26230854 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4006d50 con 0x564106869280 2024-04-25T16:52:46.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 -- 172.21.15.94:0/26230854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 msgr2=0x564106deeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 --2- 172.21.15.94:0/26230854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 0x564106deeea0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f97b4000c10 tx=0x7f97b4001130 comp rx=0 tx=0).stop 2024-04-25T16:52:46.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 -- 172.21.15.94:0/26230854 shutdown_connections 2024-04-25T16:52:46.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 --2- 172.21.15.94:0/26230854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c84c20 0x564106df8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 --2- 172.21.15.94:0/26230854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c70ea0 0x564106def3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 --2- 172.21.15.94:0/26230854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 0x564106deeea0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 -- 172.21.15.94:0/26230854 >> 172.21.15.94:0/26230854 conn(0x5641068a8ff0 msgr2=0x564106da0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:46.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 -- 172.21.15.94:0/26230854 shutdown_connections 2024-04-25T16:52:46.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.459+0000 7f97d04a1000 1 -- 172.21.15.94:0/26230854 wait complete. 2024-04-25T16:52:46.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 1 -- start start 2024-04-25T16:52:46.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 1 Processor -- start 2024-04-25T16:52:46.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 0x564106e59790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c70ea0 0x564106e59cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c84c20 0x564106ded3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564106be7130 con 0x564106869280 2024-04-25T16:52:46.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564106b7d7d0 con 0x564106c70ea0 2024-04-25T16:52:46.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564106bec450 con 0x564106c84c20 2024-04-25T16:52:46.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 10 client.0 _ll_register_callbacks cb 0x7f97b8074070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:52:46.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97c2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 0x564106e59790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97c2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 0x564106e59790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49922/0 (socket says 172.21.15.94:49922) 2024-04-25T16:52:46.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97c2d9d640 1 -- 172.21.15.94:0/347464202 learned_addr learned my addr 172.21.15.94:0/347464202 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:46.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97c2d9d640 1 -- 172.21.15.94:0/347464202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c84c20 msgr2=0x564106ded3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:46.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97c359e640 1 --2- 172.21.15.94:0/347464202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c84c20 0x564106ded3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/347464202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c84c20 0x564106ded3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97c2d9d640 1 -- 172.21.15.94:0/347464202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c70ea0 msgr2=0x564106e59cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:46.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97c259c640 1 --2- 172.21.15.94:0/347464202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c70ea0 0x564106e59cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/347464202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c70ea0 0x564106e59cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97c2d9d640 1 -- 172.21.15.94:0/347464202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564106da5430 con 0x564106869280 2024-04-25T16:52:46.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97c2d9d640 1 --2- 172.21.15.94:0/347464202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 0x564106e59790 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f97b4007fd0 tx=0x7f97b40190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60512827df037cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:46.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T16:52:46.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97a1ffb640 1 -- 172.21.15.94:0/347464202 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4006d30 con 0x564106869280 2024-04-25T16:52:46.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97a1ffb640 1 -- 172.21.15.94:0/347464202 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f97b40cfca0 con 0x564106869280 2024-04-25T16:52:46.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 1 -- 172.21.15.94:0/347464202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b4006980 con 0x564106869280 2024-04-25T16:52:46.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97a1ffb640 1 -- 172.21.15.94:0/347464202 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f97b4003600 con 0x564106869280 2024-04-25T16:52:46.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 10 client.5706 fetch_fsmap learned FSMap version 18 2024-04-25T16:52:46.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 1 -- 172.21.15.94:0/347464202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x564106ba0c70 con 0x564106869280 2024-04-25T16:52:46.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97a1ffb640 1 -- 172.21.15.94:0/347464202 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b4006980 con 0x564106869280 2024-04-25T16:52:46.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97a1ffb640 1 -- 172.21.15.94:0/347464202 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f97c406f570 con 0x564106869280 2024-04-25T16:52:46.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 10 client.5706 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T16:52:46.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 10 client.5706 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T16:52:46.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 20 client.5706 populate_metadata read hostname 'smithi094' 2024-04-25T16:52:46.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 1 --2- 172.21.15.94:0/347464202 >> [v2:172.21.15.94:6836/2235039329,v1:172.21.15.94:6837/2235039329] conn(0x56410687b830 0x564106dedd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 4 client.5706 mds_command: new command op to 5442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T16:52:46.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97d04a1000 1 -- 172.21.15.94:0/347464202 --> [v2:172.21.15.94:6836/2235039329,v1:172.21.15.94:6837/2235039329] -- command(tid 0: {"prefix": "status"}) -- 0x564106bf5a30 con 0x56410687b830 2024-04-25T16:52:46.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.463+0000 7f97c259c640 1 --2- 172.21.15.94:0/347464202 >> [v2:172.21.15.94:6836/2235039329,v1:172.21.15.94:6837/2235039329] conn(0x56410687b830 0x564106dedd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97a1ffb640 10 client.5706 ms_handle_connect on v2:172.21.15.94:6836/2235039329 2024-04-25T16:52:46.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97c259c640 1 --2- 172.21.15.94:0/347464202 >> [v2:172.21.15.94:6836/2235039329,v1:172.21.15.94:6837/2235039329] conn(0x56410687b830 0x564106dedd00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18e311c763e38ff server_cookie=89ad22bb64d4d8d in_seq=0 out_seq=0 2024-04-25T16:52:46.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97a1ffb640 1 -- 172.21.15.94:0/347464202 <== mds.0 v2:172.21.15.94:6836/2235039329 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x564106bf5a30 con 0x56410687b830 2024-04-25T16:52:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97a1ffb640 10 client.5706 handle_command_reply: tid=0 2024-04-25T16:52:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97d04a1000 1 client.5706 shutdown 2024-04-25T16:52:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97d04a1000 1 -- 172.21.15.94:0/347464202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 msgr2=0x564106e59790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97d04a1000 1 --2- 172.21.15.94:0/347464202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 0x564106e59790 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f97b4007fd0 tx=0x7f97b40190a0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97d04a1000 1 -- 172.21.15.94:0/347464202 shutdown_connections 2024-04-25T16:52:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97d04a1000 1 --2- 172.21.15.94:0/347464202 >> [v2:172.21.15.94:6836/2235039329,v1:172.21.15.94:6837/2235039329] conn(0x56410687b830 0x564106dedd00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97d04a1000 1 --2- 172.21.15.94:0/347464202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564106c84c20 0x564106ded3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97a1ffb640 0 client.5706 ms_handle_reset on v2:172.21.15.94:6836/2235039329 2024-04-25T16:52:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97d04a1000 1 --2- 172.21.15.94:0/347464202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564106c70ea0 0x564106e59cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97d04a1000 1 --2- 172.21.15.94:0/347464202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564106869280 0x564106e59790 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97d04a1000 1 -- 172.21.15.94:0/347464202 >> 172.21.15.94:0/347464202 conn(0x5641068a8ff0 msgr2=0x564106de8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97d04a1000 1 -- 172.21.15.94:0/347464202 shutdown_connections 2024-04-25T16:52:46.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97d04a1000 1 -- 172.21.15.94:0/347464202 wait complete. 2024-04-25T16:52:46.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97d04a1000 20 client.5706 trim_cache size 0 max 16384 2024-04-25T16:52:46.470 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:52:46.470 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "cc26689a-ed7a-405e-9abc-b7fa2c1906cf", 2024-04-25T16:52:46.470 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-04-25T16:52:46.470 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 5442, 2024-04-25T16:52:46.471 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T16:52:46.471 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T16:52:46.471 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T16:52:46.471 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 19.658876405000001, 2024-04-25T16:52:46.471 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 18, 2024-04-25T16:52:46.471 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 40, 2024-04-25T16:52:46.471 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 38, 2024-04-25T16:52:46.471 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 27.569282762 2024-04-25T16:52:46.471 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:52:46.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97c9144640 1 -- 172.21.15.94:0/1293022632 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f97b006f400 msgr2=0x7f97b0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97c9144640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f97b006f400 0x7f97b0071910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97c40cd2f0 tx=0x7f97b40011a0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97c9144640 1 -- 172.21.15.94:0/1293022632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97c41f77a0 msgr2=0x7f97c41f7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97c9144640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97c41f77a0 0x7f97c41f7bb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f97b800d7d0 tx=0x7f97b800dc90 comp rx=0 tx=0).stop 2024-04-25T16:52:46.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97c9144640 1 -- 172.21.15.94:0/1293022632 shutdown_connections 2024-04-25T16:52:46.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97c9144640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c4151480 0x7f97c4151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97c9144640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97c41f77a0 0x7f97c41f7bb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97c9144640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f97b006f400 0x7f97b0071910 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97c9144640 1 --2- 172.21.15.94:0/1293022632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97ac0039c0 0x7f97c414e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97c9144640 1 -- 172.21.15.94:0/1293022632 >> 172.21.15.94:0/1293022632 conn(0x7f97c40b47f0 msgr2=0x7f97c415cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97c9144640 1 -- 172.21.15.94:0/1293022632 shutdown_connections 2024-04-25T16:52:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.467+0000 7f97c9144640 1 -- 172.21.15.94:0/1293022632 wait complete. 2024-04-25T16:52:46.482 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T16:52:46.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f459327a640 1 Processor -- start 2024-04-25T16:52:46.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f459327a640 1 -- start start 2024-04-25T16:52:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f459327a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f458c070080 0x7f458c15e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f459327a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f458c15ebd0 0x7f458c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f459327a640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c06fc60 con 0x7f458c15e7c0 2024-04-25T16:52:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f459327a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f458c158550 con 0x7f458c15ebd0 2024-04-25T16:52:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f459327a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f458c06f7d0 con 0x7f458c070080 2024-04-25T16:52:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f4591817640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f458c15ebd0 0x7f458c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f4590815640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f458c15e7c0 0x7f458c06c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:33624/0 (socket says 172.21.15.94:33624) 2024-04-25T16:52:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f4590815640 1 -- 172.21.15.94:0/1998330734 learned_addr learned my addr 172.21.15.94:0/1998330734 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-25T16:52:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f4591016640 1 --2- 172.21.15.94:0/1998330734 >> v2:172.21.15.138:3300/0 conn(0x7f458c070080 0x7f458c15e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f4591817640 1 -- 172.21.15.94:0/1998330734 >> v2:172.21.15.138:3300/0 conn(0x7f458c070080 msgr2=0x7f458c15e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f4591817640 1 --2- 172.21.15.94:0/1998330734 >> v2:172.21.15.138:3300/0 conn(0x7f458c070080 0x7f458c15e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f4591817640 1 -- 172.21.15.94:0/1998330734 >> v1:172.21.15.94:6789/0 conn(0x7f458c15e7c0 legacy=0x7f458c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f4583fff640 1 -- 172.21.15.94:0/1998330734 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 772575673 0 0) 0x7f458c06fc60 con 0x7f458c15e7c0 2024-04-25T16:52:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.619+0000 7f4591817640 1 -- 172.21.15.94:0/1998330734 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c0aef70 con 0x7f458c15ebd0 2024-04-25T16:52:46.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f4591817640 1 --2- 172.21.15.94:0/1998330734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c15ebd0 0x7f458c06cd10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f458c071f10 tx=0x7f458401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df66a81d9bb76851 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:46.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f4583fff640 1 -- 172.21.15.94:0/1998330734 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45840b2070 con 0x7f458c15ebd0 2024-04-25T16:52:46.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f4583fff640 1 -- 172.21.15.94:0/1998330734 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45840a58d0 con 0x7f458c15ebd0 2024-04-25T16:52:46.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f4583fff640 1 -- 172.21.15.94:0/1998330734 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45840a5ca0 con 0x7f458c15ebd0 2024-04-25T16:52:46.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f459327a640 1 -- 172.21.15.94:0/1998330734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c15ebd0 msgr2=0x7f458c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f459327a640 1 --2- 172.21.15.94:0/1998330734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c15ebd0 0x7f458c06cd10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f458c071f10 tx=0x7f458401c900 comp rx=0 tx=0).stop 2024-04-25T16:52:46.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f459327a640 1 -- 172.21.15.94:0/1998330734 shutdown_connections 2024-04-25T16:52:46.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f459327a640 1 --2- 172.21.15.94:0/1998330734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c15ebd0 0x7f458c06cd10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f459327a640 1 --2- 172.21.15.94:0/1998330734 >> v2:172.21.15.138:3300/0 conn(0x7f458c070080 0x7f458c15e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f459327a640 1 -- 172.21.15.94:0/1998330734 >> 172.21.15.94:0/1998330734 conn(0x7f458c0b47f0 msgr2=0x7f458c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:46.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f459327a640 1 -- 172.21.15.94:0/1998330734 shutdown_connections 2024-04-25T16:52:46.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f459327a640 1 -- 172.21.15.94:0/1998330734 wait complete. 2024-04-25T16:52:46.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f459327a640 1 Processor -- start 2024-04-25T16:52:46.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.623+0000 7f459327a640 1 -- start start 2024-04-25T16:52:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f459327a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f458c070080 0x7f458c099290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f459327a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c15e7c0 0x7f458c154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f459327a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c15ebd0 0x7f458c15cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f459327a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f458c070490 con 0x7f458c15ebd0 2024-04-25T16:52:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f459327a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f458c0ae6c0 con 0x7f458c15e7c0 2024-04-25T16:52:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f459327a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f458c16ae90 con 0x7f458c070080 2024-04-25T16:52:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4591817640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c15ebd0 0x7f458c15cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4591817640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c15ebd0 0x7f458c15cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49946/0 (socket says 172.21.15.94:49946) 2024-04-25T16:52:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4591817640 1 -- 172.21.15.94:0/3971752281 learned_addr learned my addr 172.21.15.94:0/3971752281 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:52:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4591016640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f458c070080 0x7f458c099290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4590815640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c15e7c0 0x7f458c154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4591817640 1 -- 172.21.15.94:0/3971752281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f458c070080 msgr2=0x7f458c099290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4591817640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f458c070080 0x7f458c099290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4591817640 1 -- 172.21.15.94:0/3971752281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c15e7c0 msgr2=0x7f458c154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4591817640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c15e7c0 0x7f458c154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4591817640 1 -- 172.21.15.94:0/3971752281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c073a60 con 0x7f458c15ebd0 2024-04-25T16:52:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4591016640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f458c070080 0x7f458c099290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4590815640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c15e7c0 0x7f458c154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4591817640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c15ebd0 0x7f458c15cae0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f45840b0040 tx=0x7f45840196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4581ffb640 1 -- 172.21.15.94:0/3971752281 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45840b2050 con 0x7f458c15ebd0 2024-04-25T16:52:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4581ffb640 1 -- 172.21.15.94:0/3971752281 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45840be070 con 0x7f458c15ebd0 2024-04-25T16:52:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f459327a640 1 -- 172.21.15.94:0/3971752281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c073d40 con 0x7f458c15ebd0 2024-04-25T16:52:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4581ffb640 1 -- 172.21.15.94:0/3971752281 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45840ada90 con 0x7f458c15ebd0 2024-04-25T16:52:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f459327a640 1 -- 172.21.15.94:0/3971752281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c074300 con 0x7f458c15ebd0 2024-04-25T16:52:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f459327a640 1 -- 172.21.15.94:0/3971752281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458c076830 con 0x7f458c15ebd0 2024-04-25T16:52:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.627+0000 7f4581ffb640 1 -- 172.21.15.94:0/3971752281 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f45840adc30 con 0x7f458c15ebd0 2024-04-25T16:52:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.631+0000 7f4581ffb640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f457006f400 0x7f4570071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.631+0000 7f4591016640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f457006f400 0x7f4570071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.631+0000 7f4581ffb640 1 -- 172.21.15.94:0/3971752281 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f458412a5d0 con 0x7f458c15ebd0 2024-04-25T16:52:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.631+0000 7f4591016640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f457006f400 0x7f4570071910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f458c0b50c0 tx=0x7f4588008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.635+0000 7f4581ffb640 1 -- 172.21.15.94:0/3971752281 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4584016520 con 0x7f458c15ebd0 2024-04-25T16:52:46.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.831+0000 7f459327a640 1 -- 172.21.15.94:0/3971752281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f458c0737b0 con 0x7f458c15ebd0 2024-04-25T16:52:46.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.831+0000 7f4581ffb640 1 -- 172.21.15.94:0/3971752281 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f45840f73a0 con 0x7f458c15ebd0 2024-04-25T16:52:46.830 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T16:52:46.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.835+0000 7f459327a640 1 -- 172.21.15.94:0/3971752281 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f457006f400 msgr2=0x7f4570071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.835+0000 7f459327a640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f457006f400 0x7f4570071910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f458c0b50c0 tx=0x7f4588008040 comp rx=0 tx=0).stop 2024-04-25T16:52:46.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.835+0000 7f459327a640 1 -- 172.21.15.94:0/3971752281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c15ebd0 msgr2=0x7f458c15cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:46.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.835+0000 7f459327a640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c15ebd0 0x7f458c15cae0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f45840b0040 tx=0x7f45840196a0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.835+0000 7f459327a640 1 -- 172.21.15.94:0/3971752281 shutdown_connections 2024-04-25T16:52:46.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.835+0000 7f459327a640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f458c15ebd0 0x7f458c15cae0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.835+0000 7f459327a640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f458c15e7c0 0x7f458c154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.835+0000 7f459327a640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f457006f400 0x7f4570071910 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.835+0000 7f459327a640 1 --2- 172.21.15.94:0/3971752281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f458c070080 0x7f458c099290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:46.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.835+0000 7f459327a640 1 -- 172.21.15.94:0/3971752281 >> 172.21.15.94:0/3971752281 conn(0x7f458c0b47f0 msgr2=0x7f458c06c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:46.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.835+0000 7f459327a640 1 -- 172.21.15.94:0/3971752281 shutdown_connections 2024-04-25T16:52:46.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:46.835+0000 7f459327a640 1 -- 172.21.15.94:0/3971752281 wait complete. 2024-04-25T16:52:46.850 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T16:52:46.850 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T16:52:46.850 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-04-25T16:52:46.850 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T16:52:46.850 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T16:52:46.851 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T16:52:46.851 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T16:52:46.851 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T16:52:46.851 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T16:52:46.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:46.851 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-04-25T16:52:46.857 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:52:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:52:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:52:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:52:46.857 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42359sec preferred_lft 42359sec 2024-04-25T16:52:46.858 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:f8:37:e2:6a brd ff:ff:ff:ff:ff:ff 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:52:46.859 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T16:52:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:52:46.860 INFO:teuthology.orchestra.run.smithi094.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T16:52:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:52:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T16:52:46.860 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:52:46.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:46.860 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T16:52:46.911 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-04-25T16:52:46.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:46.915 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-04-25T16:52:46.963 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-04-25T16:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T16:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T16:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T16:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T16:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T16:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T16:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T16:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T16:52:46.965 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:52:46.965 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T16:52:46.965 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T16:52:46.965 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T16:52:46.965 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T16:52:46.965 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T16:52:47.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:52:47.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:52:47.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:47.083 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T16:52:47.133 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T16:52:47.133 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T16:52:47.133 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T16:52:47.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:47.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-04-25T16:52:47.190 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T16:52:47.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:52:47.190 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T16:52:47.236 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T16:52:47.236 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:52:47.236 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T16:52:47.284 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:52:47.341 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:52:47.389 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:52:47.389 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:52:47.389 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:52:47.389 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:52:47.389 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:52:47.389 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:52:47.389 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:52:47.389 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:52:47.389 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-04-25T16:52:47.389 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:52:47.389 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:52:47.389 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:52:47.390 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:52:47.390 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:52:47.390 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:52:47.390 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:52:47.390 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:52:47.390 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-04-25T16:52:47.390 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:52:47.390 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:52:47.390 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:52:47.390 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-04-25T16:52:47.390 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:52:47.390 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:52:47.390 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:52:47.391 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:52:47.391 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:52:47.391 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:52:47.391 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:52:47.391 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:52:47.391 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:52:47.391 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:52:47.391 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:52:47.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:47.392 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:52:47.439 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T16:52:47.439 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T16:52:47.485 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:52:47.499 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:52:47.548 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:52:47.548 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:52:47.548 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:52:47.548 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:52:47.548 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:52:47.549 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:52:47.550 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-04-25T16:52:47.550 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:52:47.550 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:52:47.550 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:52:47.550 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:52:47.550 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:52:47.550 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:52:47.550 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:52:47.550 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:52:47.550 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:52:47.550 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:52:47.550 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:52:47.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:47.551 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:52:47.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-04-25T16:52:47.599+0000 7eff960af580 -1 init, newargv = 0x7eff880033b0 newargc=15 2024-04-25T16:52:47.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[39350]: starting ceph client 2024-04-25T16:52:47.603 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[39350]: starting fuse 2024-04-25T16:52:48.700 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-04-25T16:52:48.717 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T16:52:48.767 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-04-25T16:52:48.767 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-04-25T16:52:48.767 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-04-25T16:52:48.767 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-04-25T16:52:48.767 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-04-25T16:52:48.767 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-04-25T16:52:48.767 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-04-25T16:52:48.767 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T16:52:48.768 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-04-25T16:52:48.769 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-04-25T16:52:48.769 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-04-25T16:52:48.769 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T16:52:48.769 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T16:52:48.769 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T16:52:48.769 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T16:52:48.769 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-04-25T16:52:48.769 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:52:48.769 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:52:48.769 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:52:48.769 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T16:52:48.769 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T16:52:48.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:48.770 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-04-25T16:52:48.816 INFO:teuthology.orchestra.run.smithi094.stdout:45 2024-04-25T16:52:48.816 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T16:52:48.816 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:52:48.817 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-04-25T16:52:48.818 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T16:52:48.818 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T16:52:48.818 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T16:52:48.818 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-04-25T16:52:48.818 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-04-25T16:52:48.818 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-04-25T16:52:48.818 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T16:52:48.818 DEBUG:teuthology.orchestra.run.smithi094:> 2024-04-25T16:52:48.818 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-04-25T16:52:48.818 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-04-25T16:52:48.940 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.39350.asok 2024-04-25T16:52:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:52:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:52:48.951 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39350.asok 2024-04-25T16:52:48.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:48.952 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39350.asok status 2024-04-25T16:52:49.134 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T16:52:49.134 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-04-25T16:52:49.134 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T16:52:49.134 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T16:52:49.134 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-04-25T16:52:49.134 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-04-25T16:52:49.134 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T16:52:49.134 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "39350", 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 5721, 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 5721 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-04-25T16:52:49.135 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 622917418 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 622917418 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.5721 192.168.0.1:0/622917418", 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/622917418", 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 18, 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 40, 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-04-25T16:52:49.136 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-04-25T16:52:49.137 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T16:52:49.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:49.151 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-04-25T16:52:49.151 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T16:52:49.157 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-04-25T16:52:49.157 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T16:52:49.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:49.157 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T16:52:49.210 DEBUG:teuthology.orchestra.run.smithi094:> mktemp -d --suffix=xfstests-dev 2024-04-25T16:52:49.256 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.FooT1Le6w4xfstests-dev 2024-04-25T16:52:49.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:49.256 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.FooT1Le6w4xfstests-dev) 2024-04-25T16:52:49.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:49.313 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.FooT1Le6w4xfstests-dev) 2024-04-25T16:52:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:Cloning into '/tmp/tmp.FooT1Le6w4xfstests-dev'... 2024-04-25T16:52:50.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:50.175 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-04-25T16:52:50.230 DEBUG:teuthology.orchestra.run.smithi094:> mktemp -d --suffix=test 2024-04-25T16:52:50.276 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.PS50Hvgg8stest 2024-04-25T16:52:50.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:52:50.277 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-04-25T16:52:50.329 DEBUG:teuthology.orchestra.run.smithi094:> mktemp -d --suffix=scratch 2024-04-25T16:52:50.378 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.cwl13YwnfEscratch 2024-04-25T16:52:50.379 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "ubuntu" and "22.04". 2024-04-25T16:52:50.379 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc git indent libacl1-dev libaio-dev libcap-dev libgdbm-dev libtool libtool-bin liburing-dev libuuid1 lvm2 make psmisc python3 quota sed uuid-dev uuid-runtime xfsprogs sqlite3 exfatprogs f2fs-tools ocfs2-tools udftools xfsdump xfslibs-dev 2024-04-25T16:52:50.490 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-04-25T16:52:50.696 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2024-04-25T16:52:50.697 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2024-04-25T16:52:50.906 INFO:teuthology.orchestra.run.smithi094.stdout:acl is already the newest version (2.3.1-1). 2024-04-25T16:52:50.907 INFO:teuthology.orchestra.run.smithi094.stdout:acl set to manually installed. 2024-04-25T16:52:50.907 INFO:teuthology.orchestra.run.smithi094.stdout:attr is already the newest version (1:2.5.1-1build1). 2024-04-25T16:52:50.907 INFO:teuthology.orchestra.run.smithi094.stdout:automake is already the newest version (1:1.16.5-1.3). 2024-04-25T16:52:50.907 INFO:teuthology.orchestra.run.smithi094.stdout:bc is already the newest version (1.07.1-3build1). 2024-04-25T16:52:50.907 INFO:teuthology.orchestra.run.smithi094.stdout:gcc is already the newest version (4:11.2.0-1ubuntu1). 2024-04-25T16:52:50.907 INFO:teuthology.orchestra.run.smithi094.stdout:gcc set to manually installed. 2024-04-25T16:52:50.907 INFO:teuthology.orchestra.run.smithi094.stdout:libacl1-dev is already the newest version (2.3.1-1). 2024-04-25T16:52:50.907 INFO:teuthology.orchestra.run.smithi094.stdout:libaio-dev is already the newest version (0.3.112-13build1). 2024-04-25T16:52:50.907 INFO:teuthology.orchestra.run.smithi094.stdout:libtool is already the newest version (2.4.6-15build2). 2024-04-25T16:52:50.907 INFO:teuthology.orchestra.run.smithi094.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T16:52:50.907 INFO:teuthology.orchestra.run.smithi094.stdout:lvm2 is already the newest version (2.03.11-2.1ubuntu4). 2024-04-25T16:52:50.908 INFO:teuthology.orchestra.run.smithi094.stdout:make is already the newest version (4.3-4.1build1). 2024-04-25T16:52:50.908 INFO:teuthology.orchestra.run.smithi094.stdout:make set to manually installed. 2024-04-25T16:52:50.908 INFO:teuthology.orchestra.run.smithi094.stdout:psmisc is already the newest version (23.4-2build3). 2024-04-25T16:52:50.908 INFO:teuthology.orchestra.run.smithi094.stdout:quota is already the newest version (4.06-1build2). 2024-04-25T16:52:50.908 INFO:teuthology.orchestra.run.smithi094.stdout:sed is already the newest version (4.8-1ubuntu2). 2024-04-25T16:52:50.908 INFO:teuthology.orchestra.run.smithi094.stdout:xfsdump is already the newest version (3.1.9+0+nmu2). 2024-04-25T16:52:50.908 INFO:teuthology.orchestra.run.smithi094.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T16:52:50.908 INFO:teuthology.orchestra.run.smithi094.stdout:xfsprogs is already the newest version (5.13.0-1ubuntu2). 2024-04-25T16:52:50.908 INFO:teuthology.orchestra.run.smithi094.stdout:dbench is already the newest version (4.0-2build1). 2024-04-25T16:52:50.908 INFO:teuthology.orchestra.run.smithi094.stdout:dump is already the newest version (0.4b47-3). 2024-04-25T16:52:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:e2fsprogs is already the newest version (1.46.5-2ubuntu1.1). 2024-04-25T16:52:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:git is already the newest version (1:2.34.1-1ubuntu1.10). 2024-04-25T16:52:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:indent is already the newest version (2.2.12-1ubuntu0.22.04.1). 2024-04-25T16:52:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:libuuid1 is already the newest version (2.37.2-4ubuntu3.4). 2024-04-25T16:52:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:python3 is already the newest version (3.10.6-1~22.04). 2024-04-25T16:52:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3.4). 2024-04-25T16:52:50.909 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T16:52:50.909 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libiscsi7 libpmemobj1 2024-04-25T16:52:50.909 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2-dev libpython2-stdlib libpython2.7 libpython2.7-dev 2024-04-25T16:52:50.910 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7-minimal libpython2.7-stdlib linux-image-5.4.0-135-generic 2024-04-25T16:52:50.910 INFO:teuthology.orchestra.run.smithi094.stdout: linux-modules-5.4.0-135-generic python2 python2-dev python2-minimal 2024-04-25T16:52:50.911 INFO:teuthology.orchestra.run.smithi094.stdout: python2.7 python2.7-dev python2.7-minimal 2024-04-25T16:52:50.911 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T16:52:50.911 INFO:teuthology.orchestra.run.smithi094.stdout:The following additional packages will be installed: 2024-04-25T16:52:50.911 INFO:teuthology.orchestra.run.smithi094.stdout: libcap2 libcmap4 libcorosync-common4 libdlm3 libpmemblk1 libqb100 2024-04-25T16:52:50.911 INFO:teuthology.orchestra.run.smithi094.stdout: libsqlite3-0 2024-04-25T16:52:50.911 INFO:teuthology.orchestra.run.smithi094.stdout:Suggested packages: 2024-04-25T16:52:50.911 INFO:teuthology.orchestra.run.smithi094.stdout: gnuplot gfio python-scipy gawk-doc sqlite3-doc dvd+rw-tools udfclient 2024-04-25T16:52:50.945 INFO:teuthology.orchestra.run.smithi094.stdout:The following NEW packages will be installed: 2024-04-25T16:52:50.945 INFO:teuthology.orchestra.run.smithi094.stdout: exfatprogs f2fs-tools fio libcap-dev libcmap4 libcorosync-common4 libdlm3 2024-04-25T16:52:50.946 INFO:teuthology.orchestra.run.smithi094.stdout: libgdbm-dev libpmemblk1 libqb100 liburing-dev ocfs2-tools sqlite3 udftools 2024-04-25T16:52:50.947 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages will be upgraded: 2024-04-25T16:52:50.948 INFO:teuthology.orchestra.run.smithi094.stdout: gawk libcap2 libsqlite3-0 uuid-runtime 2024-04-25T16:52:51.008 INFO:teuthology.orchestra.run.smithi094.stdout:4 upgraded, 14 newly installed, 0 to remove and 294 not upgraded. 2024-04-25T16:52:51.008 INFO:teuthology.orchestra.run.smithi094.stdout:Need to get 7,314 kB of archives. 2024-04-25T16:52:51.008 INFO:teuthology.orchestra.run.smithi094.stdout:After this operation, 14.2 MB of additional disk space will be used. 2024-04-25T16:52:51.009 INFO:teuthology.orchestra.run.smithi094.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-runtime amd64 2.37.2-4ubuntu3.4 [32.0 kB] 2024-04-25T16:52:51.058 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 gawk amd64 1:5.1.0-1ubuntu0.1 [447 kB] 2024-04-25T16:52:51.141 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 libcorosync-common4 amd64 3.1.6-1ubuntu1 [7,322 B] 2024-04-25T16:52:51.142 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqb100 amd64 2.0.4-1ubuntu0.2 [74.2 kB] 2024-04-25T16:52:51.145 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 libcmap4 amd64 3.1.6-1ubuntu1 [10.4 kB] 2024-04-25T16:52:51.146 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 libdlm3 amd64 4.1.1-1build1 [14.4 kB] 2024-04-25T16:52:51.147 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 ocfs2-tools amd64 1.8.7-1build1 [646 kB] 2024-04-25T16:52:51.171 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libcap2 amd64 1:2.44-1ubuntu0.22.04.1 [18.3 kB] 2024-04-25T16:52:51.172 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libsqlite3-0 amd64 3.37.2-2ubuntu0.3 [641 kB] 2024-04-25T16:52:51.187 INFO:teuthology.orchestra.run.smithi094.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 exfatprogs amd64 1.1.3-1 [40.3 kB] 2024-04-25T16:52:51.188 INFO:teuthology.orchestra.run.smithi094.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 f2fs-tools amd64 1.14.0-2build1 [169 kB] 2024-04-25T16:52:51.192 INFO:teuthology.orchestra.run.smithi094.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libpmemblk1 amd64 1.11.1-3build1 [65.6 kB] 2024-04-25T16:52:51.193 INFO:teuthology.orchestra.run.smithi094.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 fio amd64 3.28-1 [4,047 kB] 2024-04-25T16:52:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libcap-dev amd64 1:2.44-1ubuntu0.22.04.1 [39.4 kB] 2024-04-25T16:52:51.247 INFO:teuthology.orchestra.run.smithi094.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgdbm-dev amd64 1.23-1 [117 kB] 2024-04-25T16:52:51.248 INFO:teuthology.orchestra.run.smithi094.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liburing-dev amd64 2.1-2build1 [51.8 kB] 2024-04-25T16:52:51.248 INFO:teuthology.orchestra.run.smithi094.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 sqlite3 amd64 3.37.2-2ubuntu0.3 [768 kB] 2024-04-25T16:52:51.255 INFO:teuthology.orchestra.run.smithi094.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 udftools amd64 2.3-1 [125 kB] 2024-04-25T16:52:51.557 INFO:teuthology.orchestra.run.smithi094.stderr:debconf: unable to initialize frontend: Dialog 2024-04-25T16:52:51.557 INFO:teuthology.orchestra.run.smithi094.stderr:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-04-25T16:52:51.557 INFO:teuthology.orchestra.run.smithi094.stderr:debconf: falling back to frontend: Readline 2024-04-25T16:52:51.568 INFO:teuthology.orchestra.run.smithi094.stderr:debconf: unable to initialize frontend: Readline 2024-04-25T16:52:51.568 INFO:teuthology.orchestra.run.smithi094.stderr:debconf: (This frontend requires a controlling tty.) 2024-04-25T16:52:51.568 INFO:teuthology.orchestra.run.smithi094.stderr:debconf: falling back to frontend: Teletype 2024-04-25T16:52:51.572 INFO:teuthology.orchestra.run.smithi094.stderr:dpkg-preconfigure: unable to re-open stdin: 2024-04-25T16:52:51.606 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 7,314 kB in 0s (24.3 MB/s) 2024-04-25T16:52:51.912 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176797 files and directories currently installed.) 2024-04-25T16:52:51.918 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../0-uuid-runtime_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T16:52:52.054 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking uuid-runtime (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T16:52:52.515 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../1-gawk_1%3a5.1.0-1ubuntu0.1_amd64.deb ... 2024-04-25T16:52:52.646 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking gawk (1:5.1.0-1ubuntu0.1) over (1:5.1.0-1build3) ... 2024-04-25T16:52:53.117 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libcorosync-common4:amd64. 2024-04-25T16:52:53.146 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../2-libcorosync-common4_3.1.6-1ubuntu1_amd64.deb ... 2024-04-25T16:52:53.209 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcorosync-common4:amd64 (3.1.6-1ubuntu1) ... 2024-04-25T16:52:53.546 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqb100:amd64. 2024-04-25T16:52:53.574 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../3-libqb100_2.0.4-1ubuntu0.2_amd64.deb ... 2024-04-25T16:52:53.613 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqb100:amd64 (2.0.4-1ubuntu0.2) ... 2024-04-25T16:52:53.925 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libcmap4:amd64. 2024-04-25T16:52:53.942 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../4-libcmap4_3.1.6-1ubuntu1_amd64.deb ... 2024-04-25T16:52:53.983 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcmap4:amd64 (3.1.6-1ubuntu1) ... 2024-04-25T16:52:54.296 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libdlm3:amd64. 2024-04-25T16:52:54.324 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../5-libdlm3_4.1.1-1build1_amd64.deb ... 2024-04-25T16:52:54.362 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libdlm3:amd64 (4.1.1-1build1) ... 2024-04-25T16:52:54.809 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ocfs2-tools. 2024-04-25T16:52:54.837 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../6-ocfs2-tools_1.8.7-1build1_amd64.deb ... 2024-04-25T16:52:54.875 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ocfs2-tools (1.8.7-1build1) ... 2024-04-25T16:52:55.310 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../7-libcap2_1%3a2.44-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:52:55.441 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcap2:amd64 (1:2.44-1ubuntu0.22.04.1) over (1:2.44-1build3) ... 2024-04-25T16:52:55.804 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcap2:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-04-25T16:52:56.267 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176871 files and directories currently installed.) 2024-04-25T16:52:56.273 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../0-libsqlite3-0_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-25T16:52:56.410 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) over (3.37.2-2ubuntu0.1) ... 2024-04-25T16:52:56.723 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package exfatprogs. 2024-04-25T16:52:56.752 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../1-exfatprogs_1.1.3-1_amd64.deb ... 2024-04-25T16:52:56.790 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking exfatprogs (1.1.3-1) ... 2024-04-25T16:52:57.068 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package f2fs-tools. 2024-04-25T16:52:57.085 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../2-f2fs-tools_1.14.0-2build1_amd64.deb ... 2024-04-25T16:52:57.177 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking f2fs-tools (1.14.0-2build1) ... 2024-04-25T16:52:57.490 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libpmemblk1:amd64. 2024-04-25T16:52:57.519 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../3-libpmemblk1_1.11.1-3build1_amd64.deb ... 2024-04-25T16:52:57.548 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libpmemblk1:amd64 (1.11.1-3build1) ... 2024-04-25T16:52:57.902 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package fio. 2024-04-25T16:52:57.931 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../4-fio_3.28-1_amd64.deb ... 2024-04-25T16:52:57.969 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking fio (3.28-1) ... 2024-04-25T16:52:58.416 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libcap-dev:amd64. 2024-04-25T16:52:58.432 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../5-libcap-dev_1%3a2.44-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T16:52:58.474 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcap-dev:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-04-25T16:52:58.714 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libgdbm-dev:amd64. 2024-04-25T16:52:58.730 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../6-libgdbm-dev_1.23-1_amd64.deb ... 2024-04-25T16:52:58.764 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libgdbm-dev:amd64 (1.23-1) ... 2024-04-25T16:52:59.060 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liburing-dev:amd64. 2024-04-25T16:52:59.076 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../7-liburing-dev_2.1-2build1_amd64.deb ... 2024-04-25T16:52:59.118 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liburing-dev:amd64 (2.1-2build1) ... 2024-04-25T16:52:59.390 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package sqlite3. 2024-04-25T16:52:59.417 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../8-sqlite3_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-25T16:52:59.455 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-25T16:52:59.843 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package udftools. 2024-04-25T16:52:59.871 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../9-udftools_2.3-1_amd64.deb ... 2024-04-25T16:52:59.922 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking udftools (2.3-1) ... 2024-04-25T16:53:00.221 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up exfatprogs (1.1.3-1) ... 2024-04-25T16:53:00.349 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up gawk (1:5.1.0-1ubuntu0.1) ... 2024-04-25T16:53:00.577 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up f2fs-tools (1.14.0-2build1) ... 2024-04-25T16:53:00.703 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) ... 2024-04-25T16:53:00.830 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqb100:amd64 (2.0.4-1ubuntu0.2) ... 2024-04-25T16:53:00.956 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcorosync-common4:amd64 (3.1.6-1ubuntu1) ... 2024-04-25T16:53:01.099 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libpmemblk1:amd64 (1.11.1-3build1) ... 2024-04-25T16:53:01.200 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up uuid-runtime (2.37.2-4ubuntu3.4) ... 2024-04-25T16:53:02.084 INFO:teuthology.orchestra.run.smithi094.stdout:uuidd.service is a disabled or a static unit not running, not starting it. 2024-04-25T16:53:02.136 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up udftools (2.3-1) ... 2024-04-25T16:53:02.288 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libdlm3:amd64 (4.1.1-1build1) ... 2024-04-25T16:53:02.414 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libgdbm-dev:amd64 (1.23-1) ... 2024-04-25T16:53:02.540 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcap-dev:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-04-25T16:53:02.658 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-25T16:53:02.759 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liburing-dev:amd64 (2.1-2build1) ... 2024-04-25T16:53:02.878 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcmap4:amd64 (3.1.6-1ubuntu1) ... 2024-04-25T16:53:03.004 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up fio (3.28-1) ... 2024-04-25T16:53:03.515 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ocfs2-tools (1.8.7-1build1) ... 2024-04-25T16:53:03.848 INFO:teuthology.orchestra.run.smithi094.stdout:debconf: unable to initialize frontend: Dialog 2024-04-25T16:53:03.848 INFO:teuthology.orchestra.run.smithi094.stdout:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-04-25T16:53:03.849 INFO:teuthology.orchestra.run.smithi094.stdout:debconf: falling back to frontend: Readline 2024-04-25T16:53:04.570 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for initramfs-tools (0.140ubuntu13.1) ... 2024-04-25T16:53:04.670 INFO:teuthology.orchestra.run.smithi094.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T16:53:14.594 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:53:14.597 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:53:22.848 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T16:53:22.992 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T16:53:24.315 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T16:53:24.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo useradd -m fsgqa 2024-04-25T16:53:24.848 INFO:teuthology.orchestra.run.smithi094.stderr:useradd: user 'fsgqa' already exists 2024-04-25T16:53:24.849 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-25T16:53:24.849 DEBUG:teuthology.orchestra.run.smithi094:> sudo groupadd fsgqa 2024-04-25T16:53:24.899 INFO:teuthology.orchestra.run.smithi094.stderr:groupadd: group 'fsgqa' already exists 2024-04-25T16:53:24.900 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-25T16:53:24.901 DEBUG:teuthology.orchestra.run.smithi094:> sudo useradd fsgqa2 2024-04-25T16:53:25.319 DEBUG:teuthology.orchestra.run.smithi094:> sudo useradd 123456-fsgqa 2024-04-25T16:53:25.682 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-04-25T16:53:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32e1a29640 1 Processor -- start 2024-04-25T16:53:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32e1a29640 1 -- start start 2024-04-25T16:53:25.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32e1a29640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f32dc06e960 0x7f32dc072670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:25.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32e1a29640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f32dc160ae0 0x7f32dc0cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:25.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32e1a29640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32dc072fc0 con 0x7f32dc072bb0 2024-04-25T16:53:25.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32e1a29640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f32dc0d3230 con 0x7f32dc160ae0 2024-04-25T16:53:25.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32e1a29640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f32dc0d2990 con 0x7f32dc06e960 2024-04-25T16:53:25.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32db7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f32dc160ae0 0x7f32dc0cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:25.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32db7fe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f32dc160ae0 0x7f32dc0cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52740/0 (socket says 172.21.15.94:52740) 2024-04-25T16:53:25.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32db7fe640 1 -- 172.21.15.94:0/1556645352 learned_addr learned my addr 172.21.15.94:0/1556645352 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:53:25.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32daffd640 1 --2- 172.21.15.94:0/1556645352 >> v2:172.21.15.138:3300/0 conn(0x7f32dc06e960 0x7f32dc072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:25.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32db7fe640 1 -- 172.21.15.94:0/1556645352 >> v2:172.21.15.138:3300/0 conn(0x7f32dc06e960 msgr2=0x7f32dc072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:25.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32db7fe640 1 --2- 172.21.15.94:0/1556645352 >> v2:172.21.15.138:3300/0 conn(0x7f32dc06e960 0x7f32dc072670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:25.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32db7fe640 1 -- 172.21.15.94:0/1556645352 >> v1:172.21.15.138:6789/0 conn(0x7f32dc072bb0 legacy=0x7f32dc0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:25.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.846+0000 7f32db7fe640 1 -- 172.21.15.94:0/1556645352 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc0d2c70 con 0x7f32dc160ae0 2024-04-25T16:53:25.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.850+0000 7f32db7fe640 1 --2- 172.21.15.94:0/1556645352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc160ae0 0x7f32dc0cea50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f32d4009e50 tx=0x7f32d401c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f56a37364b1c8df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:25.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.850+0000 7f32d9ffb640 1 -- 172.21.15.94:0/1556645352 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40a55d0 con 0x7f32dc160ae0 2024-04-25T16:53:25.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.850+0000 7f32d9ffb640 1 -- 172.21.15.94:0/1556645352 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d40a5770 con 0x7f32dc160ae0 2024-04-25T16:53:25.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.850+0000 7f32d9ffb640 1 -- 172.21.15.94:0/1556645352 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40a5b40 con 0x7f32dc160ae0 2024-04-25T16:53:25.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.850+0000 7f32e1a29640 1 -- 172.21.15.94:0/1556645352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc160ae0 msgr2=0x7f32dc0cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:25.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.850+0000 7f32e1a29640 1 --2- 172.21.15.94:0/1556645352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc160ae0 0x7f32dc0cea50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f32d4009e50 tx=0x7f32d401c710 comp rx=0 tx=0).stop 2024-04-25T16:53:25.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.850+0000 7f32e1a29640 1 -- 172.21.15.94:0/1556645352 shutdown_connections 2024-04-25T16:53:25.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.850+0000 7f32e1a29640 1 --2- 172.21.15.94:0/1556645352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc160ae0 0x7f32dc0cea50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:25.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.850+0000 7f32e1a29640 1 --2- 172.21.15.94:0/1556645352 >> v2:172.21.15.138:3300/0 conn(0x7f32dc06e960 0x7f32dc072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:25.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.850+0000 7f32e1a29640 1 -- 172.21.15.94:0/1556645352 >> 172.21.15.94:0/1556645352 conn(0x7f32dc0d3850 msgr2=0x7f32dc0d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:25.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.850+0000 7f32e1a29640 1 -- 172.21.15.94:0/1556645352 shutdown_connections 2024-04-25T16:53:25.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.850+0000 7f32e1a29640 1 -- 172.21.15.94:0/1556645352 wait complete. 2024-04-25T16:53:25.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32e1a29640 1 Processor -- start 2024-04-25T16:53:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32e1a29640 1 -- start start 2024-04-25T16:53:25.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32e1a29640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32dc06e960 0x7f32dc0ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:25.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32e1a29640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32dc072bb0 0x7f32dc0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:25.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32e1a29640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc160ae0 0x7f32dc157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32e1a29640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc06eef0 con 0x7f32dc160ae0 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32e1a29640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc06ed70 con 0x7f32dc072bb0 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32e1a29640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f32dc16c7d0 con 0x7f32dc06e960 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32db7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc160ae0 0x7f32dc157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32db7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc160ae0 0x7f32dc157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52756/0 (socket says 172.21.15.94:52756) 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32db7fe640 1 -- 172.21.15.94:0/494243938 learned_addr learned my addr 172.21.15.94:0/494243938 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32daffd640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32dc06e960 0x7f32dc0ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32da7fc640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32dc072bb0 0x7f32dc0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32db7fe640 1 -- 172.21.15.94:0/494243938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32dc06e960 msgr2=0x7f32dc0ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32db7fe640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32dc06e960 0x7f32dc0ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32db7fe640 1 -- 172.21.15.94:0/494243938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32dc072bb0 msgr2=0x7f32dc0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32db7fe640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32dc072bb0 0x7f32dc0cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32db7fe640 1 -- 172.21.15.94:0/494243938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc0af160 con 0x7f32dc160ae0 2024-04-25T16:53:25.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32db7fe640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc160ae0 0x7f32dc157590 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f32d401a340 tx=0x7f32dc0cf7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:25.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32c3fff640 1 -- 172.21.15.94:0/494243938 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40bd020 con 0x7f32dc160ae0 2024-04-25T16:53:25.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32c3fff640 1 -- 172.21.15.94:0/494243938 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d40b8670 con 0x7f32dc160ae0 2024-04-25T16:53:25.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32e1a29640 1 -- 172.21.15.94:0/494243938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32dc0af440 con 0x7f32dc160ae0 2024-04-25T16:53:25.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32c3fff640 1 -- 172.21.15.94:0/494243938 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4009510 con 0x7f32dc160ae0 2024-04-25T16:53:25.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.854+0000 7f32e1a29640 1 -- 172.21.15.94:0/494243938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32dc0afa00 con 0x7f32dc160ae0 2024-04-25T16:53:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.858+0000 7f32e1a29640 1 -- 172.21.15.94:0/494243938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32dc06a730 con 0x7f32dc160ae0 2024-04-25T16:53:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.858+0000 7f32c3fff640 1 -- 172.21.15.94:0/494243938 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f32d40c0070 con 0x7f32dc160ae0 2024-04-25T16:53:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.858+0000 7f32c3fff640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f32c806f400 0x7f32c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.858+0000 7f32daffd640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f32c806f400 0x7f32c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.858+0000 7f32c3fff640 1 -- 172.21.15.94:0/494243938 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f32d41296a0 con 0x7f32dc160ae0 2024-04-25T16:53:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.858+0000 7f32daffd640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f32c806f400 0x7f32c8071910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f32dc1612e0 tx=0x7f32d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:25.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:25.862+0000 7f32c3fff640 1 -- 172.21.15.94:0/494243938 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f32dc072fc0 con 0x7f32dc160ae0 2024-04-25T16:53:26.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.086+0000 7f32e1a29640 1 -- 172.21.15.94:0/494243938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f32dc0d0ca0 con 0x7f32dc160ae0 2024-04-25T16:53:26.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.086+0000 7f32c3fff640 1 -- 172.21.15.94:0/494243938 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1285 (secure 0 0 0) 0x7f32d40f63e0 con 0x7f32dc160ae0 2024-04-25T16:53:26.086 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:53:26.086 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":1,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","modified":"2024-04-25T16:49:45.643473Z","created":"2024-04-25T16:49:45.643473Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3301","nonce":0},{"type":"v1","addr":"172.21.15.138:6790","nonce":0}]},"addr":"172.21.15.138:6790/0","public_addr":"172.21.15.138:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T16:53:26.086 INFO:teuthology.orchestra.run.smithi094.stderr:dumped monmap epoch 1 2024-04-25T16:53:26.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.090+0000 7f32e1a29640 1 -- 172.21.15.94:0/494243938 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f32c806f400 msgr2=0x7f32c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:26.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.090+0000 7f32e1a29640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f32c806f400 0x7f32c8071910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f32dc1612e0 tx=0x7f32d0008040 comp rx=0 tx=0).stop 2024-04-25T16:53:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.090+0000 7f32e1a29640 1 -- 172.21.15.94:0/494243938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc160ae0 msgr2=0x7f32dc157590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.090+0000 7f32e1a29640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc160ae0 0x7f32dc157590 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f32d401a340 tx=0x7f32dc0cf7a0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.090+0000 7f32e1a29640 1 -- 172.21.15.94:0/494243938 shutdown_connections 2024-04-25T16:53:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.090+0000 7f32e1a29640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc160ae0 0x7f32dc157590 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.090+0000 7f32e1a29640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32dc072bb0 0x7f32dc0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.090+0000 7f32e1a29640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f32c806f400 0x7f32c8071910 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.090+0000 7f32e1a29640 1 --2- 172.21.15.94:0/494243938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32dc06e960 0x7f32dc0ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.090+0000 7f32e1a29640 1 -- 172.21.15.94:0/494243938 >> 172.21.15.94:0/494243938 conn(0x7f32dc0d3850 msgr2=0x7f32dc15ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.090+0000 7f32e1a29640 1 -- 172.21.15.94:0/494243938 shutdown_connections 2024-04-25T16:53:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.090+0000 7f32e1a29640 1 -- 172.21.15.94:0/494243938 wait complete. 2024-04-25T16:53:26.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2024-04-25T16:53:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.270+0000 7f599d156640 1 Processor -- start 2024-04-25T16:53:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.270+0000 7f599d156640 1 -- start start 2024-04-25T16:53:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.270+0000 7f599d156640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f599815c7e0 0x7f59980afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.270+0000 7f599d156640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f599806de20 0x7f599806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:26.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.270+0000 7f599d156640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5998074380 con 0x7f59980b0330 2024-04-25T16:53:26.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.270+0000 7f599d156640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f59980727e0 con 0x7f599806de20 2024-04-25T16:53:26.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.270+0000 7f599d156640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5998072ac0 con 0x7f599815c7e0 2024-04-25T16:53:26.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.270+0000 7f599759e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f599806de20 0x7f599806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:26.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.270+0000 7f599759e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f599806de20 0x7f599806e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52760/0 (socket says 172.21.15.94:52760) 2024-04-25T16:53:26.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.270+0000 7f599759e640 1 -- 172.21.15.94:0/3888426701 learned_addr learned my addr 172.21.15.94:0/3888426701 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:53:26.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.270+0000 7f5996d9d640 1 --2- 172.21.15.94:0/3888426701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f599815c7e0 0x7f59980afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:26.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.270+0000 7f599759e640 1 -- 172.21.15.94:0/3888426701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f599815c7e0 msgr2=0x7f59980afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:26.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.270+0000 7f599759e640 1 --2- 172.21.15.94:0/3888426701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f599815c7e0 0x7f59980afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f599759e640 1 -- 172.21.15.94:0/3888426701 >> v1:172.21.15.138:6789/0 conn(0x7f59980b0330 legacy=0x7f599806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:26.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f599759e640 1 -- 172.21.15.94:0/3888426701 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5998072da0 con 0x7f599806de20 2024-04-25T16:53:26.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f599759e640 1 --2- 172.21.15.94:0/3888426701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f599806de20 0x7f599806e7a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5990009e50 tx=0x7f599001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd595ddab71c2ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:26.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f5995d9b640 1 -- 172.21.15.94:0/3888426701 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59900a58d0 con 0x7f599806de20 2024-04-25T16:53:26.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f5995d9b640 1 -- 172.21.15.94:0/3888426701 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59900a5a70 con 0x7f599806de20 2024-04-25T16:53:26.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f5995d9b640 1 -- 172.21.15.94:0/3888426701 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59900a5e40 con 0x7f599806de20 2024-04-25T16:53:26.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f599d156640 1 -- 172.21.15.94:0/3888426701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f599806de20 msgr2=0x7f599806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:26.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f599d156640 1 --2- 172.21.15.94:0/3888426701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f599806de20 0x7f599806e7a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5990009e50 tx=0x7f599001ca00 comp rx=0 tx=0).stop 2024-04-25T16:53:26.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f599d156640 1 -- 172.21.15.94:0/3888426701 shutdown_connections 2024-04-25T16:53:26.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f599d156640 1 --2- 172.21.15.94:0/3888426701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f599806de20 0x7f599806e7a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f599d156640 1 --2- 172.21.15.94:0/3888426701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f599815c7e0 0x7f59980afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f599d156640 1 -- 172.21.15.94:0/3888426701 >> 172.21.15.94:0/3888426701 conn(0x7f59980b47f0 msgr2=0x7f59980b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:26.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f599d156640 1 -- 172.21.15.94:0/3888426701 shutdown_connections 2024-04-25T16:53:26.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.274+0000 7f599d156640 1 -- 172.21.15.94:0/3888426701 wait complete. 2024-04-25T16:53:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f599d156640 1 Processor -- start 2024-04-25T16:53:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f599d156640 1 -- start start 2024-04-25T16:53:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f599d156640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59980b0330 0x7f599809a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f599d156640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f599815c7e0 0x7f599814e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f599d156640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f599814d0c0 0x7f599814e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f599d156640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59981695d0 con 0x7f59980b0330 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f599d156640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f599815cbf0 con 0x7f599815c7e0 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f599d156640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5998169750 con 0x7f599814d0c0 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f5996d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59980b0330 0x7f599809a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f5996d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59980b0330 0x7f599809a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52772/0 (socket says 172.21.15.94:52772) 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f5996d9d640 1 -- 172.21.15.94:0/4142961078 learned_addr learned my addr 172.21.15.94:0/4142961078 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f599759e640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f599814d0c0 0x7f599814e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f599659c640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f599815c7e0 0x7f599814e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f5996d9d640 1 -- 172.21.15.94:0/4142961078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f599814d0c0 msgr2=0x7f599814e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f5996d9d640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f599814d0c0 0x7f599814e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f5996d9d640 1 -- 172.21.15.94:0/4142961078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f599815c7e0 msgr2=0x7f599814e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f5996d9d640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f599815c7e0 0x7f599814e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f5996d9d640 1 -- 172.21.15.94:0/4142961078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599814d940 con 0x7f59980b0330 2024-04-25T16:53:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f599659c640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f599815c7e0 0x7f599814e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:53:26.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f5996d9d640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59980b0330 0x7f599809a920 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f598800e8a0 tx=0x7f598800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:26.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f597f7fe640 1 -- 172.21.15.94:0/4142961078 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5988017070 con 0x7f59980b0330 2024-04-25T16:53:26.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f597f7fe640 1 -- 172.21.15.94:0/4142961078 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f598800c940 con 0x7f59980b0330 2024-04-25T16:53:26.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f599d156640 1 -- 172.21.15.94:0/4142961078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599809d4d0 con 0x7f59980b0330 2024-04-25T16:53:26.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f597f7fe640 1 -- 172.21.15.94:0/4142961078 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598800ccf0 con 0x7f59980b0330 2024-04-25T16:53:26.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.278+0000 7f599d156640 1 -- 172.21.15.94:0/4142961078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f599809da50 con 0x7f59980b0330 2024-04-25T16:53:26.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.282+0000 7f599d156640 1 -- 172.21.15.94:0/4142961078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5998076830 con 0x7f59980b0330 2024-04-25T16:53:26.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.282+0000 7f597f7fe640 1 -- 172.21.15.94:0/4142961078 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f598801a070 con 0x7f59980b0330 2024-04-25T16:53:26.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.282+0000 7f597f7fe640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f598406f430 0x7f5984071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:26.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.282+0000 7f597f7fe640 1 -- 172.21.15.94:0/4142961078 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f598808fac0 con 0x7f59980b0330 2024-04-25T16:53:26.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.282+0000 7f599659c640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f598406f430 0x7f5984071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:26.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.282+0000 7f599659c640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f598406f430 0x7f5984071940 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f599813f6c0 tx=0x7f598c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:26.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.286+0000 7f597f7fe640 1 -- 172.21.15.94:0/4142961078 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f598805c8e0 con 0x7f59980b0330 2024-04-25T16:53:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f599d156640 1 -- 172.21.15.94:0/4142961078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) -- 0x7f59980b0e00 con 0x7f59980b0330 2024-04-25T16:53:26.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f597f7fe640 1 -- 172.21.15.94:0/4142961078 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v11) ==== 82+0+151 (secure 0 0 0) 0x7f598805c1e0 con 0x7f59980b0330 2024-04-25T16:53:26.504 INFO:teuthology.orchestra.run.smithi094.stdout:[client.admin] 2024-04-25T16:53:26.504 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCpiSpmAupiMxAAurdin/1JEoAui1v0rpMSPA== 2024-04-25T16:53:26.504 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow *" 2024-04-25T16:53:26.504 INFO:teuthology.orchestra.run.smithi094.stdout: caps mgr = "allow *" 2024-04-25T16:53:26.504 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow *" 2024-04-25T16:53:26.504 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow *" 2024-04-25T16:53:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f599d156640 1 -- 172.21.15.94:0/4142961078 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f598406f430 msgr2=0x7f5984071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f599d156640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f598406f430 0x7f5984071940 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f599813f6c0 tx=0x7f598c009040 comp rx=0 tx=0).stop 2024-04-25T16:53:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f599d156640 1 -- 172.21.15.94:0/4142961078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59980b0330 msgr2=0x7f599809a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f599d156640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59980b0330 0x7f599809a920 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f598800e8a0 tx=0x7f598800eca0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f599d156640 1 -- 172.21.15.94:0/4142961078 shutdown_connections 2024-04-25T16:53:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f599d156640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f598406f430 0x7f5984071940 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f599d156640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f599814d0c0 0x7f599814e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f599d156640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f599815c7e0 0x7f599814e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f599d156640 1 --2- 172.21.15.94:0/4142961078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59980b0330 0x7f599809a920 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f599d156640 1 -- 172.21.15.94:0/4142961078 >> 172.21.15.94:0/4142961078 conn(0x7f59980b47f0 msgr2=0x7f599806f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f599d156640 1 -- 172.21.15.94:0/4142961078 shutdown_connections 2024-04-25T16:53:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T16:53:26.506+0000 7f599d156640 1 -- 172.21.15.94:0/4142961078 wait complete. 2024-04-25T16:53:26.527 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-04-25T16:53:26.532 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.ELaIAXU9jw 2024-04-25T16:53:26.532 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:53:26.532 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/tmp/tmp.ELaIAXU9jw 2024-04-25T16:53:26.580 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:53:26.580 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/tmp/tmp.FooT1Le6w4xfstests-dev/local.config 2024-04-25T16:53:26.636 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.PS50Hvgg8stest export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.cwl13YwnfEscratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.94:6789 -k /tmp/tmp.ELaIAXU9jw --client_mountpoint /test" 2024-04-25T16:53:26.636 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T16:53:26.637 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/tmp/tmp.FooT1Le6w4xfstests-dev/ceph.exclude 2024-04-25T16:53:26.693 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo make) 2024-04-25T16:53:26.752 INFO:teuthology.orchestra.run.smithi094.stdout:libtoolize -cfi 2024-04-25T16:53:27.029 INFO:teuthology.orchestra.run.smithi094.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-25T16:53:27.030 INFO:teuthology.orchestra.run.smithi094.stdout:libtoolize: copying file './config.guess' 2024-04-25T16:53:27.035 INFO:teuthology.orchestra.run.smithi094.stdout:libtoolize: copying file './config.sub' 2024-04-25T16:53:27.047 INFO:teuthology.orchestra.run.smithi094.stdout:libtoolize: copying file './install-sh' 2024-04-25T16:53:27.054 INFO:teuthology.orchestra.run.smithi094.stdout:libtoolize: copying file './ltmain.sh' 2024-04-25T16:53:27.074 INFO:teuthology.orchestra.run.smithi094.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-25T16:53:27.074 INFO:teuthology.orchestra.run.smithi094.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-25T16:53:27.088 INFO:teuthology.orchestra.run.smithi094.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-25T16:53:27.093 INFO:teuthology.orchestra.run.smithi094.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-25T16:53:27.099 INFO:teuthology.orchestra.run.smithi094.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-25T16:53:27.104 INFO:teuthology.orchestra.run.smithi094.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-25T16:53:27.109 INFO:teuthology.orchestra.run.smithi094.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-25T16:53:27.110 INFO:teuthology.orchestra.run.smithi094.stdout:cp include/install-sh . 2024-04-25T16:53:27.111 INFO:teuthology.orchestra.run.smithi094.stdout:aclocal -I m4 2024-04-25T16:53:28.195 INFO:teuthology.orchestra.run.smithi094.stdout:autoheader 2024-04-25T16:53:28.562 INFO:teuthology.orchestra.run.smithi094.stdout:autoconf 2024-04-25T16:53:28.962 INFO:teuthology.orchestra.run.smithi094.stdout:./configure \ 2024-04-25T16:53:28.962 INFO:teuthology.orchestra.run.smithi094.stdout: --libexecdir=/usr/lib \ 2024-04-25T16:53:28.962 INFO:teuthology.orchestra.run.smithi094.stdout: --exec_prefix=/var/lib 2024-04-25T16:53:29.405 INFO:teuthology.orchestra.run.smithi094.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-25T16:53:29.405 INFO:teuthology.orchestra.run.smithi094.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-25T16:53:29.442 INFO:teuthology.orchestra.run.smithi094.stdout:checking how to print strings... printf 2024-04-25T16:53:29.443 INFO:teuthology.orchestra.run.smithi094.stdout:checking for gcc... gcc 2024-04-25T16:53:29.648 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether the C compiler works... yes 2024-04-25T16:53:29.648 INFO:teuthology.orchestra.run.smithi094.stdout:checking for C compiler default output file name... a.out 2024-04-25T16:53:29.686 INFO:teuthology.orchestra.run.smithi094.stdout:checking for suffix of executables... 2024-04-25T16:53:29.777 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether we are cross compiling... no 2024-04-25T16:53:29.795 INFO:teuthology.orchestra.run.smithi094.stdout:checking for suffix of object files... o 2024-04-25T16:53:29.813 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether the compiler supports GNU C... yes 2024-04-25T16:53:29.829 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether gcc accepts -g... yes 2024-04-25T16:53:29.954 INFO:teuthology.orchestra.run.smithi094.stdout:checking for gcc option to enable C11 features... none needed 2024-04-25T16:53:29.959 INFO:teuthology.orchestra.run.smithi094.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-25T16:53:29.962 INFO:teuthology.orchestra.run.smithi094.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-25T16:53:29.964 INFO:teuthology.orchestra.run.smithi094.stdout:checking for egrep... /usr/bin/grep -E 2024-04-25T16:53:29.966 INFO:teuthology.orchestra.run.smithi094.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-25T16:53:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-25T16:53:29.974 INFO:teuthology.orchestra.run.smithi094.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-25T16:53:29.992 INFO:teuthology.orchestra.run.smithi094.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-25T16:53:30.016 INFO:teuthology.orchestra.run.smithi094.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-25T16:53:30.016 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether ln -s works... yes 2024-04-25T16:53:30.025 INFO:teuthology.orchestra.run.smithi094.stdout:checking the maximum length of command line arguments... 1572864 2024-04-25T16:53:30.028 INFO:teuthology.orchestra.run.smithi094.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-25T16:53:30.028 INFO:teuthology.orchestra.run.smithi094.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-25T16:53:30.028 INFO:teuthology.orchestra.run.smithi094.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-25T16:53:30.028 INFO:teuthology.orchestra.run.smithi094.stdout:checking for objdump... objdump 2024-04-25T16:53:30.030 INFO:teuthology.orchestra.run.smithi094.stdout:checking how to recognize dependent libraries... pass_all 2024-04-25T16:53:30.030 INFO:teuthology.orchestra.run.smithi094.stdout:checking for dlltool... no 2024-04-25T16:53:30.030 INFO:teuthology.orchestra.run.smithi094.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-25T16:53:30.031 INFO:teuthology.orchestra.run.smithi094.stdout:checking for ar... ar 2024-04-25T16:53:30.079 INFO:teuthology.orchestra.run.smithi094.stdout:checking for archiver @FILE support... @ 2024-04-25T16:53:30.079 INFO:teuthology.orchestra.run.smithi094.stdout:checking for strip... strip 2024-04-25T16:53:30.079 INFO:teuthology.orchestra.run.smithi094.stdout:checking for ranlib... ranlib 2024-04-25T16:53:30.080 INFO:teuthology.orchestra.run.smithi094.stdout:checking for gawk... gawk 2024-04-25T16:53:30.141 INFO:teuthology.orchestra.run.smithi094.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-25T16:53:30.142 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sysroot... no 2024-04-25T16:53:30.146 INFO:teuthology.orchestra.run.smithi094.stdout:checking for a working dd... /usr/bin/dd 2024-04-25T16:53:30.150 INFO:teuthology.orchestra.run.smithi094.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-25T16:53:30.162 INFO:teuthology.orchestra.run.smithi094.stdout:checking for mt... mt 2024-04-25T16:53:30.187 INFO:teuthology.orchestra.run.smithi094.stdout:checking if mt is a manifest tool... no 2024-04-25T16:53:30.208 INFO:teuthology.orchestra.run.smithi094.stdout:checking for stdio.h... yes 2024-04-25T16:53:30.278 INFO:teuthology.orchestra.run.smithi094.stdout:checking for stdlib.h... yes 2024-04-25T16:53:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:checking for string.h... yes 2024-04-25T16:53:30.341 INFO:teuthology.orchestra.run.smithi094.stdout:checking for inttypes.h... yes 2024-04-25T16:53:30.370 INFO:teuthology.orchestra.run.smithi094.stdout:checking for stdint.h... yes 2024-04-25T16:53:30.395 INFO:teuthology.orchestra.run.smithi094.stdout:checking for strings.h... yes 2024-04-25T16:53:30.422 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/stat.h... yes 2024-04-25T16:53:30.447 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/types.h... yes 2024-04-25T16:53:30.483 INFO:teuthology.orchestra.run.smithi094.stdout:checking for unistd.h... yes 2024-04-25T16:53:30.514 INFO:teuthology.orchestra.run.smithi094.stdout:checking for dlfcn.h... yes 2024-04-25T16:53:30.518 INFO:teuthology.orchestra.run.smithi094.stdout:checking for objdir... .libs 2024-04-25T16:53:30.594 INFO:teuthology.orchestra.run.smithi094.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-25T16:53:30.594 INFO:teuthology.orchestra.run.smithi094.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-25T16:53:30.610 INFO:teuthology.orchestra.run.smithi094.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-25T16:53:30.882 INFO:teuthology.orchestra.run.smithi094.stdout:checking if gcc static flag -static works... yes 2024-04-25T16:53:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:checking if gcc supports -c -o file.o... yes 2024-04-25T16:53:30.911 INFO:teuthology.orchestra.run.smithi094.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-25T16:53:30.920 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-25T16:53:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether -lc should be explicitly linked in... no 2024-04-25T16:53:31.037 INFO:teuthology.orchestra.run.smithi094.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-25T16:53:31.037 INFO:teuthology.orchestra.run.smithi094.stdout:checking how to hardcode library paths into programs... immediate 2024-04-25T16:53:31.047 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether stripping libraries is possible... yes 2024-04-25T16:53:31.047 INFO:teuthology.orchestra.run.smithi094.stdout:checking if libtool supports shared libraries... yes 2024-04-25T16:53:31.047 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether to build shared libraries... yes 2024-04-25T16:53:31.047 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether to build static libraries... yes 2024-04-25T16:53:31.056 INFO:teuthology.orchestra.run.smithi094.stdout:checking for gcc... (cached) gcc 2024-04-25T16:53:31.081 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether the compiler supports GNU C... (cached) yes 2024-04-25T16:53:31.081 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T16:53:31.081 INFO:teuthology.orchestra.run.smithi094.stdout:checking for gcc option to enable C11 features... (cached) none needed 2024-04-25T16:53:31.081 INFO:teuthology.orchestra.run.smithi094.stdout:checking for gmake... /usr/bin/gmake 2024-04-25T16:53:31.082 INFO:teuthology.orchestra.run.smithi094.stderr:./configure: line 12753: test: $(SHELL): binary operator expected 2024-04-25T16:53:31.082 INFO:teuthology.orchestra.run.smithi094.stdout:checking for tar... /bin/tar 2024-04-25T16:53:31.083 INFO:teuthology.orchestra.run.smithi094.stdout:checking for gzip... /bin/gzip 2024-04-25T16:53:31.094 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether gcc -MM is supported... yes 2024-04-25T16:53:31.094 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sort... /bin/sort 2024-04-25T16:53:31.094 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether ln -s works... yes 2024-04-25T16:53:31.095 INFO:teuthology.orchestra.run.smithi094.stdout:checking for rpm... no 2024-04-25T16:53:31.130 INFO:teuthology.orchestra.run.smithi094.stdout:checking for assert.h... yes 2024-04-25T16:53:31.162 INFO:teuthology.orchestra.run.smithi094.stdout:checking for bstring.h... no 2024-04-25T16:53:31.194 INFO:teuthology.orchestra.run.smithi094.stdout:checking for libgen.h... yes 2024-04-25T16:53:31.252 INFO:teuthology.orchestra.run.smithi094.stdout:checking for dirent.h... yes 2024-04-25T16:53:31.339 INFO:teuthology.orchestra.run.smithi094.stdout:checking for errno.h... yes 2024-04-25T16:53:31.379 INFO:teuthology.orchestra.run.smithi094.stdout:checking for malloc.h... yes 2024-04-25T16:53:31.415 INFO:teuthology.orchestra.run.smithi094.stdout:checking for uuid.h... no 2024-04-25T16:53:31.453 INFO:teuthology.orchestra.run.smithi094.stdout:checking for uuid/uuid.h... yes 2024-04-25T16:53:31.488 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/uuid.h... no 2024-04-25T16:53:31.542 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/file.h... yes 2024-04-25T16:53:31.574 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/fcntl.h... yes 2024-04-25T16:53:31.606 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/syssgi.h... no 2024-04-25T16:53:31.669 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/param.h... yes 2024-04-25T16:53:31.669 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/stat.h... (cached) yes 2024-04-25T16:53:31.701 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/statvfs.h... yes 2024-04-25T16:53:31.736 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/time.h... yes 2024-04-25T16:53:31.776 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/ioctl.h... yes 2024-04-25T16:53:31.813 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/wait.h... yes 2024-04-25T16:53:31.813 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/types.h... (cached) yes 2024-04-25T16:53:31.813 INFO:teuthology.orchestra.run.smithi094.stdout:checking for strings.h... (cached) yes 2024-04-25T16:53:31.845 INFO:teuthology.orchestra.run.smithi094.stdout:checking for err.h... yes 2024-04-25T16:53:31.875 INFO:teuthology.orchestra.run.smithi094.stdout:checking for linux/falloc.h... yes 2024-04-25T16:53:31.906 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-25T16:53:31.936 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-25T16:53:31.971 INFO:teuthology.orchestra.run.smithi094.stdout:checking for xfs/platform_defs.h... no 2024-04-25T16:53:32.009 INFO:teuthology.orchestra.run.smithi094.stdout:checking for btrfs/ioctl.h... no 2024-04-25T16:53:32.041 INFO:teuthology.orchestra.run.smithi094.stdout:checking for cifs/ioctl.h... no 2024-04-25T16:53:32.074 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/mman.h... yes 2024-04-25T16:53:32.106 INFO:teuthology.orchestra.run.smithi094.stdout:checking for linux/ext4.h... no 2024-04-25T16:53:32.122 INFO:teuthology.orchestra.run.smithi094.stdout:checking for xfs/xfs_log_format.h... no 2024-04-25T16:53:32.164 INFO:teuthology.orchestra.run.smithi094.stdout:checking for uuid_compare... no 2024-04-25T16:53:32.199 INFO:teuthology.orchestra.run.smithi094.stdout:checking for uuid_compare in -luuid... yes 2024-04-25T16:53:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:checking for xfs/xfs.h... yes 2024-04-25T16:53:32.318 INFO:teuthology.orchestra.run.smithi094.stdout:checking for xfs/libxfs.h... no 2024-04-25T16:53:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:checking for gcc options needed to detect all undeclared functions... none needed 2024-04-25T16:53:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-25T16:53:32.420 INFO:teuthology.orchestra.run.smithi094.stdout:checking for xfs/xqm.h... yes 2024-04-25T16:53:32.486 INFO:teuthology.orchestra.run.smithi094.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-25T16:53:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:checking for xfs/handle.h... yes 2024-04-25T16:53:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-25T16:53:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/acl.h... yes 2024-04-25T16:53:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:checking for acl/libacl.h... yes 2024-04-25T16:53:32.660 INFO:teuthology.orchestra.run.smithi094.stdout:checking for acl_init in -lacl... yes 2024-04-25T16:53:32.691 INFO:teuthology.orchestra.run.smithi094.stdout:checking for gdbm-ndbm.h... no 2024-04-25T16:53:32.730 INFO:teuthology.orchestra.run.smithi094.stdout:checking for gdbm/ndbm.h... no 2024-04-25T16:53:32.767 INFO:teuthology.orchestra.run.smithi094.stdout:checking for ndbm.h... no 2024-04-25T16:53:32.804 INFO:teuthology.orchestra.run.smithi094.stdout:checking for libaio.h... yes 2024-04-25T16:53:32.854 INFO:teuthology.orchestra.run.smithi094.stdout:checking for liburing.h... yes 2024-04-25T16:53:32.886 INFO:teuthology.orchestra.run.smithi094.stdout:checking for sys/capability.h... yes 2024-04-25T16:53:32.917 INFO:teuthology.orchestra.run.smithi094.stdout:checking for linux/fiemap.h... yes 2024-04-25T16:53:32.955 INFO:teuthology.orchestra.run.smithi094.stdout:checking for fallocate... yes 2024-04-25T16:53:32.992 INFO:teuthology.orchestra.run.smithi094.stdout:checking for open_by_handle_at... yes 2024-04-25T16:53:33.025 INFO:teuthology.orchestra.run.smithi094.stdout:checking for linux/fs.h... yes 2024-04-25T16:53:33.056 INFO:teuthology.orchestra.run.smithi094.stdout:checking for btrfsutil.h... no 2024-04-25T16:53:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:checking for copy_file_range... yes 2024-04-25T16:53:33.145 INFO:teuthology.orchestra.run.smithi094.stdout:checking for SEEK_DATA... yes 2024-04-25T16:53:33.206 INFO:teuthology.orchestra.run.smithi094.stdout:checking for BMV_OF_SHARED... yes 2024-04-25T16:53:33.246 INFO:teuthology.orchestra.run.smithi094.stdout:checking for nftw... yes 2024-04-25T16:53:33.283 INFO:teuthology.orchestra.run.smithi094.stdout:checking for RLIMIT_NOFILE... yes 2024-04-25T16:53:33.327 INFO:teuthology.orchestra.run.smithi094.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-25T16:53:33.362 INFO:teuthology.orchestra.run.smithi094.stdout:checking for FICLONE... yes 2024-04-25T16:53:33.400 INFO:teuthology.orchestra.run.smithi094.stdout:checking for renameat2... yes 2024-04-25T16:53:33.435 INFO:teuthology.orchestra.run.smithi094.stdout:checking for reallocarray... yes 2024-04-25T16:53:33.464 INFO:teuthology.orchestra.run.smithi094.stdout:checking for struct mount_attr... yes 2024-04-25T16:53:33.496 INFO:teuthology.orchestra.run.smithi094.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-25T16:53:33.527 INFO:teuthology.orchestra.run.smithi094.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-25T16:53:33.557 INFO:teuthology.orchestra.run.smithi094.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-25T16:53:33.590 INFO:teuthology.orchestra.run.smithi094.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-25T16:53:33.618 INFO:teuthology.orchestra.run.smithi094.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-25T16:53:33.649 INFO:teuthology.orchestra.run.smithi094.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-25T16:53:33.680 INFO:teuthology.orchestra.run.smithi094.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-25T16:53:33.710 INFO:teuthology.orchestra.run.smithi094.stdout:checking for linux/btrfs.h... yes 2024-04-25T16:53:33.740 INFO:teuthology.orchestra.run.smithi094.stdout:checking for linux/btrfs_tree.h... yes 2024-04-25T16:53:33.757 INFO:teuthology.orchestra.run.smithi094.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-25T16:53:33.786 INFO:teuthology.orchestra.run.smithi094.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-25T16:53:33.803 INFO:teuthology.orchestra.run.smithi094.stdout:configure: creating ./config.status 2024-04-25T16:53:34.135 INFO:teuthology.orchestra.run.smithi094.stdout:config.status: creating include/builddefs 2024-04-25T16:53:34.145 INFO:teuthology.orchestra.run.smithi094.stdout:config.status: creating include/config.h 2024-04-25T16:53:34.154 INFO:teuthology.orchestra.run.smithi094.stdout:config.status: executing libtool commands 2024-04-25T16:53:34.190 INFO:teuthology.orchestra.run.smithi094.stdout:Building include 2024-04-25T16:53:34.194 INFO:teuthology.orchestra.run.smithi094.stdout:Building lib 2024-04-25T16:53:34.196 INFO:teuthology.orchestra.run.smithi094.stdout: [LTDEP] 2024-04-25T16:53:34.332 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] dataascii.lo 2024-04-25T16:53:34.446 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] databin.lo 2024-04-25T16:53:34.592 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] datapid.lo 2024-04-25T16:53:34.692 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] file_lock.lo 2024-04-25T16:53:34.848 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] forker.lo 2024-04-25T16:53:34.991 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] pattern.lo 2024-04-25T16:53:35.097 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] open_flags.lo 2024-04-25T16:53:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] random_range.lo 2024-04-25T16:53:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] string_to_tokens.lo 2024-04-25T16:53:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] str_to_bytes.lo 2024-04-25T16:53:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] tlibio.lo 2024-04-25T16:53:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] write_log.lo 2024-04-25T16:53:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:write_log.c: In function 'wlog_record_write': 2024-04-25T16:53:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:write_log.c:205:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:36.028 INFO:teuthology.orchestra.run.smithi094.stderr: 205 | write(wfile->w_afd, wbuf, reclen); 2024-04-25T16:53:36.028 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:write_log.c:209:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:36.028 INFO:teuthology.orchestra.run.smithi094.stderr: 209 | write(wfile->w_rfd, wbuf, reclen); 2024-04-25T16:53:36.028 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] random.lo 2024-04-25T16:53:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: [LD] 2024-04-25T16:53:36.398 INFO:teuthology.orchestra.run.smithi094.stdout:Building ltp 2024-04-25T16:53:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: [DEP] 2024-04-25T16:53:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] doio 2024-04-25T16:53:37.295 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fsstress 2024-04-25T16:53:38.773 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fsx 2024-04-25T16:53:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] growfiles 2024-04-25T16:53:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] iogen 2024-04-25T16:53:40.192 INFO:teuthology.orchestra.run.smithi094.stderr:iogen.c: In function 'main': 2024-04-25T16:53:40.192 INFO:teuthology.orchestra.run.smithi094.stderr:iogen.c:280:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:40.192 INFO:teuthology.orchestra.run.smithi094.stderr: 280 | write(outfd, (char *)&req, sizeof(req)); 2024-04-25T16:53:40.192 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:40.194 INFO:teuthology.orchestra.run.smithi094.stderr:iogen.c: In function 'parse_cmdline': 2024-04-25T16:53:40.194 INFO:teuthology.orchestra.run.smithi094.stderr:iogen.c:1535:17: warning: ignoring return value of 'getcwd' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:40.195 INFO:teuthology.orchestra.run.smithi094.stderr: 1535 | getcwd(fptr->f_path, 2024-04-25T16:53:40.195 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:40.195 INFO:teuthology.orchestra.run.smithi094.stderr: 1536 | sizeof(fptr->f_path)-1); 2024-04-25T16:53:40.195 INFO:teuthology.orchestra.run.smithi094.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:40.486 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-stress 2024-04-25T16:53:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:aio-stress.c: In function 'setup_shared_mem': 2024-04-25T16:53:40.616 INFO:teuthology.orchestra.run.smithi094.stderr:aio-stress.c:1013:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:40.616 INFO:teuthology.orchestra.run.smithi094.stderr: 1013 | ftruncate(fd, total_ram); 2024-04-25T16:53:40.616 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:40.956 INFO:teuthology.orchestra.run.smithi094.stdout:Building src 2024-04-25T16:53:40.963 INFO:teuthology.orchestra.run.smithi094.stdout: [DEP] 2024-04-25T16:53:43.078 INFO:teuthology.orchestra.run.smithi094.stdout: [DEP] 2024-04-25T16:53:43.113 INFO:teuthology.orchestra.run.smithi094.stdout: [DEP] 2024-04-25T16:53:43.121 INFO:teuthology.orchestra.run.smithi094.stdout: [DEP] 2024-04-25T16:53:43.373 INFO:teuthology.orchestra.run.smithi094.stdout: [DEP] 2024-04-25T16:53:43.398 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] dirstress 2024-04-25T16:53:43.645 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fill 2024-04-25T16:53:43.789 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fill2 2024-04-25T16:53:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] getpagesize 2024-04-25T16:53:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] holes 2024-04-25T16:53:44.314 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] lstat64 2024-04-25T16:53:44.493 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] nametest 2024-04-25T16:53:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:nametest.c: In function 'main': 2024-04-25T16:53:44.609 INFO:teuthology.orchestra.run.smithi094.stderr:nametest.c:236:33: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:44.609 INFO:teuthology.orchestra.run.smithi094.stderr: 236 | write(1, "\n", 1); 2024-04-25T16:53:44.609 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T16:53:44.609 INFO:teuthology.orchestra.run.smithi094.stderr:nametest.c:238:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:44.609 INFO:teuthology.orchestra.run.smithi094.stderr: 238 | write(1, ".", 1); 2024-04-25T16:53:44.609 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T16:53:44.609 INFO:teuthology.orchestra.run.smithi094.stderr:nametest.c:288:33: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:44.609 INFO:teuthology.orchestra.run.smithi094.stderr: 288 | write(1, ".", 1); 2024-04-25T16:53:44.610 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T16:53:44.708 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:53:44.711 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:53:44.726 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] permname 2024-04-25T16:53:44.915 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] randholes 2024-04-25T16:53:45.175 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] runas 2024-04-25T16:53:45.356 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] truncfile 2024-04-25T16:53:45.538 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] usemem 2024-04-25T16:53:45.689 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] mmapcat 2024-04-25T16:53:45.798 INFO:teuthology.orchestra.run.smithi094.stderr:mmapcat.c: In function 'main': 2024-04-25T16:53:45.798 INFO:teuthology.orchestra.run.smithi094.stderr:mmapcat.c:29:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:45.798 INFO:teuthology.orchestra.run.smithi094.stderr: 29 | write(1,ptr2,ptr - ptr2); 2024-04-25T16:53:45.798 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:45.833 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] append_reader 2024-04-25T16:53:45.972 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] append_writer 2024-04-25T16:53:46.107 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] dirperf 2024-04-25T16:53:46.340 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] metaperf 2024-04-25T16:53:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:metaperf.c: In function 'crfiles': 2024-04-25T16:53:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:metaperf.c:166:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:46.454 INFO:teuthology.orchestra.run.smithi094.stderr: 166 | write(fd, buf, fsize); 2024-04-25T16:53:46.454 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:metaperf.c: In function 't_linkun': 2024-04-25T16:53:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:metaperf.c:454:25: warning: ignoring return value of 'link' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:46.455 INFO:teuthology.orchestra.run.smithi094.stderr: 454 | link("a", *fnp); 2024-04-25T16:53:46.455 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~ 2024-04-25T16:53:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:metaperf.c: In function 't_chown': 2024-04-25T16:53:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:metaperf.c:404:33: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:46.455 INFO:teuthology.orchestra.run.smithi094.stderr: 404 | chown(*fnp, 2, -1); 2024-04-25T16:53:46.455 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:metaperf.c:406:33: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:46.455 INFO:teuthology.orchestra.run.smithi094.stderr: 406 | chown(*fnp, 1, -1); 2024-04-25T16:53:46.455 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:metaperf.c: In function 'i_chown': 2024-04-25T16:53:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:metaperf.c:291:17: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:46.456 INFO:teuthology.orchestra.run.smithi094.stderr: 291 | chown(*fnp, 1, -1); 2024-04-25T16:53:46.456 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:46.456 INFO:teuthology.orchestra.run.smithi094.stderr:metaperf.c: In function 'main': 2024-04-25T16:53:46.456 INFO:teuthology.orchestra.run.smithi094.stderr:metaperf.c:152:9: warning: ignoring return value of 'chdir' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:46.456 INFO:teuthology.orchestra.run.smithi094.stderr: 152 | chdir(".."); 2024-04-25T16:53:46.456 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~ 2024-04-25T16:53:46.632 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] devzero 2024-04-25T16:53:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] feature 2024-04-25T16:53:47.024 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] alloc 2024-04-25T16:53:47.154 INFO:teuthology.orchestra.run.smithi094.stderr:alloc.c: In function 'main': 2024-04-25T16:53:47.154 INFO:teuthology.orchestra.run.smithi094.stderr:alloc.c:247:17: warning: ignoring return value of 'mkstemp' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:47.154 INFO:teuthology.orchestra.run.smithi094.stderr: 247 | mkstemp(tmpfile); 2024-04-25T16:53:47.154 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T16:53:47.248 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fault 2024-04-25T16:53:47.408 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fstest 2024-04-25T16:53:47.639 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_access_root 2024-04-25T16:53:47.742 INFO:teuthology.orchestra.run.smithi094.stderr:t_access_root.c: In function 'main': 2024-04-25T16:53:47.743 INFO:teuthology.orchestra.run.smithi094.stderr:t_access_root.c:55:5: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:47.743 INFO:teuthology.orchestra.run.smithi094.stderr: 55 | system(cmd); 2024-04-25T16:53:47.743 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~ 2024-04-25T16:53:47.785 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] godown 2024-04-25T16:53:47.936 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] resvtest 2024-04-25T16:53:48.117 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] writemod 2024-04-25T16:53:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] writev_on_pagefault 2024-04-25T16:53:48.401 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] makeextents 2024-04-25T16:53:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] itrash 2024-04-25T16:53:48.719 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] rename 2024-04-25T16:53:48.851 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] multi_open_unlink 2024-04-25T16:53:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] unwritten_sync 2024-04-25T16:53:49.185 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] genhashnames 2024-04-25T16:53:49.370 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_holes 2024-04-25T16:53:49.516 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_mmap_writev 2024-04-25T16:53:49.647 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_truncate_cmtime 2024-04-25T16:53:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] dirhash_collide 2024-04-25T16:53:49.977 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_rename_overwrite 2024-04-25T16:53:50.105 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] holetest 2024-04-25T16:53:50.212 INFO:teuthology.orchestra.run.smithi094.stderr:holetest.c: In function 'pt_write_marker': 2024-04-25T16:53:50.212 INFO:teuthology.orchestra.run.smithi094.stderr:holetest.c:140:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:50.212 INFO:teuthology.orchestra.run.smithi094.stderr: 140 | pwrite(fd, &tid, sizeof(tid), i * page_size + pgoff); 2024-04-25T16:53:50.212 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:50.329 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_truncate_self 2024-04-25T16:53:50.472 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] af_unix 2024-04-25T16:53:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_mmap_stale_pmd 2024-04-25T16:53:50.720 INFO:teuthology.orchestra.run.smithi094.stderr:t_mmap_stale_pmd.c: In function 'main': 2024-04-25T16:53:50.720 INFO:teuthology.orchestra.run.smithi094.stderr:t_mmap_stale_pmd.c:42:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:50.720 INFO:teuthology.orchestra.run.smithi094.stderr: 42 | ftruncate(fd, 0); 2024-04-25T16:53:50.720 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T16:53:50.720 INFO:teuthology.orchestra.run.smithi094.stderr:t_mmap_stale_pmd.c:43:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:50.720 INFO:teuthology.orchestra.run.smithi094.stderr: 43 | ftruncate(fd, MiB(4)); 2024-04-25T16:53:50.721 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:50.721 INFO:teuthology.orchestra.run.smithi094.stderr:t_mmap_stale_pmd.c:55:9: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:50.721 INFO:teuthology.orchestra.run.smithi094.stderr: 55 | pwrite(fd, buffer, strlen(buffer), MiB(2)); 2024-04-25T16:53:50.721 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:50.759 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_mmap_cow_race 2024-04-25T16:53:50.865 INFO:teuthology.orchestra.run.smithi094.stderr:t_mmap_cow_race.c: In function 'main': 2024-04-25T16:53:50.866 INFO:teuthology.orchestra.run.smithi094.stderr:t_mmap_cow_race.c:61:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:50.866 INFO:teuthology.orchestra.run.smithi094.stderr: 61 | ftruncate(fd, 0); 2024-04-25T16:53:50.866 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T16:53:50.866 INFO:teuthology.orchestra.run.smithi094.stderr:t_mmap_cow_race.c:62:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:50.866 INFO:teuthology.orchestra.run.smithi094.stderr: 62 | ftruncate(fd, MiB(2)); 2024-04-25T16:53:50.866 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:50.916 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_mmap_fallocate 2024-04-25T16:53:51.055 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fsync-err 2024-04-25T16:53:51.230 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_mmap_write_ro 2024-04-25T16:53:51.376 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-25T16:53:51.475 INFO:teuthology.orchestra.run.smithi094.stderr:t_ext4_dax_journal_corruption.c: In function 'main': 2024-04-25T16:53:51.475 INFO:teuthology.orchestra.run.smithi094.stderr:t_ext4_dax_journal_corruption.c:61:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:51.476 INFO:teuthology.orchestra.run.smithi094.stderr: 61 | ftruncate(fd, 0); 2024-04-25T16:53:51.476 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T16:53:51.519 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-25T16:53:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_ofd_locks 2024-04-25T16:53:51.851 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_mmap_collision 2024-04-25T16:53:52.061 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] mmap-write-concurrent 2024-04-25T16:53:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_get_file_time 2024-04-25T16:53:52.364 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_create_short_dirs 2024-04-25T16:53:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_create_long_dirs 2024-04-25T16:53:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_enospc 2024-04-25T16:53:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_mmap_writev_overlap 2024-04-25T16:53:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] checkpoint_journal 2024-04-25T16:53:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] mmap-rw-fault 2024-04-25T16:53:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] allocstale 2024-04-25T16:53:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_mmap_cow_memory_failure 2024-04-25T16:53:53.764 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fake-dump-rootino 2024-04-25T16:53:53.907 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] dio-buf-fault 2024-04-25T16:53:54.058 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] rewinddir-test 2024-04-25T16:53:54.230 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] readdir-while-renames 2024-04-25T16:53:54.392 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] xfsctl 2024-04-25T16:53:54.579 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] bstat 2024-04-25T16:53:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_mtab 2024-04-25T16:53:54.909 INFO:teuthology.orchestra.run.smithi094.stderr:t_mtab.c: In function 'update_mtab': 2024-04-25T16:53:54.909 INFO:teuthology.orchestra.run.smithi094.stderr:t_mtab.c:256:17: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:54.909 INFO:teuthology.orchestra.run.smithi094.stderr: 256 | chown (mounted_temp, sbuf.st_uid, sbuf.st_gid); 2024-04-25T16:53:54.910 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:54.986 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] getdevicesize 2024-04-25T16:53:55.119 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] preallo_rw_pattern_reader 2024-04-25T16:53:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:In file included from preallo_rw_pattern_reader.c:2: 2024-04-25T16:53:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:iopat.c: In function 'main': 2024-04-25T16:53:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:iopat.c:38:4: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:55.239 INFO:teuthology.orchestra.run.smithi094.stderr: 38 | read(fd, &x, 1048576); 2024-04-25T16:53:55.240 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:55.273 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] preallo_rw_pattern_writer 2024-04-25T16:53:55.381 INFO:teuthology.orchestra.run.smithi094.stderr:In file included from preallo_rw_pattern_writer.c:2: 2024-04-25T16:53:55.381 INFO:teuthology.orchestra.run.smithi094.stderr:iopat.c: In function 'main': 2024-04-25T16:53:55.381 INFO:teuthology.orchestra.run.smithi094.stderr:iopat.c:34:4: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:55.381 INFO:teuthology.orchestra.run.smithi094.stderr: 34 | write(fd, &x, 1048576); 2024-04-25T16:53:55.382 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:55.414 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] ftrunc 2024-04-25T16:53:55.548 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] trunc 2024-04-25T16:53:55.706 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fs_perms 2024-04-25T16:53:55.817 INFO:teuthology.orchestra.run.smithi094.stderr:fs_perms.c: In function 'main': 2024-04-25T16:53:55.817 INFO:teuthology.orchestra.run.smithi094.stderr:fs_perms.c:54:4: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:55.817 INFO:teuthology.orchestra.run.smithi094.stderr: 54 | system("rm -f test.file"); 2024-04-25T16:53:55.817 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:55.817 INFO:teuthology.orchestra.run.smithi094.stderr:fs_perms.c: In function 'testsetup': 2024-04-25T16:53:55.817 INFO:teuthology.orchestra.run.smithi094.stderr:fs_perms.c:67:4: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:55.817 INFO:teuthology.orchestra.run.smithi094.stderr: 67 | system("cp testx.file test.file"); 2024-04-25T16:53:55.818 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:55.818 INFO:teuthology.orchestra.run.smithi094.stderr:fs_perms.c:69:4: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:55.818 INFO:teuthology.orchestra.run.smithi094.stderr: 69 | chown("test.file",cuserId,cgroupId); 2024-04-25T16:53:55.818 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:55.818 INFO:teuthology.orchestra.run.smithi094.stderr:fs_perms.c: In function 'testfperm': 2024-04-25T16:53:55.818 INFO:teuthology.orchestra.run.smithi094.stderr:fs_perms.c:123:5: warning: ignoring return value of 'seteuid' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:55.818 INFO:teuthology.orchestra.run.smithi094.stderr: 123 | seteuid(0); 2024-04-25T16:53:55.818 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~ 2024-04-25T16:53:55.818 INFO:teuthology.orchestra.run.smithi094.stderr:fs_perms.c:124:5: warning: ignoring return value of 'setegid' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:55.818 INFO:teuthology.orchestra.run.smithi094.stderr: 124 | setegid(0); 2024-04-25T16:53:55.818 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~ 2024-04-25T16:53:55.867 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] testx 2024-04-25T16:53:55.996 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] looptest 2024-04-25T16:53:56.196 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] locktest 2024-04-25T16:53:56.523 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] unwritten_mmap 2024-04-25T16:53:56.680 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] bulkstat_unlink_test 2024-04-25T16:53:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:bulkstat_unlink_test.c: In function 'main': 2024-04-25T16:53:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:bulkstat_unlink_test.c:106:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:56.801 INFO:teuthology.orchestra.run.smithi094.stderr: 106 | write(fd[i], fname, sizeof(fname)); 2024-04-25T16:53:56.801 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:bulkstat_unlink_test.c:178:17: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:56.801 INFO:teuthology.orchestra.run.smithi094.stderr: 178 | system(fname); 2024-04-25T16:53:56.801 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~ 2024-04-25T16:53:56.888 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] deduperace 2024-04-25T16:53:57.086 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] bulkstat_unlink_test_modified 2024-04-25T16:53:57.203 INFO:teuthology.orchestra.run.smithi094.stderr:bulkstat_unlink_test_modified.c: In function 'main': 2024-04-25T16:53:57.203 INFO:teuthology.orchestra.run.smithi094.stderr:bulkstat_unlink_test_modified.c:83:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.203 INFO:teuthology.orchestra.run.smithi094.stderr: 83 | write(fd, fname, sizeof(fname)); 2024-04-25T16:53:57.203 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.204 INFO:teuthology.orchestra.run.smithi094.stderr:bulkstat_unlink_test_modified.c:139:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.204 INFO:teuthology.orchestra.run.smithi094.stderr: 139 | write(fd, fname, sizeof(fname)); 2024-04-25T16:53:57.204 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.204 INFO:teuthology.orchestra.run.smithi094.stderr:bulkstat_unlink_test_modified.c:185:9: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.204 INFO:teuthology.orchestra.run.smithi094.stderr: 185 | system(fname); 2024-04-25T16:53:57.204 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~ 2024-04-25T16:53:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_dir_offset 2024-04-25T16:53:57.419 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_futimens 2024-04-25T16:53:57.542 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_immutable 2024-04-25T16:53:57.679 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c: In function 'create_test_area': 2024-04-25T16:53:57.679 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:2061:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.679 INFO:teuthology.orchestra.run.smithi094.stderr: 2061 | asprintf(&path, "%s/immutable.d", dir); 2024-04-25T16:53:57.679 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.679 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:2096:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.679 INFO:teuthology.orchestra.run.smithi094.stderr: 2096 | asprintf(&path, "%s/append-only.d", dir); 2024-04-25T16:53:57.680 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.680 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c: In function 'remove_test_area': 2024-04-25T16:53:57.680 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:2152:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.680 INFO:teuthology.orchestra.run.smithi094.stderr: 2152 | asprintf(&path, "%s/immutable.d", dir); 2024-04-25T16:53:57.680 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.680 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:2163:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.680 INFO:teuthology.orchestra.run.smithi094.stderr: 2163 | asprintf(&path, "%s/empty-immutable.d", dir); 2024-04-25T16:53:57.681 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.681 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:2175:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.682 INFO:teuthology.orchestra.run.smithi094.stderr: 2175 | asprintf(&path, "%s/append-only.d", dir); 2024-04-25T16:53:57.682 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.682 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:2186:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.682 INFO:teuthology.orchestra.run.smithi094.stderr: 2186 | asprintf(&path, "%s/empty-append-only.d", dir); 2024-04-25T16:53:57.682 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.682 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:2197:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.682 INFO:teuthology.orchestra.run.smithi094.stderr: 2197 | asprintf(&path, "%s/append-only.f", dir); 2024-04-25T16:53:57.682 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.682 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:2209:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.682 INFO:teuthology.orchestra.run.smithi094.stderr: 2209 | asprintf(&path, "%s/immutable.f", dir); 2024-04-25T16:53:57.682 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.682 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c: In function 'check_test_area': 2024-04-25T16:53:57.682 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1788:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.683 INFO:teuthology.orchestra.run.smithi094.stderr: 1788 | asprintf(&path, "%s/", dir); 2024-04-25T16:53:57.683 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.683 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1800:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.683 INFO:teuthology.orchestra.run.smithi094.stderr: 1800 | asprintf(&path, "%s/immutable.f", dir); 2024-04-25T16:53:57.683 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1812:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.686 INFO:teuthology.orchestra.run.smithi094.stderr: 1812 | asprintf(&path, "%s/append-only.f", dir); 2024-04-25T16:53:57.686 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1824:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.686 INFO:teuthology.orchestra.run.smithi094.stderr: 1824 | asprintf(&path, "%s/immutable.d", dir); 2024-04-25T16:53:57.686 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1837:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr: 1837 | asprintf(&path, "%s/append-only.d", dir); 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1850:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr: 1850 | asprintf(&path, "%s/immutable.d/file", dir); 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1862:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr: 1862 | asprintf(&path, "%s/append-only.d/file", dir); 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1874:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr: 1874 | asprintf(&path, "%s/immutable.d/dir", dir); 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1887:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.687 INFO:teuthology.orchestra.run.smithi094.stderr: 1887 | asprintf(&path, "%s/append-only.d/dir", dir); 2024-04-25T16:53:57.688 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.688 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c: In function 'test_immutable': 2024-04-25T16:53:57.688 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:151:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.688 INFO:teuthology.orchestra.run.smithi094.stderr: 151 | asprintf(&path, "%s/immutable.f", dir); 2024-04-25T16:53:57.688 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.688 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:381:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.688 INFO:teuthology.orchestra.run.smithi094.stderr: 381 | asprintf(&linkpath, "%s/immutable.f.hardlink", dir); 2024-04-25T16:53:57.688 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.688 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:408:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.688 INFO:teuthology.orchestra.run.smithi094.stderr: 408 | chown(path, 0, 0); 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:485:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr: 485 | asprintf(&linkpath, "%s/immutable.f.newname", dir); 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:532:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr: 532 | asprintf(&path, "%s/immutable.d/file", dir); 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:553:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr: 553 | chown(path, 0, 0); 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:556:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr: 556 | asprintf(&linkpath, "%s/immutable.d/file.link", dir); 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:575:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.689 INFO:teuthology.orchestra.run.smithi094.stderr: 575 | asprintf(&linkpath, "%s/immutable.d/file.newname", dir); 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:595:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr: 595 | asprintf(&path, "%s/immutable.d/newfile", dir); 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:619:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr: 619 | asprintf(&path, "%s/immutable.d/newdir", dir); 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:631:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr: 631 | asprintf(&path, "%s/immutable.d/dir/newfile-%d", dir, getuid()); 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:653:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr: 653 | chown(path, 0, 0); 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.690 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:657:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.691 INFO:teuthology.orchestra.run.smithi094.stderr: 657 | asprintf(&path, "%s/immutable.d/dir", dir); 2024-04-25T16:53:57.691 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.691 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:668:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.691 INFO:teuthology.orchestra.run.smithi094.stderr: 668 | asprintf(&path, "%s/immutable.d", dir); 2024-04-25T16:53:57.691 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.691 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:705:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.691 INFO:teuthology.orchestra.run.smithi094.stderr: 705 | chown(path, 0, 0); 2024-04-25T16:53:57.691 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.691 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:782:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.691 INFO:teuthology.orchestra.run.smithi094.stderr: 782 | asprintf(&path, "%s/empty-immutable.d", dir); 2024-04-25T16:53:57.691 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.694 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c: In function 'test_append': 2024-04-25T16:53:57.694 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:822:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.694 INFO:teuthology.orchestra.run.smithi094.stderr: 822 | asprintf(&path, "%s/append-only.f", dir); 2024-04-25T16:53:57.694 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.694 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1024:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.694 INFO:teuthology.orchestra.run.smithi094.stderr: 1024 | asprintf(&linkpath, "%s/append-only.f.hardlink", dir); 2024-04-25T16:53:57.694 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.694 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1051:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.694 INFO:teuthology.orchestra.run.smithi094.stderr: 1051 | chown(path, 0, 0); 2024-04-25T16:53:57.694 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.694 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1126:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.694 INFO:teuthology.orchestra.run.smithi094.stderr: 1126 | asprintf(&linkpath, "%s/append-only.f.newname", dir); 2024-04-25T16:53:57.694 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1495:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr: 1495 | asprintf(&path, "%s/append-only.d/file", dir); 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1516:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr: 1516 | chown(path, 0, 0); 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1519:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr: 1519 | asprintf(&linkpath, "%s/append-only.d/file.link-%d", dir, getuid()); 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1529:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr: 1529 | asprintf(&linkpath, "%s/append-only.d/file.symlink-%d", dir, getuid()); 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1539:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr: 1539 | asprintf(&linkpath, "%s/append-only.d/file.newname", dir); 2024-04-25T16:53:57.695 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1559:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr: 1559 | asprintf(&path, "%s/append-only.d/newfile-%d", dir, getuid()); 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1573:11: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr: 1573 | asprintf(&path, "%s/append-only.d/newdev-%d", dir, getuid()); 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1586:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr: 1586 | asprintf(&path, "%s/append-only.d/newdir-%d", dir, getuid()); 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1596:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr: 1596 | asprintf(&path, "%s/append-only.d/newdir-%d/newfile-%d", dir, getuid(), getuid()); 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1618:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.696 INFO:teuthology.orchestra.run.smithi094.stderr: 1618 | chown(path, 0, 0); 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1622:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr: 1622 | asprintf(&path, "%s/append-only.d/dir/newfile-%d", dir, getuid()); 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1644:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr: 1644 | chown(path, 0, 0); 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1648:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr: 1648 | asprintf(&path, "%s/append-only.d/dir", dir); 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1659:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr: 1659 | asprintf(&path, "%s/append-only.d", dir); 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1692:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.697 INFO:teuthology.orchestra.run.smithi094.stderr: 1692 | chown(path, 0, 0); 2024-04-25T16:53:57.698 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T16:53:57.698 INFO:teuthology.orchestra.run.smithi094.stderr:t_immutable.c:1769:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:57.698 INFO:teuthology.orchestra.run.smithi094.stderr: 1769 | asprintf(&path, "%s/empty-append-only.d", dir); 2024-04-25T16:53:57.698 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] stale_handle 2024-04-25T16:53:58.385 INFO:teuthology.orchestra.run.smithi094.stderr:stale_handle.c: In function 'main': 2024-04-25T16:53:58.385 INFO:teuthology.orchestra.run.smithi094.stderr:stale_handle.c:102:9: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:58.385 INFO:teuthology.orchestra.run.smithi094.stderr: 102 | system("echo 3 > /proc/sys/vm/drop_caches"); 2024-04-25T16:53:58.385 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:58.579 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] pwrite_mmap_blocked 2024-04-25T16:53:58.792 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_dir_offset2 2024-04-25T16:53:58.985 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] seek_sanity_test 2024-04-25T16:53:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:seek_sanity_test.c: In function 'test18': 2024-04-25T16:53:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:seek_sanity_test.c:460:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:59.112 INFO:teuthology.orchestra.run.smithi094.stderr: 460 | ftruncate(fd, 0); 2024-04-25T16:53:59.112 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T16:53:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:seek_sanity_test.c: In function 'test_basic_support': 2024-04-25T16:53:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:seek_sanity_test.c:1207:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:59.115 INFO:teuthology.orchestra.run.smithi094.stderr: 1207 | ftruncate(fd, 0); 2024-04-25T16:53:59.115 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T16:53:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:seek_sanity_test.c:1256:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:59.115 INFO:teuthology.orchestra.run.smithi094.stderr: 1256 | ftruncate(fd, 0); 2024-04-25T16:53:59.115 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T16:53:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:seek_sanity_test.c: In function 'get_io_sizes': 2024-04-25T16:53:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:seek_sanity_test.c:88:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:59.115 INFO:teuthology.orchestra.run.smithi094.stderr: 88 | ftruncate(fd, 0); 2024-04-25T16:53:59.116 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T16:53:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:seek_sanity_test.c:89:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:59.116 INFO:teuthology.orchestra.run.smithi094.stderr: 89 | pwrite(fd, "a", 1, offset); 2024-04-25T16:53:59.116 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:seek_sanity_test.c:98:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:59.116 INFO:teuthology.orchestra.run.smithi094.stderr: 98 | ftruncate(fd, 0); 2024-04-25T16:53:59.116 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T16:53:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:seek_sanity_test.c:99:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:59.116 INFO:teuthology.orchestra.run.smithi094.stderr: 99 | pwrite(fd, "a", 1, offset); 2024-04-25T16:53:59.116 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:59.443 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] seek_copy_test 2024-04-25T16:53:59.617 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_readdir_1 2024-04-25T16:53:59.719 INFO:teuthology.orchestra.run.smithi094.stderr:t_readdir_1.c: In function 'main': 2024-04-25T16:53:59.719 INFO:teuthology.orchestra.run.smithi094.stderr:t_readdir_1.c:30:25: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:53:59.719 INFO:teuthology.orchestra.run.smithi094.stderr: 30 | read(fd, buf, 100); 2024-04-25T16:53:59.719 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-04-25T16:53:59.751 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_readdir_2 2024-04-25T16:53:59.898 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fsync-tester 2024-04-25T16:54:00.003 INFO:teuthology.orchestra.run.smithi094.stderr:fsync-tester.c: In function 'main': 2024-04-25T16:54:00.003 INFO:teuthology.orchestra.run.smithi094.stderr:fsync-tester.c:391:17: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:54:00.003 INFO:teuthology.orchestra.run.smithi094.stderr: 391 | read(fd, &seed, sizeof(seed)); 2024-04-25T16:54:00.003 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:54:00.003 INFO:teuthology.orchestra.run.smithi094.stderr:fsync-tester.c:541:17: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:54:00.003 INFO:teuthology.orchestra.run.smithi094.stderr: 541 | system("reboot -fn"); 2024-04-25T16:54:00.003 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:54:00.004 INFO:teuthology.orchestra.run.smithi094.stderr:fsync-tester.c: In function 'drop_all_caches': 2024-04-25T16:54:00.004 INFO:teuthology.orchestra.run.smithi094.stderr:fsync-tester.c:90:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:54:00.004 INFO:teuthology.orchestra.run.smithi094.stderr: 90 | write(fd, value, sizeof(value)-1); 2024-04-25T16:54:00.004 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:54:00.149 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] nsexec 2024-04-25T16:54:00.318 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] cloner 2024-04-25T16:54:00.481 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] renameat2 2024-04-25T16:54:00.637 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_getcwd 2024-04-25T16:54:00.744 INFO:teuthology.orchestra.run.smithi094.stderr:t_getcwd.c: In function 'test_getcwd': 2024-04-25T16:54:00.744 INFO:teuthology.orchestra.run.smithi094.stderr:t_getcwd.c:21:17: warning: ignoring return value of 'getcwd' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:54:00.745 INFO:teuthology.orchestra.run.smithi094.stderr: 21 | getcwd(cur_cwd, BUF_SIZE); 2024-04-25T16:54:00.745 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:54:00.793 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] e4compact 2024-04-25T16:54:01.003 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] test-nextquota 2024-04-25T16:54:01.165 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] punch-alternating 2024-04-25T16:54:01.350 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-25T16:54:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] listxattr 2024-04-25T16:54:01.653 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] dio-interleaved 2024-04-25T16:54:01.807 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_dir_type 2024-04-25T16:54:01.960 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] dio-invalidate-cache 2024-04-25T16:54:02.065 INFO:teuthology.orchestra.run.smithi094.stderr:dio-invalidate-cache.c: In function 'main': 2024-04-25T16:54:02.065 INFO:teuthology.orchestra.run.smithi094.stderr:dio-invalidate-cache.c:286:17: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:54:02.065 INFO:teuthology.orchestra.run.smithi094.stderr: 286 | write(fd, buf, blksz * nr_child); 2024-04-25T16:54:02.065 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:54:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] stat_test 2024-04-25T16:54:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_encrypted_d_revalidate 2024-04-25T16:54:02.640 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] attr_replace_test 2024-04-25T16:54:02.795 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] swapon 2024-04-25T16:54:02.949 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] mkswap 2024-04-25T16:54:03.099 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_attr_corruption 2024-04-25T16:54:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_open_tmpfiles 2024-04-25T16:54:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fscrypt-crypt-util 2024-04-25T16:54:04.079 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] bulkstat_null_ocount 2024-04-25T16:54:04.234 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] splice-test 2024-04-25T16:54:04.416 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] chprojid_fail 2024-04-25T16:54:04.571 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] detached_mounts_propagation 2024-04-25T16:54:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] ext4_resize 2024-04-25T16:54:04.903 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_readdir_3 2024-04-25T16:54:05.087 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] splice2pipe 2024-04-25T16:54:05.187 INFO:teuthology.orchestra.run.smithi094.stderr:splice2pipe.c: In function 'prepare_pipe': 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi094.stderr:splice2pipe.c:57:17: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi094.stderr: 57 | write(p[1], buffer, n); 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi094.stderr:splice2pipe.c:65:17: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi094.stderr: 65 | read(p[0], buffer, n); 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:54:05.241 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] uuid_ioctl 2024-04-25T16:54:05.392 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-25T16:54:05.548 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fiemap-fault 2024-04-25T16:54:05.685 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fiemap-tester 2024-04-25T16:54:05.955 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_stripealign 2024-04-25T16:54:06.108 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] open_by_handle 2024-04-25T16:54:06.355 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_mmap_dio 2024-04-25T16:54:06.515 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] uring_read_fault 2024-04-25T16:54:06.700 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] xfsfind 2024-04-25T16:54:06.915 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] t_reflink_read_race 2024-04-25T16:54:07.125 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] fssum 2024-04-25T16:54:07.588 INFO:teuthology.orchestra.run.smithi094.stdout:Building log-writes 2024-04-25T16:54:07.595 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] replay-log 2024-04-25T16:54:07.954 INFO:teuthology.orchestra.run.smithi094.stdout:Building perf 2024-04-25T16:54:07.957 INFO:teuthology.orchestra.run.smithi094.stdout:Building aio-dio-regress 2024-04-25T16:54:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aiocp 2024-04-25T16:54:08.204 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-25T16:54:08.374 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-dio-append-write-read-race 2024-04-25T16:54:08.556 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-dio-cow-race 2024-04-25T16:54:08.708 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-dio-cycle-write 2024-04-25T16:54:08.808 INFO:teuthology.orchestra.run.smithi094.stderr:aio-dio-cycle-write.c: In function 'init_test': 2024-04-25T16:54:08.808 INFO:teuthology.orchestra.run.smithi094.stderr:aio-dio-cycle-write.c:83:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:54:08.808 INFO:teuthology.orchestra.run.smithi094.stderr: 83 | ftruncate(fd, buf_size); 2024-04-25T16:54:08.809 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:54:08.898 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-dio-eof-race 2024-04-25T16:54:09.088 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-dio-extend-stat 2024-04-25T16:54:09.255 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-dio-fcntl-race 2024-04-25T16:54:09.415 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-dio-hole-filling-race 2024-04-25T16:54:09.562 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-dio-invalidate-failure 2024-04-25T16:54:09.717 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-dio-invalidate-readahead 2024-04-25T16:54:09.888 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aiodio_sparse2 2024-04-25T16:54:09.995 INFO:teuthology.orchestra.run.smithi094.stderr:aiodio_sparse2.c: In function 'aiodio_sparse': 2024-04-25T16:54:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:aiodio_sparse2.c:111:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:54:09.996 INFO:teuthology.orchestra.run.smithi094.stderr: 111 | ftruncate(fd, filesize); 2024-04-25T16:54:09.996 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T16:54:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:aiodio_sparse2.c: In function 'dirty_freeblocks': 2024-04-25T16:54:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:aiodio_sparse2.c:274:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T16:54:09.997 INFO:teuthology.orchestra.run.smithi094.stderr: 274 | ftruncate(fd, size); 2024-04-25T16:54:09.997 INFO:teuthology.orchestra.run.smithi094.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T16:54:10.122 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-dio-subblock-eof-read 2024-04-25T16:54:10.270 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-dio-write-verify 2024-04-25T16:54:10.479 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-25T16:54:10.621 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-25T16:54:10.753 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] aio-last-ref-held-by-io 2024-04-25T16:54:10.934 INFO:teuthology.orchestra.run.smithi094.stdout:Building vfs 2024-04-25T16:54:10.941 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] vfstest 2024-04-25T16:54:14.752 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:54:14.754 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:54:15.100 INFO:teuthology.orchestra.run.smithi094.stdout: [CC] mount-idmapped 2024-04-25T16:54:15.560 INFO:teuthology.orchestra.run.smithi094.stdout:Building m4 2024-04-25T16:54:15.565 INFO:teuthology.orchestra.run.smithi094.stdout:Building common 2024-04-25T16:54:15.568 INFO:teuthology.orchestra.run.smithi094.stdout:Building tests 2024-04-25T16:54:15.575 INFO:teuthology.orchestra.run.smithi094.stdout:Building btrfs 2024-04-25T16:54:15.582 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/btrfs/group.list 2024-04-25T16:54:15.653 INFO:teuthology.orchestra.run.smithi094.stdout:Building ceph 2024-04-25T16:54:15.658 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/ceph/group.list 2024-04-25T16:54:15.674 INFO:teuthology.orchestra.run.smithi094.stdout:Building cifs 2024-04-25T16:54:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/cifs/group.list 2024-04-25T16:54:15.695 INFO:teuthology.orchestra.run.smithi094.stdout:Building ext4 2024-04-25T16:54:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/ext4/group.list 2024-04-25T16:54:15.740 INFO:teuthology.orchestra.run.smithi094.stdout:Building f2fs 2024-04-25T16:54:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/f2fs/group.list 2024-04-25T16:54:15.763 INFO:teuthology.orchestra.run.smithi094.stdout:Building generic 2024-04-25T16:54:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/generic/group.list 2024-04-25T16:54:15.878 INFO:teuthology.orchestra.run.smithi094.stdout:Building nfs 2024-04-25T16:54:15.883 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/nfs/group.list 2024-04-25T16:54:15.899 INFO:teuthology.orchestra.run.smithi094.stdout:Building ocfs2 2024-04-25T16:54:15.904 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/ocfs2/group.list 2024-04-25T16:54:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:Building overlay 2024-04-25T16:54:15.924 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/overlay/group.list 2024-04-25T16:54:15.969 INFO:teuthology.orchestra.run.smithi094.stdout:Building perf 2024-04-25T16:54:15.975 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/perf/group.list 2024-04-25T16:54:15.988 INFO:teuthology.orchestra.run.smithi094.stdout:Building selftest 2024-04-25T16:54:15.993 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/selftest/group.list 2024-04-25T16:54:16.007 INFO:teuthology.orchestra.run.smithi094.stdout:Building shared 2024-04-25T16:54:16.012 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/shared/group.list 2024-04-25T16:54:16.030 INFO:teuthology.orchestra.run.smithi094.stdout:Building tmpfs 2024-04-25T16:54:16.035 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/tmpfs/group.list 2024-04-25T16:54:16.050 INFO:teuthology.orchestra.run.smithi094.stdout:Building udf 2024-04-25T16:54:16.055 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/udf/group.list 2024-04-25T16:54:16.068 INFO:teuthology.orchestra.run.smithi094.stdout:Building xfs 2024-04-25T16:54:16.075 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/xfs/group.list 2024-04-25T16:54:16.180 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo make install) 2024-04-25T16:54:16.205 INFO:teuthology.orchestra.run.smithi094.stdout:Building include 2024-04-25T16:54:16.209 INFO:teuthology.orchestra.run.smithi094.stdout:Building lib 2024-04-25T16:54:16.213 INFO:teuthology.orchestra.run.smithi094.stdout:Building ltp 2024-04-25T16:54:16.218 INFO:teuthology.orchestra.run.smithi094.stdout:Building src 2024-04-25T16:54:16.253 INFO:teuthology.orchestra.run.smithi094.stdout:Building log-writes 2024-04-25T16:54:16.257 INFO:teuthology.orchestra.run.smithi094.stdout:Building perf 2024-04-25T16:54:16.260 INFO:teuthology.orchestra.run.smithi094.stdout:Building aio-dio-regress 2024-04-25T16:54:16.265 INFO:teuthology.orchestra.run.smithi094.stdout:Building vfs 2024-04-25T16:54:16.270 INFO:teuthology.orchestra.run.smithi094.stdout:Building m4 2024-04-25T16:54:16.273 INFO:teuthology.orchestra.run.smithi094.stdout:Building common 2024-04-25T16:54:16.277 INFO:teuthology.orchestra.run.smithi094.stdout:Building tests 2024-04-25T16:54:16.283 INFO:teuthology.orchestra.run.smithi094.stdout:Building btrfs 2024-04-25T16:54:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/btrfs/group.list 2024-04-25T16:54:16.347 INFO:teuthology.orchestra.run.smithi094.stdout:Building ceph 2024-04-25T16:54:16.352 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/ceph/group.list 2024-04-25T16:54:16.366 INFO:teuthology.orchestra.run.smithi094.stdout:Building cifs 2024-04-25T16:54:16.371 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/cifs/group.list 2024-04-25T16:54:16.382 INFO:teuthology.orchestra.run.smithi094.stdout:Building ext4 2024-04-25T16:54:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/ext4/group.list 2024-04-25T16:54:16.422 INFO:teuthology.orchestra.run.smithi094.stdout:Building f2fs 2024-04-25T16:54:16.427 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/f2fs/group.list 2024-04-25T16:54:16.444 INFO:teuthology.orchestra.run.smithi094.stdout:Building generic 2024-04-25T16:54:16.451 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/generic/group.list 2024-04-25T16:54:16.550 INFO:teuthology.orchestra.run.smithi094.stdout:Building nfs 2024-04-25T16:54:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/nfs/group.list 2024-04-25T16:54:16.571 INFO:teuthology.orchestra.run.smithi094.stdout:Building ocfs2 2024-04-25T16:54:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/ocfs2/group.list 2024-04-25T16:54:16.591 INFO:teuthology.orchestra.run.smithi094.stdout:Building overlay 2024-04-25T16:54:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/overlay/group.list 2024-04-25T16:54:16.637 INFO:teuthology.orchestra.run.smithi094.stdout:Building perf 2024-04-25T16:54:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/perf/group.list 2024-04-25T16:54:16.654 INFO:teuthology.orchestra.run.smithi094.stdout:Building selftest 2024-04-25T16:54:16.659 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/selftest/group.list 2024-04-25T16:54:16.671 INFO:teuthology.orchestra.run.smithi094.stdout:Building shared 2024-04-25T16:54:16.676 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/shared/group.list 2024-04-25T16:54:16.693 INFO:teuthology.orchestra.run.smithi094.stdout:Building tmpfs 2024-04-25T16:54:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/tmpfs/group.list 2024-04-25T16:54:16.710 INFO:teuthology.orchestra.run.smithi094.stdout:Building udf 2024-04-25T16:54:16.715 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/udf/group.list 2024-04-25T16:54:16.726 INFO:teuthology.orchestra.run.smithi094.stdout:Building xfs 2024-04-25T16:54:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/xfs/group.list 2024-04-25T16:54:16.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-04-25T16:54:16.825 INFO:teuthology.orchestra.run.smithi094.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-25T16:54:16.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-04-25T16:54:16.828 INFO:teuthology.orchestra.run.smithi094.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-25T16:54:16.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-04-25T16:54:16.831 INFO:teuthology.orchestra.run.smithi094.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-04-25T16:54:16.839 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-04-25T16:54:16.910 INFO:teuthology.orchestra.run.smithi094.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-04-25T16:54:16.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-04-25T16:54:16.938 INFO:teuthology.orchestra.run.smithi094.stdout:Building log-writes 2024-04-25T16:54:16.941 INFO:teuthology.orchestra.run.smithi094.stdout:Building perf 2024-04-25T16:54:16.944 INFO:teuthology.orchestra.run.smithi094.stdout:Building aio-dio-regress 2024-04-25T16:54:16.948 INFO:teuthology.orchestra.run.smithi094.stdout:Building vfs 2024-04-25T16:54:16.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake -C log-writes install 2024-04-25T16:54:16.953 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-04-25T16:54:16.962 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-04-25T16:54:16.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake -C perf install 2024-04-25T16:54:16.973 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-04-25T16:54:16.981 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-04-25T16:54:17.000 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-04-25T16:54:17.002 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-04-25T16:54:17.010 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-04-25T16:54:17.052 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake -C vfs install 2024-04-25T16:54:17.055 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-04-25T16:54:17.063 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-04-25T16:54:17.075 INFO:teuthology.orchestra.run.smithi094.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-04-25T16:54:17.084 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault fiemap-tester t_stripealign open_by_handle t_mmap_dio uring_read_fault xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-04-25T16:54:18.436 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-04-25T16:54:18.543 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-04-25T16:54:18.568 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-04-25T16:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-25T16:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-04-25T16:54:18.573 INFO:teuthology.orchestra.run.smithi094.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-04-25T16:54:18.582 INFO:teuthology.orchestra.run.smithi094.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-04-25T16:54:18.706 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-04-25T16:54:18.708 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/btrfs/ install 2024-04-25T16:54:18.712 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/btrfs/group.list 2024-04-25T16:54:18.772 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-04-25T16:54:18.782 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 320 330 /var/lib/xfstests/tests/btrfs 2024-04-25T16:54:19.699 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-04-25T16:54:19.710 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 320.out 330.out /var/lib/xfstests/tests/btrfs 2024-04-25T16:54:20.648 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/ceph/ install 2024-04-25T16:54:20.653 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/ceph/group.list 2024-04-25T16:54:20.671 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-04-25T16:54:20.683 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-04-25T16:54:20.707 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-04-25T16:54:20.718 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-04-25T16:54:20.742 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/cifs/ install 2024-04-25T16:54:20.747 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/cifs/group.list 2024-04-25T16:54:20.761 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-04-25T16:54:20.771 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-04-25T16:54:20.783 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-04-25T16:54:20.794 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-04-25T16:54:20.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/ext4/ install 2024-04-25T16:54:20.809 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/ext4/group.list 2024-04-25T16:54:20.855 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-04-25T16:54:20.868 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-04-25T16:54:21.073 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-04-25T16:54:21.085 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-04-25T16:54:21.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/f2fs/ install 2024-04-25T16:54:21.314 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/f2fs/group.list 2024-04-25T16:54:21.333 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-04-25T16:54:21.344 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-04-25T16:54:21.357 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-04-25T16:54:21.368 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-04-25T16:54:21.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/generic/ install 2024-04-25T16:54:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/generic/group.list 2024-04-25T16:54:21.509 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-04-25T16:54:21.521 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 741 742 743 /var/lib/xfstests/tests/generic 2024-04-25T16:54:24.007 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-04-25T16:54:24.019 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 741.out 742.out 743.out /var/lib/xfstests/tests/generic 2024-04-25T16:54:26.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/nfs/ install 2024-04-25T16:54:26.510 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/nfs/group.list 2024-04-25T16:54:26.528 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-04-25T16:54:26.540 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-04-25T16:54:26.552 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-04-25T16:54:26.564 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-04-25T16:54:26.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/ocfs2/ install 2024-04-25T16:54:26.580 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/ocfs2/group.list 2024-04-25T16:54:26.597 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-04-25T16:54:26.608 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-04-25T16:54:26.620 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-04-25T16:54:26.632 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-04-25T16:54:26.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/overlay/ install 2024-04-25T16:54:26.648 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/overlay/group.list 2024-04-25T16:54:26.699 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-04-25T16:54:26.711 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-04-25T16:54:27.044 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-04-25T16:54:27.056 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-04-25T16:54:27.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/perf/ install 2024-04-25T16:54:27.421 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/perf/group.list 2024-04-25T16:54:27.435 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-04-25T16:54:27.447 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-04-25T16:54:27.458 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-04-25T16:54:27.471 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-04-25T16:54:27.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/selftest/ install 2024-04-25T16:54:27.487 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/selftest/group.list 2024-04-25T16:54:27.502 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-04-25T16:54:27.513 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-04-25T16:54:27.539 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-04-25T16:54:27.550 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-04-25T16:54:27.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/shared/ install 2024-04-25T16:54:27.580 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/shared/group.list 2024-04-25T16:54:27.601 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-04-25T16:54:27.612 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-04-25T16:54:27.629 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-04-25T16:54:27.641 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-04-25T16:54:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/tmpfs/ install 2024-04-25T16:54:27.663 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/tmpfs/group.list 2024-04-25T16:54:27.678 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-04-25T16:54:27.688 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-04-25T16:54:27.699 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-04-25T16:54:27.710 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-04-25T16:54:27.721 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/udf/ install 2024-04-25T16:54:27.725 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/udf/group.list 2024-04-25T16:54:27.739 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-04-25T16:54:27.751 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-04-25T16:54:27.761 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-04-25T16:54:27.771 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-04-25T16:54:27.782 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.FooT1Le6w4xfstests-dev/tests/xfs/ install 2024-04-25T16:54:27.787 INFO:teuthology.orchestra.run.smithi094.stdout: [GROUP] /tmp/tmp.FooT1Le6w4xfstests-dev/tests/xfs/group.list 2024-04-25T16:54:27.907 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-04-25T16:54:27.919 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-04-25T16:54:30.128 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-04-25T16:54:30.140 INFO:teuthology.orchestra.run.smithi094.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-04-25T16:54:32.407 INFO:teuthology.orchestra.run.smithi094.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-04-25T16:54:32.419 INFO:teuthology.orchestra.run.smithi094.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-04-25T16:54:32.431 INFO:teuthology.orchestra.run.smithi094.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-04-25T16:54:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-04-25T16:54:32.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:54:32.458 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec ls tests/generic) 2024-04-25T16:54:32.506 INFO:teuthology.orchestra.run.smithi094.stdout:001 2024-04-25T16:54:32.506 INFO:teuthology.orchestra.run.smithi094.stdout:001.out 2024-04-25T16:54:32.506 INFO:teuthology.orchestra.run.smithi094.stdout:002 2024-04-25T16:54:32.506 INFO:teuthology.orchestra.run.smithi094.stdout:002.out 2024-04-25T16:54:32.506 INFO:teuthology.orchestra.run.smithi094.stdout:003 2024-04-25T16:54:32.506 INFO:teuthology.orchestra.run.smithi094.stdout:003.out 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:004 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:004.out 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:005 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:005.out 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:006 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:006.out 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:007 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:007.out 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:008 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:008.out 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:009 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:009.out 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:010 2024-04-25T16:54:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:010.out 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:011 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:011.out 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:012 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:012.out 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:013 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:013.out 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:014 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:014.out 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:015 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:015.out 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:016 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:016.out 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:017 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:017.out 2024-04-25T16:54:32.508 INFO:teuthology.orchestra.run.smithi094.stdout:018 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:018.out 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:019 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:019.out 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:020 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:020.out 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:021 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:021.out 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:022 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:022.out 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:023 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:023.out 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:024 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:024.out 2024-04-25T16:54:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:025 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:025.out 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:026 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:026.out 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:027 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:027.out 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:028 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:028.out 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:029 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:029.out 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:030 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:030.out 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:031 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:031.out 2024-04-25T16:54:32.510 INFO:teuthology.orchestra.run.smithi094.stdout:032 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:032.out 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:033 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:033.out 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:034 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:034.out 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:035 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:035.cfg 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:035.out.default 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:035.out.nfs 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:036 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:036.out 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:037 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:037.out 2024-04-25T16:54:32.511 INFO:teuthology.orchestra.run.smithi094.stdout:038 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:038.out 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:039 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:039.out 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:040 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:040.out 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:041 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:041.out 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:042 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:042.out 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:043 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:043.out 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:044 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:044.out 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:045 2024-04-25T16:54:32.512 INFO:teuthology.orchestra.run.smithi094.stdout:045.out 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:046 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:046.out 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:047 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:047.out 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:048 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:048.out 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:049 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:049.out 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:050 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:050.cfg 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:050.out.default 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:050.out.nojournal 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:050.out.xfsquota 2024-04-25T16:54:32.513 INFO:teuthology.orchestra.run.smithi094.stdout:051 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:051.out 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:052 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:052.out 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:053 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:053.out 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:054 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:054.out 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:055 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:055.out 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:056 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:056.out 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:057 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:057.out 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:058 2024-04-25T16:54:32.514 INFO:teuthology.orchestra.run.smithi094.stdout:058.out 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:059 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:059.out 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:060 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:060.out 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:061 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:061.out 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:062 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:062.out 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:063 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:063.out 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:064 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:064.out 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:065 2024-04-25T16:54:32.515 INFO:teuthology.orchestra.run.smithi094.stdout:065.out 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:066 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:066.out 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:067 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:067.out 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:068 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:068.out 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:069 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:069.out 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:070 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:070.out 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:071 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:071.out 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:072 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:072.out 2024-04-25T16:54:32.516 INFO:teuthology.orchestra.run.smithi094.stdout:073 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:073.out 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:074 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:074.out 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:075 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:075.out 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:076 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:076.out 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:077 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:077.out 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:078 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:078.out 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:079 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:079.out 2024-04-25T16:54:32.517 INFO:teuthology.orchestra.run.smithi094.stdout:080 2024-04-25T16:54:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:080.out 2024-04-25T16:54:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:081 2024-04-25T16:54:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:081.out 2024-04-25T16:54:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:082 2024-04-25T16:54:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:082.out 2024-04-25T16:54:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:083 2024-04-25T16:54:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:083.out 2024-04-25T16:54:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:084 2024-04-25T16:54:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:084.out 2024-04-25T16:54:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:085 2024-04-25T16:54:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:085.out 2024-04-25T16:54:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:086 2024-04-25T16:54:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:086.out 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:087 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:087.out 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:088 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:088.out 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:089 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:089.out 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:090 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:090.out 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:091 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:091.out 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:092 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:092.out 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:093 2024-04-25T16:54:32.519 INFO:teuthology.orchestra.run.smithi094.stdout:093.out 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:094 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:094.out 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:095 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:095.out 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:096 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:096.out 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:097 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:097.out 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:098 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:098.out 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:099 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:099.out 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:100 2024-04-25T16:54:32.520 INFO:teuthology.orchestra.run.smithi094.stdout:100.out 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:101 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:101.out 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:102 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:102.out 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:103 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:103.out 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:104 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:104.out 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:105 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:105.out 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:106 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:106.out 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:107 2024-04-25T16:54:32.521 INFO:teuthology.orchestra.run.smithi094.stdout:107.out 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:108 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:108.out 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:109 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:109.out 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:110 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:110.out 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:111 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:111.out 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:112 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:112.out 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:113 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:113.out 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:114 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:114.out 2024-04-25T16:54:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:115 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:115.out 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:116 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:116.out 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:117 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:117.out 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:118 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:118.out 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:119 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:119.out 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:120 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:120.out 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:121 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:121.out 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:122 2024-04-25T16:54:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:122.out 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:123 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:123.out 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:124 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:124.out 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:125 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:125.out 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:126 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:126.out 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:127 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:127.out 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:128 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:128.out 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:129 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:129.out 2024-04-25T16:54:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:130 2024-04-25T16:54:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:130.out 2024-04-25T16:54:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:131 2024-04-25T16:54:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:131.out 2024-04-25T16:54:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:132 2024-04-25T16:54:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:132.out 2024-04-25T16:54:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:133 2024-04-25T16:54:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:133.out 2024-04-25T16:54:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:134 2024-04-25T16:54:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:134.out 2024-04-25T16:54:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:135 2024-04-25T16:54:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:135.out 2024-04-25T16:54:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:136 2024-04-25T16:54:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:136.out 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:137 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:137.out 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:138 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:138.out 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:139 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:139.out 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:140 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:140.out 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:141 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:141.out 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:142 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:142.out 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:143 2024-04-25T16:54:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:143.out 2024-04-25T16:54:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:144 2024-04-25T16:54:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:144.out 2024-04-25T16:54:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:145 2024-04-25T16:54:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:145.out 2024-04-25T16:54:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:146 2024-04-25T16:54:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:146.out 2024-04-25T16:54:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:147 2024-04-25T16:54:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:147.out 2024-04-25T16:54:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:148 2024-04-25T16:54:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:148.out 2024-04-25T16:54:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:149 2024-04-25T16:54:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:149.out 2024-04-25T16:54:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:150 2024-04-25T16:54:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:150.out 2024-04-25T16:54:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:151 2024-04-25T16:54:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:151.out 2024-04-25T16:54:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:152 2024-04-25T16:54:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:152.out 2024-04-25T16:54:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:153 2024-04-25T16:54:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:153.out 2024-04-25T16:54:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:154 2024-04-25T16:54:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:154.out 2024-04-25T16:54:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:155 2024-04-25T16:54:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:155.out 2024-04-25T16:54:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:156 2024-04-25T16:54:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:156.out 2024-04-25T16:54:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:157 2024-04-25T16:54:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:157.out 2024-04-25T16:54:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:158 2024-04-25T16:54:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:158.out 2024-04-25T16:54:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:159 2024-04-25T16:54:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:159.out 2024-04-25T16:54:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:160 2024-04-25T16:54:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:160.out 2024-04-25T16:54:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:161 2024-04-25T16:54:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:161.out 2024-04-25T16:54:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:162 2024-04-25T16:54:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:162.out 2024-04-25T16:54:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:163 2024-04-25T16:54:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:163.out 2024-04-25T16:54:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:164 2024-04-25T16:54:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:164.out 2024-04-25T16:54:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:165 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:165.out 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:166 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:166.out 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:167 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:167.out 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:168 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:168.out 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:169 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:169.out 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:170 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:170.out 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:171 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:171.out 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:172 2024-04-25T16:54:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:172.out 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:173 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:173.out 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:174 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:174.out 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:175 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:175.out 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:176 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:176.out 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:177 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:177.out 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:178 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:178.out 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:179 2024-04-25T16:54:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:179.out 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:180 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:180.out 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:181 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:181.out 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:182 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:182.out 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:183 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:183.out 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:184 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:184.out 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:185 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:185.out 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:186 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:186.out 2024-04-25T16:54:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:187 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:187.out 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:188 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:188.out 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:189 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:189.out 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:190 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:190.out 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:191 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:191.out 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:192 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:192.out 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:193 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:193.out 2024-04-25T16:54:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:194 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:194.out 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:195 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:195.out 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:196 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:196.out 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:197 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:197.out 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:198 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:198.out 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:199 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:199.out 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:200 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:200.out 2024-04-25T16:54:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:201 2024-04-25T16:54:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:201.out 2024-04-25T16:54:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:202 2024-04-25T16:54:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:202.out 2024-04-25T16:54:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:203 2024-04-25T16:54:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:203.out 2024-04-25T16:54:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:204 2024-04-25T16:54:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:204.out 2024-04-25T16:54:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:205 2024-04-25T16:54:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:205.out 2024-04-25T16:54:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:206 2024-04-25T16:54:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:206.out 2024-04-25T16:54:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:207 2024-04-25T16:54:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:207.out 2024-04-25T16:54:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:208 2024-04-25T16:54:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:208.out 2024-04-25T16:54:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:209 2024-04-25T16:54:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:209.out 2024-04-25T16:54:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:210 2024-04-25T16:54:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:210.out 2024-04-25T16:54:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:211 2024-04-25T16:54:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:211.out 2024-04-25T16:54:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:212 2024-04-25T16:54:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:212.out 2024-04-25T16:54:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:213 2024-04-25T16:54:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:213.out 2024-04-25T16:54:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:214 2024-04-25T16:54:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:214.out 2024-04-25T16:54:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:215 2024-04-25T16:54:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:215.out 2024-04-25T16:54:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:216 2024-04-25T16:54:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:216.out 2024-04-25T16:54:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:217 2024-04-25T16:54:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:217.out 2024-04-25T16:54:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:218 2024-04-25T16:54:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:218.out 2024-04-25T16:54:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:219 2024-04-25T16:54:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:219.out 2024-04-25T16:54:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:220 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:220.out 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:221 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:221.out 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:222 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:222.out 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:223 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:223.out 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:224 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:224.out 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:225 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:225.out 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:226 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:226.out 2024-04-25T16:54:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:227 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:227.out 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:228 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:228.out 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:229 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:229.out 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:230 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:230.out 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:231 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:231.out 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:232 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:232.out 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:233 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:233.out 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:234 2024-04-25T16:54:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:234.out 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:235 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:235.out 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:236 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:236.out 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:237 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:237.out 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:238 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:238.out 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:239 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:239.out 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:240 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:240.out 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:241 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:241.out 2024-04-25T16:54:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:242 2024-04-25T16:54:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:242.out 2024-04-25T16:54:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:243 2024-04-25T16:54:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:243.out 2024-04-25T16:54:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:244 2024-04-25T16:54:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:244.out 2024-04-25T16:54:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:245 2024-04-25T16:54:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:245.out 2024-04-25T16:54:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:246 2024-04-25T16:54:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:246.out 2024-04-25T16:54:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:247 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:247.out 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:248 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:248.out 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:249 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:249.out 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:250 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:250.out 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:251 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:251.out 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:252 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:252.out 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:253 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:253.out 2024-04-25T16:54:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:254 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:254.out 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:255 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:255.out 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:256 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:256.out 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:257 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:257.out 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:258 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:258.out 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:259 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:259.out 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:260 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:260.out 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:261 2024-04-25T16:54:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:261.out 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:262 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:262.out 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:263 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:263.out 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:264 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:264.out 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:265 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:265.out 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:266 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:266.out 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:267 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:267.out 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:268 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:268.out 2024-04-25T16:54:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:269 2024-04-25T16:54:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:269.out 2024-04-25T16:54:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:270 2024-04-25T16:54:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:270.out 2024-04-25T16:54:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:271 2024-04-25T16:54:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:271.out 2024-04-25T16:54:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:272 2024-04-25T16:54:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:272.out 2024-04-25T16:54:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:273 2024-04-25T16:54:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:273.out 2024-04-25T16:54:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:274 2024-04-25T16:54:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:274.out 2024-04-25T16:54:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:275 2024-04-25T16:54:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:275.out 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:276 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:276.out 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:277 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:277.out 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:278 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:278.out 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:279 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:279.out 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:280 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:280.out 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:281 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:281.out 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:282 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:282.out 2024-04-25T16:54:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:283 2024-04-25T16:54:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:283.out 2024-04-25T16:54:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:284 2024-04-25T16:54:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:284.out 2024-04-25T16:54:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:285 2024-04-25T16:54:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:285.out 2024-04-25T16:54:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:286 2024-04-25T16:54:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:286.out 2024-04-25T16:54:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:287 2024-04-25T16:54:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:287.out 2024-04-25T16:54:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:288 2024-04-25T16:54:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:288.out 2024-04-25T16:54:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:289 2024-04-25T16:54:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:289.out 2024-04-25T16:54:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:290 2024-04-25T16:54:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:290.out 2024-04-25T16:54:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:291 2024-04-25T16:54:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:291.out 2024-04-25T16:54:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:292 2024-04-25T16:54:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:292.out 2024-04-25T16:54:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:293 2024-04-25T16:54:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:293.out 2024-04-25T16:54:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:294 2024-04-25T16:54:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:294.out 2024-04-25T16:54:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:295 2024-04-25T16:54:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:295.out 2024-04-25T16:54:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:296 2024-04-25T16:54:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:296.out 2024-04-25T16:54:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:297 2024-04-25T16:54:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:297.out 2024-04-25T16:54:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:298 2024-04-25T16:54:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:298.out 2024-04-25T16:54:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:299 2024-04-25T16:54:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:299.out 2024-04-25T16:54:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:300 2024-04-25T16:54:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:300.out 2024-04-25T16:54:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:301 2024-04-25T16:54:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:301.out 2024-04-25T16:54:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:302 2024-04-25T16:54:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:302.out 2024-04-25T16:54:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:303 2024-04-25T16:54:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:303.out 2024-04-25T16:54:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:304 2024-04-25T16:54:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:304.out 2024-04-25T16:54:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:305 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:305.out 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:306 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:306.out 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:307 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:307.out 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:308 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:308.out 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:309 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:309.out 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:310 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:310.out 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:311 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:311.out 2024-04-25T16:54:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:312 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:312.out 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:313 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:313.out 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:314 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:314.out 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:315 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:315.out 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:316 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:316.out 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:317 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:317.out 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:318 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:318.out 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:319 2024-04-25T16:54:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:319.out 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:320 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:320.out 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:321 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:321.out 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:322 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:322.out 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:323 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:323.out 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:324 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:324.out 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:325 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:325.out 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:326 2024-04-25T16:54:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:326.out 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:327 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:327.out 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:328 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:328.out 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:329 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:329.out 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:330 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:330.out 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:331 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:331.out 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:332 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:332.out 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:333 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:333.out 2024-04-25T16:54:32.559 INFO:teuthology.orchestra.run.smithi094.stdout:334 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:334.out 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:335 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:335.out 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:336 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:336.out 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:337 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:337.out 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:338 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:338.out 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:339 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:339.out 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:340 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:340.out 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:341 2024-04-25T16:54:32.560 INFO:teuthology.orchestra.run.smithi094.stdout:341.out 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:342 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:342.out 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:343 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:343.out 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:344 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:344.out 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:345 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:345.out 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:346 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:346.out 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:347 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:347.out 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:348 2024-04-25T16:54:32.561 INFO:teuthology.orchestra.run.smithi094.stdout:348.out 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:349 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:349.out 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:350 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:350.out 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:351 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:351.out 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:352 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:352.out 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:353 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:353.out 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:354 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:354.out 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:355 2024-04-25T16:54:32.562 INFO:teuthology.orchestra.run.smithi094.stdout:355.out 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:356 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:356.out 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:357 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:357.out 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:358 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:358.out 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:359 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:359.out 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:360 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:360.out 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:361 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:361.out 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:362 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:362.out 2024-04-25T16:54:32.563 INFO:teuthology.orchestra.run.smithi094.stdout:363 2024-04-25T16:54:32.564 INFO:teuthology.orchestra.run.smithi094.stdout:363.out 2024-04-25T16:54:32.564 INFO:teuthology.orchestra.run.smithi094.stdout:364 2024-04-25T16:54:32.564 INFO:teuthology.orchestra.run.smithi094.stdout:364.out 2024-04-25T16:54:32.564 INFO:teuthology.orchestra.run.smithi094.stdout:365 2024-04-25T16:54:32.564 INFO:teuthology.orchestra.run.smithi094.stdout:365.out 2024-04-25T16:54:32.564 INFO:teuthology.orchestra.run.smithi094.stdout:366 2024-04-25T16:54:32.564 INFO:teuthology.orchestra.run.smithi094.stdout:366.out 2024-04-25T16:54:32.564 INFO:teuthology.orchestra.run.smithi094.stdout:367 2024-04-25T16:54:32.564 INFO:teuthology.orchestra.run.smithi094.stdout:367.out 2024-04-25T16:54:32.564 INFO:teuthology.orchestra.run.smithi094.stdout:368 2024-04-25T16:54:32.564 INFO:teuthology.orchestra.run.smithi094.stdout:368.out 2024-04-25T16:54:32.564 INFO:teuthology.orchestra.run.smithi094.stdout:369 2024-04-25T16:54:32.564 INFO:teuthology.orchestra.run.smithi094.stdout:369.out 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:370 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:370.out 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:371 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:371.out 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:372 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:372.out 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:373 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:373.out 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:374 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:374.out 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:375 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:375.out 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:376 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:376.out 2024-04-25T16:54:32.565 INFO:teuthology.orchestra.run.smithi094.stdout:377 2024-04-25T16:54:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:377.out 2024-04-25T16:54:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:378 2024-04-25T16:54:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:378.out 2024-04-25T16:54:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:379 2024-04-25T16:54:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:379.out 2024-04-25T16:54:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:380 2024-04-25T16:54:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:380.out 2024-04-25T16:54:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:381 2024-04-25T16:54:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:381.out 2024-04-25T16:54:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:382 2024-04-25T16:54:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:382.out 2024-04-25T16:54:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:383 2024-04-25T16:54:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:383.out 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:384 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:384.out 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:385 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:385.out 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:386 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:386.out 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:387 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:387.out 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:388 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:388.out 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:389 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:389.out 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:390 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:390.out 2024-04-25T16:54:32.567 INFO:teuthology.orchestra.run.smithi094.stdout:391 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:391.out 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:392 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:392.out 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:393 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:393.out 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:394 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:394.out 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:395 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:395.out 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:396 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:396.out 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:397 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:397.out 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:398 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:398.out 2024-04-25T16:54:32.568 INFO:teuthology.orchestra.run.smithi094.stdout:399 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:399.out 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:400 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:400.out 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:401 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:401.out 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:402 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:402.out 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:403 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:403.out 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:404 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:404.out 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:405 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:405.out 2024-04-25T16:54:32.569 INFO:teuthology.orchestra.run.smithi094.stdout:406 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:406.out 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:407 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:407.out 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:408 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:408.out 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:409 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:409.out 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:410 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:410.out 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:411 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:411.out 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:412 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:412.out 2024-04-25T16:54:32.570 INFO:teuthology.orchestra.run.smithi094.stdout:413 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:413.out 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:414 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:414.out 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:415 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:415.out 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:416 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:416.out 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:417 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:417.out 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:418 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:418.out 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:419 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:419.out 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:420 2024-04-25T16:54:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:420.out 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:421 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:421.out 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:422 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:422.out 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:423 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:423.out 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:424 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:424.out 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:425 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:425.out 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:426 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:426.out 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:427 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:427.out 2024-04-25T16:54:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:428 2024-04-25T16:54:32.573 INFO:teuthology.orchestra.run.smithi094.stdout:428.out 2024-04-25T16:54:32.573 INFO:teuthology.orchestra.run.smithi094.stdout:429 2024-04-25T16:54:32.573 INFO:teuthology.orchestra.run.smithi094.stdout:429.out 2024-04-25T16:54:32.573 INFO:teuthology.orchestra.run.smithi094.stdout:430 2024-04-25T16:54:32.573 INFO:teuthology.orchestra.run.smithi094.stdout:430.out 2024-04-25T16:54:32.573 INFO:teuthology.orchestra.run.smithi094.stdout:431 2024-04-25T16:54:32.573 INFO:teuthology.orchestra.run.smithi094.stdout:431.out 2024-04-25T16:54:32.573 INFO:teuthology.orchestra.run.smithi094.stdout:432 2024-04-25T16:54:32.573 INFO:teuthology.orchestra.run.smithi094.stdout:432.out 2024-04-25T16:54:32.573 INFO:teuthology.orchestra.run.smithi094.stdout:433 2024-04-25T16:54:32.573 INFO:teuthology.orchestra.run.smithi094.stdout:433.out 2024-04-25T16:54:32.573 INFO:teuthology.orchestra.run.smithi094.stdout:434 2024-04-25T16:54:32.573 INFO:teuthology.orchestra.run.smithi094.stdout:434.out 2024-04-25T16:54:32.574 INFO:teuthology.orchestra.run.smithi094.stdout:435 2024-04-25T16:54:32.574 INFO:teuthology.orchestra.run.smithi094.stdout:435.out 2024-04-25T16:54:32.574 INFO:teuthology.orchestra.run.smithi094.stdout:436 2024-04-25T16:54:32.574 INFO:teuthology.orchestra.run.smithi094.stdout:436.out 2024-04-25T16:54:32.574 INFO:teuthology.orchestra.run.smithi094.stdout:437 2024-04-25T16:54:32.574 INFO:teuthology.orchestra.run.smithi094.stdout:437.out 2024-04-25T16:54:32.574 INFO:teuthology.orchestra.run.smithi094.stdout:438 2024-04-25T16:54:32.574 INFO:teuthology.orchestra.run.smithi094.stdout:438.out 2024-04-25T16:54:32.574 INFO:teuthology.orchestra.run.smithi094.stdout:439 2024-04-25T16:54:32.574 INFO:teuthology.orchestra.run.smithi094.stdout:439.out 2024-04-25T16:54:32.574 INFO:teuthology.orchestra.run.smithi094.stdout:440 2024-04-25T16:54:32.574 INFO:teuthology.orchestra.run.smithi094.stdout:440.out 2024-04-25T16:54:32.574 INFO:teuthology.orchestra.run.smithi094.stdout:441 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:441.out 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:442 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:442.out 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:443 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:443.out 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:444 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:444.out 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:445 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:445.out 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:446 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:446.out 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:447 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:447.out 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:448 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi094.stdout:448.out 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:449 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:449.out 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:450 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:450.out 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:451 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:451.out 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:452 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:452.out 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:453 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:453.out 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:454 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:454.out 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:455 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:455.out 2024-04-25T16:54:32.576 INFO:teuthology.orchestra.run.smithi094.stdout:456 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:456.out 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:457 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:457.out 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:458 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:458.out 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:459 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:459.out 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:460 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:460.out 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:461 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:461.out 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:462 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:462.out 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:463 2024-04-25T16:54:32.577 INFO:teuthology.orchestra.run.smithi094.stdout:463.out 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:464 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:464.out 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:465 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:465.out 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:466 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:466.out 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:467 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:467.out 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:468 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:468.out 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:469 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:469.out 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:470 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:470.out 2024-04-25T16:54:32.578 INFO:teuthology.orchestra.run.smithi094.stdout:471 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:471.out 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:472 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:472.out 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:473 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:473.out 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:474 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:474.out 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:475 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:475.out 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:476 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:476.out 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:477 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:477.out 2024-04-25T16:54:32.579 INFO:teuthology.orchestra.run.smithi094.stdout:478 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:478.out 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:479 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:479.out 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:480 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:480.out 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:481 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:481.out 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:482 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:482.out 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:483 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:483.out 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:484 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:484.out 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:485 2024-04-25T16:54:32.580 INFO:teuthology.orchestra.run.smithi094.stdout:485.out 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:486 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:486.out 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:487 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:487.out 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:488 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:488.out 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:489 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:489.out 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:490 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:490.out 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:491 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:491.out 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:492 2024-04-25T16:54:32.581 INFO:teuthology.orchestra.run.smithi094.stdout:492.out 2024-04-25T16:54:32.582 INFO:teuthology.orchestra.run.smithi094.stdout:493 2024-04-25T16:54:32.582 INFO:teuthology.orchestra.run.smithi094.stdout:493.out 2024-04-25T16:54:32.582 INFO:teuthology.orchestra.run.smithi094.stdout:494 2024-04-25T16:54:32.582 INFO:teuthology.orchestra.run.smithi094.stdout:494.out 2024-04-25T16:54:32.582 INFO:teuthology.orchestra.run.smithi094.stdout:495 2024-04-25T16:54:32.582 INFO:teuthology.orchestra.run.smithi094.stdout:495.out 2024-04-25T16:54:32.582 INFO:teuthology.orchestra.run.smithi094.stdout:496 2024-04-25T16:54:32.582 INFO:teuthology.orchestra.run.smithi094.stdout:496.out 2024-04-25T16:54:32.582 INFO:teuthology.orchestra.run.smithi094.stdout:497 2024-04-25T16:54:32.582 INFO:teuthology.orchestra.run.smithi094.stdout:497.out 2024-04-25T16:54:32.582 INFO:teuthology.orchestra.run.smithi094.stdout:498 2024-04-25T16:54:32.582 INFO:teuthology.orchestra.run.smithi094.stdout:498.out 2024-04-25T16:54:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:499 2024-04-25T16:54:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:499.out 2024-04-25T16:54:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:500 2024-04-25T16:54:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:500.out 2024-04-25T16:54:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:501 2024-04-25T16:54:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:501.out 2024-04-25T16:54:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:502 2024-04-25T16:54:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:502.out 2024-04-25T16:54:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:503 2024-04-25T16:54:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:503.out 2024-04-25T16:54:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:504 2024-04-25T16:54:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:504.out 2024-04-25T16:54:32.583 INFO:teuthology.orchestra.run.smithi094.stdout:505 2024-04-25T16:54:32.584 INFO:teuthology.orchestra.run.smithi094.stdout:505.out 2024-04-25T16:54:32.584 INFO:teuthology.orchestra.run.smithi094.stdout:506 2024-04-25T16:54:32.584 INFO:teuthology.orchestra.run.smithi094.stdout:506.out 2024-04-25T16:54:32.584 INFO:teuthology.orchestra.run.smithi094.stdout:507 2024-04-25T16:54:32.584 INFO:teuthology.orchestra.run.smithi094.stdout:507.out 2024-04-25T16:54:32.584 INFO:teuthology.orchestra.run.smithi094.stdout:508 2024-04-25T16:54:32.584 INFO:teuthology.orchestra.run.smithi094.stdout:508.out 2024-04-25T16:54:32.584 INFO:teuthology.orchestra.run.smithi094.stdout:509 2024-04-25T16:54:32.584 INFO:teuthology.orchestra.run.smithi094.stdout:509.out 2024-04-25T16:54:32.584 INFO:teuthology.orchestra.run.smithi094.stdout:510 2024-04-25T16:54:32.584 INFO:teuthology.orchestra.run.smithi094.stdout:510.out 2024-04-25T16:54:32.585 INFO:teuthology.orchestra.run.smithi094.stdout:511 2024-04-25T16:54:32.585 INFO:teuthology.orchestra.run.smithi094.stdout:511.out 2024-04-25T16:54:32.585 INFO:teuthology.orchestra.run.smithi094.stdout:512 2024-04-25T16:54:32.585 INFO:teuthology.orchestra.run.smithi094.stdout:512.out 2024-04-25T16:54:32.585 INFO:teuthology.orchestra.run.smithi094.stdout:513 2024-04-25T16:54:32.585 INFO:teuthology.orchestra.run.smithi094.stdout:513.out 2024-04-25T16:54:32.585 INFO:teuthology.orchestra.run.smithi094.stdout:514 2024-04-25T16:54:32.585 INFO:teuthology.orchestra.run.smithi094.stdout:514.out 2024-04-25T16:54:32.585 INFO:teuthology.orchestra.run.smithi094.stdout:515 2024-04-25T16:54:32.585 INFO:teuthology.orchestra.run.smithi094.stdout:515.out 2024-04-25T16:54:32.585 INFO:teuthology.orchestra.run.smithi094.stdout:516 2024-04-25T16:54:32.585 INFO:teuthology.orchestra.run.smithi094.stdout:516.out 2024-04-25T16:54:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:517 2024-04-25T16:54:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:517.out 2024-04-25T16:54:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:518 2024-04-25T16:54:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:518.out 2024-04-25T16:54:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:519 2024-04-25T16:54:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:519.out 2024-04-25T16:54:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:520 2024-04-25T16:54:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:520.out 2024-04-25T16:54:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:521 2024-04-25T16:54:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:521.out 2024-04-25T16:54:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:522 2024-04-25T16:54:32.587 INFO:teuthology.orchestra.run.smithi094.stdout:522.out 2024-04-25T16:54:32.587 INFO:teuthology.orchestra.run.smithi094.stdout:523 2024-04-25T16:54:32.587 INFO:teuthology.orchestra.run.smithi094.stdout:523.out 2024-04-25T16:54:32.587 INFO:teuthology.orchestra.run.smithi094.stdout:524 2024-04-25T16:54:32.587 INFO:teuthology.orchestra.run.smithi094.stdout:524.out 2024-04-25T16:54:32.587 INFO:teuthology.orchestra.run.smithi094.stdout:525 2024-04-25T16:54:32.587 INFO:teuthology.orchestra.run.smithi094.stdout:525.out 2024-04-25T16:54:32.587 INFO:teuthology.orchestra.run.smithi094.stdout:526 2024-04-25T16:54:32.587 INFO:teuthology.orchestra.run.smithi094.stdout:526.out 2024-04-25T16:54:32.587 INFO:teuthology.orchestra.run.smithi094.stdout:527 2024-04-25T16:54:32.587 INFO:teuthology.orchestra.run.smithi094.stdout:527.out 2024-04-25T16:54:32.588 INFO:teuthology.orchestra.run.smithi094.stdout:528 2024-04-25T16:54:32.588 INFO:teuthology.orchestra.run.smithi094.stdout:528.out 2024-04-25T16:54:32.588 INFO:teuthology.orchestra.run.smithi094.stdout:529 2024-04-25T16:54:32.588 INFO:teuthology.orchestra.run.smithi094.stdout:529.out 2024-04-25T16:54:32.588 INFO:teuthology.orchestra.run.smithi094.stdout:530 2024-04-25T16:54:32.588 INFO:teuthology.orchestra.run.smithi094.stdout:530.out 2024-04-25T16:54:32.588 INFO:teuthology.orchestra.run.smithi094.stdout:531 2024-04-25T16:54:32.588 INFO:teuthology.orchestra.run.smithi094.stdout:531.out 2024-04-25T16:54:32.588 INFO:teuthology.orchestra.run.smithi094.stdout:532 2024-04-25T16:54:32.588 INFO:teuthology.orchestra.run.smithi094.stdout:532.out 2024-04-25T16:54:32.588 INFO:teuthology.orchestra.run.smithi094.stdout:533 2024-04-25T16:54:32.588 INFO:teuthology.orchestra.run.smithi094.stdout:533.out 2024-04-25T16:54:32.588 INFO:teuthology.orchestra.run.smithi094.stdout:534 2024-04-25T16:54:32.589 INFO:teuthology.orchestra.run.smithi094.stdout:534.out 2024-04-25T16:54:32.589 INFO:teuthology.orchestra.run.smithi094.stdout:535 2024-04-25T16:54:32.589 INFO:teuthology.orchestra.run.smithi094.stdout:535.out 2024-04-25T16:54:32.589 INFO:teuthology.orchestra.run.smithi094.stdout:536 2024-04-25T16:54:32.589 INFO:teuthology.orchestra.run.smithi094.stdout:536.out 2024-04-25T16:54:32.589 INFO:teuthology.orchestra.run.smithi094.stdout:537 2024-04-25T16:54:32.589 INFO:teuthology.orchestra.run.smithi094.stdout:537.out 2024-04-25T16:54:32.589 INFO:teuthology.orchestra.run.smithi094.stdout:538 2024-04-25T16:54:32.589 INFO:teuthology.orchestra.run.smithi094.stdout:538.out 2024-04-25T16:54:32.589 INFO:teuthology.orchestra.run.smithi094.stdout:539 2024-04-25T16:54:32.589 INFO:teuthology.orchestra.run.smithi094.stdout:539.out 2024-04-25T16:54:32.589 INFO:teuthology.orchestra.run.smithi094.stdout:540 2024-04-25T16:54:32.590 INFO:teuthology.orchestra.run.smithi094.stdout:540.out 2024-04-25T16:54:32.590 INFO:teuthology.orchestra.run.smithi094.stdout:541 2024-04-25T16:54:32.590 INFO:teuthology.orchestra.run.smithi094.stdout:541.out 2024-04-25T16:54:32.590 INFO:teuthology.orchestra.run.smithi094.stdout:542 2024-04-25T16:54:32.590 INFO:teuthology.orchestra.run.smithi094.stdout:542.out 2024-04-25T16:54:32.590 INFO:teuthology.orchestra.run.smithi094.stdout:543 2024-04-25T16:54:32.590 INFO:teuthology.orchestra.run.smithi094.stdout:543.out 2024-04-25T16:54:32.590 INFO:teuthology.orchestra.run.smithi094.stdout:544 2024-04-25T16:54:32.590 INFO:teuthology.orchestra.run.smithi094.stdout:544.out 2024-04-25T16:54:32.590 INFO:teuthology.orchestra.run.smithi094.stdout:545 2024-04-25T16:54:32.590 INFO:teuthology.orchestra.run.smithi094.stdout:545.out 2024-04-25T16:54:32.590 INFO:teuthology.orchestra.run.smithi094.stdout:546 2024-04-25T16:54:32.590 INFO:teuthology.orchestra.run.smithi094.stdout:546.out 2024-04-25T16:54:32.591 INFO:teuthology.orchestra.run.smithi094.stdout:547 2024-04-25T16:54:32.591 INFO:teuthology.orchestra.run.smithi094.stdout:547.out 2024-04-25T16:54:32.591 INFO:teuthology.orchestra.run.smithi094.stdout:548 2024-04-25T16:54:32.591 INFO:teuthology.orchestra.run.smithi094.stdout:548.out 2024-04-25T16:54:32.591 INFO:teuthology.orchestra.run.smithi094.stdout:549 2024-04-25T16:54:32.591 INFO:teuthology.orchestra.run.smithi094.stdout:549.out 2024-04-25T16:54:32.591 INFO:teuthology.orchestra.run.smithi094.stdout:550 2024-04-25T16:54:32.591 INFO:teuthology.orchestra.run.smithi094.stdout:550.out 2024-04-25T16:54:32.591 INFO:teuthology.orchestra.run.smithi094.stdout:551 2024-04-25T16:54:32.591 INFO:teuthology.orchestra.run.smithi094.stdout:551.out 2024-04-25T16:54:32.591 INFO:teuthology.orchestra.run.smithi094.stdout:552 2024-04-25T16:54:32.591 INFO:teuthology.orchestra.run.smithi094.stdout:552.out 2024-04-25T16:54:32.591 INFO:teuthology.orchestra.run.smithi094.stdout:553 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:553.out 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:554 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:554.out 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:555 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:555.out 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:556 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:556.out 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:557 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:557.out 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:558 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:558.out 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:559 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:559.out 2024-04-25T16:54:32.592 INFO:teuthology.orchestra.run.smithi094.stdout:560 2024-04-25T16:54:32.593 INFO:teuthology.orchestra.run.smithi094.stdout:560.out 2024-04-25T16:54:32.593 INFO:teuthology.orchestra.run.smithi094.stdout:561 2024-04-25T16:54:32.593 INFO:teuthology.orchestra.run.smithi094.stdout:561.out 2024-04-25T16:54:32.593 INFO:teuthology.orchestra.run.smithi094.stdout:562 2024-04-25T16:54:32.593 INFO:teuthology.orchestra.run.smithi094.stdout:562.out 2024-04-25T16:54:32.593 INFO:teuthology.orchestra.run.smithi094.stdout:563 2024-04-25T16:54:32.593 INFO:teuthology.orchestra.run.smithi094.stdout:563.out 2024-04-25T16:54:32.593 INFO:teuthology.orchestra.run.smithi094.stdout:564 2024-04-25T16:54:32.593 INFO:teuthology.orchestra.run.smithi094.stdout:564.out 2024-04-25T16:54:32.593 INFO:teuthology.orchestra.run.smithi094.stdout:565 2024-04-25T16:54:32.593 INFO:teuthology.orchestra.run.smithi094.stdout:565.out 2024-04-25T16:54:32.593 INFO:teuthology.orchestra.run.smithi094.stdout:566 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:566.out 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:567 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:567.out 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:568 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:568.out 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:569 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:569.out 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:570 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:570.out 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:571 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:571.out 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:572 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:572.out 2024-04-25T16:54:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:573 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:573.out 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:574 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:574.out 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:575 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:575.out 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:576 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:576.out 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:577 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:577.out 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:578 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:578.out 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:579 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:579.out 2024-04-25T16:54:32.595 INFO:teuthology.orchestra.run.smithi094.stdout:580 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:580.out 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:581 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:581.out 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:582 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:582.out 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:583 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:583.out 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:584 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:584.out 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:585 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:585.out 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:586 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:586.out 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:587 2024-04-25T16:54:32.596 INFO:teuthology.orchestra.run.smithi094.stdout:587.out 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:588 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:588.out 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:589 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:589.out 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:590 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:590.out 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:591 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:591.out 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:592 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:592.out 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:593 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:593.out 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:594 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:594.out 2024-04-25T16:54:32.597 INFO:teuthology.orchestra.run.smithi094.stdout:595 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:595.out 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:596 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:596.out 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:597 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:597.out 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:598 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:598.out 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:599 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:599.out 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:600 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:600.out 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:601 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:601.out 2024-04-25T16:54:32.598 INFO:teuthology.orchestra.run.smithi094.stdout:602 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:602.out 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:603 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:603.out 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:604 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:604.out 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:605 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:605.out 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:606 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:606.out 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:607 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:607.out 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:608 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:608.out 2024-04-25T16:54:32.599 INFO:teuthology.orchestra.run.smithi094.stdout:609 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:609.out 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:610 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:610.out 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:611 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:611.out 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:612 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:612.out 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:613 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:613.out 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:614 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:614.out 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:615 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:615.out 2024-04-25T16:54:32.600 INFO:teuthology.orchestra.run.smithi094.stdout:616 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:616.out 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:617 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:617.out 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:618 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:618.out 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:619 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:619.out 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:620 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:620.out 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:621 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:621.out 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:622 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:622.out 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:623 2024-04-25T16:54:32.601 INFO:teuthology.orchestra.run.smithi094.stdout:623.out 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:624 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:624.out 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:625 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:625.out 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:626 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:626.out 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:627 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:627.out 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:628 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:628.out 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:629 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:629.out 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:630 2024-04-25T16:54:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:630.out 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:631 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:631.out 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:632 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:632.out 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:633 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:633.out 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:634 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:634.out 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:635 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:635.out 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:636 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:636.out 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:637 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:637.out 2024-04-25T16:54:32.603 INFO:teuthology.orchestra.run.smithi094.stdout:638 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:638.out 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:639 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:639.out 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:640 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:640.out 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:641 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:641.out 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:642 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:642.out 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:643 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:643.out 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:644 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:644.out 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:645 2024-04-25T16:54:32.604 INFO:teuthology.orchestra.run.smithi094.stdout:645.out 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:646 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:646.out 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:647 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:647.out 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:648 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:648.out 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:649 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:649.out 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:650 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:650.out 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:651 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:651.out 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:652 2024-04-25T16:54:32.605 INFO:teuthology.orchestra.run.smithi094.stdout:652.out 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:653 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:653.out 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:654 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:654.out 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:655 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:655.out 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:656 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:656.out 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:657 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:657.out 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:658 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:658.out 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:659 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:659.out 2024-04-25T16:54:32.606 INFO:teuthology.orchestra.run.smithi094.stdout:660 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:660.out 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:661 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:661.out 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:662 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:662.out 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:663 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:663.out 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:664 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:664.out 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:665 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:665.out 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:666 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:666.out 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:667 2024-04-25T16:54:32.607 INFO:teuthology.orchestra.run.smithi094.stdout:667.out 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:668 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:668.out 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:669 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:669.out 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:670 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:670.out 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:671 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:671.out 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:672 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:672.out 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:673 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:673.out 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:674 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:674.out 2024-04-25T16:54:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:675 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:675.out 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:676 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:676.out 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:677 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:677.out 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:678 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:678.out 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:679 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:679.out 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:680 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:680.out 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:681 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:681.out 2024-04-25T16:54:32.609 INFO:teuthology.orchestra.run.smithi094.stdout:682 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:682.out 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:683 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:683.out 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:684 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:684.out 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:685 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:685.out 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:686 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:686.out 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:687 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:687.out 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:688 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:688.out 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:689 2024-04-25T16:54:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:689.out 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:690 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:690.out 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:691 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:691.out 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:692 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:692.out 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:693 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:693.out 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:694 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:694.out 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:695 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:695.out 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:696 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:696.out 2024-04-25T16:54:32.611 INFO:teuthology.orchestra.run.smithi094.stdout:697 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:697.out 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:698 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:698.out 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:699 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:699.out 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:700 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:700.out 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:701 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:701.out 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:702 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:702.out 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:703 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:703.out 2024-04-25T16:54:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:704 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:704.out 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:705 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:705.out 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:706 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:706.out 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:707 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:707.out 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:708 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:708.out 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:709 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:709.out 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:710 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:710.out 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:711 2024-04-25T16:54:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:711.out 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:712 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:712.out 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:713 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:713.out 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:714 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:714.out 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:715 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:715.out 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:716 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:716.out 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:717 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:717.out 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:718 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:718.out 2024-04-25T16:54:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:719 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:719.out 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:720 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:720.out 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:721 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:721.out 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:722 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:722.out 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:723 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:723.out 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:724 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:724.out 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:725 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:725.out 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:726 2024-04-25T16:54:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:726.out 2024-04-25T16:54:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:727 2024-04-25T16:54:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:727.out 2024-04-25T16:54:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:728 2024-04-25T16:54:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:728.out 2024-04-25T16:54:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:729 2024-04-25T16:54:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:729.out 2024-04-25T16:54:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:730 2024-04-25T16:54:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:730.out 2024-04-25T16:54:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:731 2024-04-25T16:54:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:731.out 2024-04-25T16:54:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:732 2024-04-25T16:54:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:732.out 2024-04-25T16:54:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:733 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:733.out 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:734 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:734.out 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:735 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:735.out 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:736 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:736.out 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:737 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:737.out 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:738 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:738.out 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:739 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:739.out 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:741 2024-04-25T16:54:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:741.out 2024-04-25T16:54:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:742 2024-04-25T16:54:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:742.out 2024-04-25T16:54:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:743 2024-04-25T16:54:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:743.out 2024-04-25T16:54:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:group.list 2024-04-25T16:54:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:Makefile 2024-04-25T16:54:32.618 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2024-04-25T16:54:32.802 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:54:32.807 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:54:32.808 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:54:34.968 INFO:teuthology.orchestra.run.smithi094.stdout:generic/001 2s 2024-04-25T16:54:34.975 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/001 2024-04-25T16:54:34.975 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:54:34.979 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:54:34.981 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[39350]: fuse finished with error 0 and tester_r 0 2024-04-25T16:54:34.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:54:34.992 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2024-04-25T16:54:35.210 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:54:35.215 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:54:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:54:35.642 INFO:teuthology.orchestra.run.smithi094.stdout:generic/002 0s 2024-04-25T16:54:35.646 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/002 2024-04-25T16:54:35.647 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:54:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:54:35.659 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:54:35.659 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2024-04-25T16:54:35.896 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:54:35.901 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:54:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:54:36.007 INFO:teuthology.orchestra.run.smithi094.stdout:generic/003 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T16:54:36.011 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/003 2024-04-25T16:54:36.013 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/003 2024-04-25T16:54:36.013 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:54:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:54:36.021 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:54:36.021 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2024-04-25T16:54:36.197 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:54:36.201 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:54:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:54:36.309 INFO:teuthology.orchestra.run.smithi094.stdout:generic/004 [not run] O_TMPFILE is not supported 2024-04-25T16:54:36.312 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/004 2024-04-25T16:54:36.312 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/004 2024-04-25T16:54:36.312 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:54:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:54:36.322 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:54:36.323 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2024-04-25T16:54:36.495 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:54:36.499 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:54:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:54:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:generic/005 0s 2024-04-25T16:54:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/005 2024-04-25T16:54:36.978 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:54:36.980 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:54:36.988 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:54:36.989 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2024-04-25T16:54:37.207 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:54:37.211 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:54:37.211 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:54:44.829 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:54:44.832 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:55:14.911 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:55:14.914 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:55:22.961 INFO:teuthology.orchestra.run.smithi094.stdout:generic/006 45s 2024-04-25T16:55:22.966 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/006 2024-04-25T16:55:22.966 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:55:22.969 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:55:23.022 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:55:23.023 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2024-04-25T16:55:23.207 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:55:23.212 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:55:23.212 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:55:44.966 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:55:44.969 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:56:15.369 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:56:15.371 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:56:41.076 INFO:teuthology.orchestra.run.smithi094.stdout:generic/007 78s 2024-04-25T16:56:41.082 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/007 2024-04-25T16:56:41.082 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:56:41.084 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:56:41.095 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:56:41.095 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2024-04-25T16:56:41.356 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:56:41.360 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:56:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:56:42.380 INFO:teuthology.orchestra.run.smithi094.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-25T16:56:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/008 2024-04-25T16:56:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/008 2024-04-25T16:56:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:56:42.386 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:56:42.394 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:56:42.394 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2024-04-25T16:56:42.569 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:56:42.573 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:56:42.573 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:56:43.540 INFO:teuthology.orchestra.run.smithi094.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-25T16:56:43.544 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/009 2024-04-25T16:56:43.544 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/009 2024-04-25T16:56:43.545 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:56:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:56:43.556 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:56:43.556 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2024-04-25T16:56:43.741 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:56:43.745 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:56:43.745 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:56:43.852 INFO:teuthology.orchestra.run.smithi094.stdout:generic/010 [not run] /tmp/tmp.FooT1Le6w4xfstests-dev/src/dbtest not built 2024-04-25T16:56:43.855 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/010 2024-04-25T16:56:43.856 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/010 2024-04-25T16:56:43.856 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:56:43.858 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:56:43.866 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:56:43.866 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2024-04-25T16:56:44.046 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:56:44.049 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:56:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:56:45.668 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:56:45.670 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:57:15.725 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:57:15.727 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:57:26.389 INFO:teuthology.orchestra.run.smithi094.stdout:generic/011 42s 2024-04-25T16:57:26.419 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/011 2024-04-25T16:57:26.420 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:57:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:57:26.434 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:57:26.435 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2024-04-25T16:57:26.676 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:57:26.680 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:57:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:57:27.646 INFO:teuthology.orchestra.run.smithi094.stdout:generic/012 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-25T16:57:27.649 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/012 2024-04-25T16:57:27.649 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/012 2024-04-25T16:57:27.649 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:57:27.652 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:57:27.661 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:57:27.662 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2024-04-25T16:57:27.888 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:57:27.892 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:57:27.892 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:57:46.135 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:57:46.138 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:58:16.270 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:58:16.272 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:58:46.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:58:46.988 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T16:58:47.204 INFO:tasks.ceph.mds.a.smithi094.stderr:2024-04-25T16:58:47.206+0000 7fb5271b5640 -1 Fail to open '/proc/85746/cmdline' error = (2) No such file or directory 2024-04-25T16:58:47.205 INFO:tasks.ceph.mds.a.smithi094.stderr:2024-04-25T16:58:47.206+0000 7fb5271b5640 -1 received signal: Hangup from (PID: 85746) UID: 0 2024-04-25T16:58:47.205 INFO:tasks.ceph.mds.c.smithi094.stderr:2024-04-25T16:58:47.206+0000 7f3de9731640 -1 Fail to open '/proc/85746/cmdline' error = (2) No such file or directory 2024-04-25T16:58:47.205 INFO:tasks.ceph.mds.c.smithi094.stderr:2024-04-25T16:58:47.206+0000 7f3de9731640 -1 received signal: Hangup from (PID: 85746) UID: 0 2024-04-25T16:59:26.824 INFO:teuthology.orchestra.run.smithi094.stdout:generic/013 119s 2024-04-25T16:59:26.899 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/013 2024-04-25T16:59:26.899 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:26.905 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:26.922 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:26.923 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2024-04-25T16:59:27.292 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:27.303 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:27.303 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:27.929 INFO:teuthology.orchestra.run.smithi094.stdout:generic/014 [not run] Sparse files are not supported or do not work as expected 2024-04-25T16:59:27.941 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/014 2024-04-25T16:59:27.941 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/014 2024-04-25T16:59:27.941 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:27.944 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:27.969 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:27.970 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2024-04-25T16:59:28.374 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:28.382 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:28.382 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:28.555 INFO:teuthology.orchestra.run.smithi094.stdout:generic/015 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T16:59:28.559 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/015 2024-04-25T16:59:28.559 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/015 2024-04-25T16:59:28.559 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:28.561 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:28.580 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:28.581 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2024-04-25T16:59:28.995 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:29.002 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:29.002 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:30.835 INFO:teuthology.orchestra.run.smithi094.stdout:generic/016 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-25T16:59:30.844 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/016 2024-04-25T16:59:30.844 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/016 2024-04-25T16:59:30.844 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:30.848 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:30.886 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:30.886 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2024-04-25T16:59:31.390 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:31.394 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:31.395 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:31.581 INFO:teuthology.orchestra.run.smithi094.stdout:generic/017 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T16:59:31.586 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/017 2024-04-25T16:59:31.587 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/017 2024-04-25T16:59:31.587 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:31.589 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:31.606 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:31.606 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2024-04-25T16:59:31.979 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:31.988 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:31.988 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:32.187 INFO:teuthology.orchestra.run.smithi094.stdout:generic/018 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T16:59:32.193 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/018 2024-04-25T16:59:32.193 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/018 2024-04-25T16:59:32.194 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:32.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:32.213 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2024-04-25T16:59:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:32.616 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:32.806 INFO:teuthology.orchestra.run.smithi094.stdout:generic/019 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T16:59:32.814 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/019 2024-04-25T16:59:32.815 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/019 2024-04-25T16:59:32.815 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:32.817 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:32.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:32.832 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2024-04-25T16:59:33.354 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:33.360 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:33.360 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:47.419 INFO:teuthology.orchestra.run.smithi094.stdout:generic/020 - output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/020.out.bad) 2024-04-25T16:59:47.424 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/020.out 2024-04-25 16:52:50.067100103 +0000 2024-04-25T16:59:47.424 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/020.out.bad 2024-04-25 16:59:47.421163085 +0000 2024-04-25T16:59:47.424 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -47,9 +47,13 @@ 2024-04-25T16:59:47.424 INFO:teuthology.orchestra.run.smithi094.stdout: user.snrub="fish2\012" 2024-04-25T16:59:47.424 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:47.424 INFO:teuthology.orchestra.run.smithi094.stdout: *** really long value 2024-04-25T16:59:47.424 INFO:teuthology.orchestra.run.smithi094.stdout: -0000000 00 2024-04-25T16:59:47.424 INFO:teuthology.orchestra.run.smithi094.stdout: -* 2024-04-25T16:59:47.424 INFO:teuthology.orchestra.run.smithi094.stdout: -ATTRSIZE 2024-04-25T16:59:47.425 INFO:teuthology.orchestra.run.smithi094.stdout: +attr_set: No space left on device 2024-04-25T16:59:47.425 INFO:teuthology.orchestra.run.smithi094.stdout: +Could not set "long_attr" for 2024-04-25T16:59:47.425 INFO:teuthology.orchestra.run.smithi094.stdout: +attr_get: No data available 2024-04-25T16:59:47.425 INFO:teuthology.orchestra.run.smithi094.stdout: +Could not get "long_attr" for 2024-04-25T16:59:47.425 INFO:teuthology.orchestra.run.smithi094.stdout: +0000000 2024-04-25T16:59:47.425 INFO:teuthology.orchestra.run.smithi094.stdout: +attr_remove: No data available 2024-04-25T16:59:47.425 INFO:teuthology.orchestra.run.smithi094.stdout: +Could not remove "long_attr" for 2024-04-25T16:59:47.425 INFO:teuthology.orchestra.run.smithi094.stdout: *** set/get/remove really long names (expect failure) 2024-04-25T16:59:47.425 INFO:teuthology.orchestra.run.smithi094.stdout: attr_set: Invalid argument 2024-04-25T16:59:47.425 INFO:teuthology.orchestra.run.smithi094.stdout: Could not set "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" for 2024-04-25T16:59:47.429 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/020 2024-04-25T16:59:47.429 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/020 2024-04-25T16:59:47.429 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-25T16:59:47.432 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:47.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:59:47.605 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-25T16:59:47.605 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-25T16:59:47.605 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2024-04-25T16:59:47.991 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:47.997 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:47.997 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:49.830 INFO:teuthology.orchestra.run.smithi094.stdout:generic/021 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-25T16:59:49.840 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/021 2024-04-25T16:59:49.840 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/021 2024-04-25T16:59:49.840 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:49.906 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:49.906 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2024-04-25T16:59:50.458 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:50.469 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:50.469 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:52.291 INFO:teuthology.orchestra.run.smithi094.stdout:generic/022 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-25T16:59:52.296 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/022 2024-04-25T16:59:52.296 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/022 2024-04-25T16:59:52.296 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:52.342 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:52.342 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2024-04-25T16:59:52.877 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:52.886 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:55.110 INFO:teuthology.orchestra.run.smithi094.stdout:generic/023 3s 2024-04-25T16:59:55.125 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/023 2024-04-25T16:59:55.125 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:55.128 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:55.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:55.214 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2024-04-25T16:59:55.763 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:55.779 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:55.779 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:56.009 INFO:teuthology.orchestra.run.smithi094.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2024-04-25T16:59:56.013 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/024 2024-04-25T16:59:56.014 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/024 2024-04-25T16:59:56.014 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:56.017 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:56.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:56.030 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2024-04-25T16:59:56.625 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:56.630 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:56.630 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:56.875 INFO:teuthology.orchestra.run.smithi094.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2024-04-25T16:59:56.880 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/025 2024-04-25T16:59:56.880 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/025 2024-04-25T16:59:56.880 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:56.883 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:56.895 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:56.896 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2024-04-25T16:59:57.380 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:57.387 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:57.388 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:57.647 INFO:teuthology.orchestra.run.smithi094.stdout:generic/026 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-25T16:59:57.652 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/026 2024-04-25T16:59:57.652 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/026 2024-04-25T16:59:57.652 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:57.655 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:57.672 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:57.672 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2024-04-25T16:59:58.099 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:58.124 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:58.312 INFO:teuthology.orchestra.run.smithi094.stdout:generic/027 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T16:59:58.316 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/027 2024-04-25T16:59:58.317 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/027 2024-04-25T16:59:58.317 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T16:59:58.331 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T16:59:58.343 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T16:59:58.343 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2024-04-25T16:59:58.583 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T16:59:58.588 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T16:59:58.588 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:03.937 INFO:teuthology.orchestra.run.smithi094.stdout:generic/028 5s 2024-04-25T17:00:03.970 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/028 2024-04-25T17:00:03.970 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:03.970 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:03.979 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:03.980 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2024-04-25T17:00:04.379 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:04.386 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:04.387 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:04.591 INFO:teuthology.orchestra.run.smithi094.stdout:generic/029 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:04.600 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/029 2024-04-25T17:00:04.601 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/029 2024-04-25T17:00:04.601 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:04.603 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:04.619 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:04.619 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2024-04-25T17:00:05.062 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:05.068 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:05.068 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:05.227 INFO:teuthology.orchestra.run.smithi094.stdout:generic/030 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:05.231 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/030 2024-04-25T17:00:05.231 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/030 2024-04-25T17:00:05.231 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:05.237 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:05.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:05.253 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2024-04-25T17:00:05.653 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:05.658 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:05.658 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:05.834 INFO:teuthology.orchestra.run.smithi094.stdout:generic/031 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:05.840 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/031 2024-04-25T17:00:05.840 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/031 2024-04-25T17:00:05.840 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:05.848 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:05.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:05.882 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2024-04-25T17:00:06.296 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:06.303 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:06.497 INFO:teuthology.orchestra.run.smithi094.stdout:generic/032 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:06.507 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/032 2024-04-25T17:00:06.507 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/032 2024-04-25T17:00:06.507 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:06.510 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:06.520 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:06.520 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2024-04-25T17:00:06.936 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:06.940 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:07.114 INFO:teuthology.orchestra.run.smithi094.stdout:generic/033 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:07.119 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/033 2024-04-25T17:00:07.119 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/033 2024-04-25T17:00:07.119 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:07.128 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:07.139 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:07.139 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2024-04-25T17:00:07.557 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:07.565 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:07.794 INFO:teuthology.orchestra.run.smithi094.stdout:generic/034 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:07.799 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/034 2024-04-25T17:00:07.799 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/034 2024-04-25T17:00:07.799 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:07.802 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:07.835 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:07.836 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2024-04-25T17:00:08.266 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:08.272 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:08.272 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:08.602 INFO:teuthology.orchestra.run.smithi094.stdout:generic/035 0s 2024-04-25T17:00:08.736 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/035 2024-04-25T17:00:08.737 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:08.740 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:08.754 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:08.754 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2024-04-25T17:00:09.243 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:09.266 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:09.266 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:19.680 INFO:teuthology.orchestra.run.smithi094.stdout:generic/036 10s 2024-04-25T17:00:19.688 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/036 2024-04-25T17:00:19.688 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:19.690 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:19.702 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:19.702 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2024-04-25T17:00:20.318 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:20.323 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:20.323 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:20.491 INFO:teuthology.orchestra.run.smithi094.stdout:generic/037 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:20.496 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/037 2024-04-25T17:00:20.496 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/037 2024-04-25T17:00:20.496 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:20.502 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:20.514 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:20.514 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2024-04-25T17:00:20.895 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:20.900 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:21.076 INFO:teuthology.orchestra.run.smithi094.stdout:generic/038 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:21.082 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/038 2024-04-25T17:00:21.082 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/038 2024-04-25T17:00:21.082 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:21.124 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:21.125 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2024-04-25T17:00:21.540 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:21.545 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:21.749 INFO:teuthology.orchestra.run.smithi094.stdout:generic/039 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:21.771 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/039 2024-04-25T17:00:21.771 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/039 2024-04-25T17:00:21.771 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:21.796 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:21.796 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2024-04-25T17:00:22.165 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:22.171 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:22.172 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:22.351 INFO:teuthology.orchestra.run.smithi094.stdout:generic/040 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:22.358 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/040 2024-04-25T17:00:22.358 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/040 2024-04-25T17:00:22.358 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:22.361 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:22.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:22.373 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2024-04-25T17:00:22.783 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:22.809 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:22.809 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:22.986 INFO:teuthology.orchestra.run.smithi094.stdout:generic/041 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:22.991 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/041 2024-04-25T17:00:22.991 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/041 2024-04-25T17:00:22.991 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:22.994 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:23.011 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:23.011 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2024-04-25T17:00:23.478 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:23.485 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:23.485 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:23.660 INFO:teuthology.orchestra.run.smithi094.stdout:generic/042 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:23.665 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/042 2024-04-25T17:00:23.665 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/042 2024-04-25T17:00:23.665 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:23.669 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:23.688 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:23.689 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2024-04-25T17:00:24.092 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:24.113 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:24.113 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:24.292 INFO:teuthology.orchestra.run.smithi094.stdout:generic/043 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:24.297 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/043 2024-04-25T17:00:24.298 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/043 2024-04-25T17:00:24.298 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:24.302 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:24.321 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:24.322 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2024-04-25T17:00:24.707 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:24.712 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:24.712 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:24.870 INFO:teuthology.orchestra.run.smithi094.stdout:generic/044 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:24.875 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/044 2024-04-25T17:00:24.875 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/044 2024-04-25T17:00:24.875 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:24.878 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:24.893 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:24.893 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2024-04-25T17:00:25.291 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:25.299 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:25.299 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:25.464 INFO:teuthology.orchestra.run.smithi094.stdout:generic/045 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:25.471 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/045 2024-04-25T17:00:25.471 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/045 2024-04-25T17:00:25.471 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:25.477 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:25.490 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:25.490 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2024-04-25T17:00:25.918 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:25.924 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:26.091 INFO:teuthology.orchestra.run.smithi094.stdout:generic/046 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:26.101 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/046 2024-04-25T17:00:26.101 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/046 2024-04-25T17:00:26.101 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:26.127 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:26.127 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2024-04-25T17:00:26.561 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:26.572 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:26.572 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:26.733 INFO:teuthology.orchestra.run.smithi094.stdout:generic/047 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:26.738 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/047 2024-04-25T17:00:26.738 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/047 2024-04-25T17:00:26.738 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:26.765 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:26.766 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2024-04-25T17:00:27.183 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:27.191 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:27.372 INFO:teuthology.orchestra.run.smithi094.stdout:generic/048 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:27.375 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/048 2024-04-25T17:00:27.375 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/048 2024-04-25T17:00:27.376 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:27.379 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:27.410 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:27.410 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2024-04-25T17:00:27.852 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:27.861 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:27.861 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:28.018 INFO:teuthology.orchestra.run.smithi094.stdout:generic/049 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:28.022 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/049 2024-04-25T17:00:28.022 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/049 2024-04-25T17:00:28.022 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:28.027 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:28.039 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:28.040 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2024-04-25T17:00:28.516 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:28.521 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:28.521 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:28.696 INFO:teuthology.orchestra.run.smithi094.stdout:generic/050 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:28.700 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/050 2024-04-25T17:00:28.700 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/050 2024-04-25T17:00:28.700 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:28.703 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:28.720 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:28.720 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2024-04-25T17:00:29.077 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:29.087 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:29.087 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:29.261 INFO:teuthology.orchestra.run.smithi094.stdout:generic/051 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:29.267 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/051 2024-04-25T17:00:29.268 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/051 2024-04-25T17:00:29.268 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:29.271 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:29.295 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:29.295 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2024-04-25T17:00:29.688 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:29.692 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:29.692 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:29.859 INFO:teuthology.orchestra.run.smithi094.stdout:generic/052 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:29.864 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/052 2024-04-25T17:00:29.864 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/052 2024-04-25T17:00:29.864 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:29.873 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:29.915 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:29.915 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2024-04-25T17:00:30.250 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:30.255 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:30.255 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:30.428 INFO:teuthology.orchestra.run.smithi094.stdout:generic/053 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:30.435 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/053 2024-04-25T17:00:30.435 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/053 2024-04-25T17:00:30.435 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:30.438 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:30.460 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:30.460 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2024-04-25T17:00:30.882 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:30.900 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:30.901 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:31.100 INFO:teuthology.orchestra.run.smithi094.stdout:generic/054 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:31.104 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/054 2024-04-25T17:00:31.104 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/054 2024-04-25T17:00:31.104 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:31.109 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:31.128 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:31.128 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2024-04-25T17:00:31.586 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:31.592 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:31.592 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:31.780 INFO:teuthology.orchestra.run.smithi094.stdout:generic/055 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:31.785 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/055 2024-04-25T17:00:31.785 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/055 2024-04-25T17:00:31.785 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:31.790 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:31.838 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:31.838 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2024-04-25T17:00:32.181 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:32.196 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:32.196 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:generic/056 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:32.394 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/056 2024-04-25T17:00:32.394 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/056 2024-04-25T17:00:32.394 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:32.398 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:32.466 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:32.466 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2024-04-25T17:00:32.779 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:32.783 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:32.783 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:32.998 INFO:teuthology.orchestra.run.smithi094.stdout:generic/057 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:33.005 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/057 2024-04-25T17:00:33.006 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/057 2024-04-25T17:00:33.006 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:33.012 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:33.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:33.028 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2024-04-25T17:00:33.319 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:33.324 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:33.324 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:generic/058 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-25T17:00:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/058 2024-04-25T17:00:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/058 2024-04-25T17:00:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:35.074 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:35.085 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:35.086 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2024-04-25T17:00:35.687 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:35.693 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:35.917 INFO:teuthology.orchestra.run.smithi094.stdout:generic/059 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:35.921 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/059 2024-04-25T17:00:35.921 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/059 2024-04-25T17:00:35.921 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:35.934 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:35.934 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2024-04-25T17:00:36.332 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:36.336 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:38.085 INFO:teuthology.orchestra.run.smithi094.stdout:generic/060 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-25T17:00:38.093 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/060 2024-04-25T17:00:38.093 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/060 2024-04-25T17:00:38.094 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:38.097 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:38.116 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:38.116 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2024-04-25T17:00:38.681 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:38.685 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:38.685 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:40.473 INFO:teuthology.orchestra.run.smithi094.stdout:generic/061 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-25T17:00:40.486 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/061 2024-04-25T17:00:40.487 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/061 2024-04-25T17:00:40.487 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:40.502 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:40.519 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:40.519 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2024-04-25T17:00:41.078 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:41.083 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:41.083 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:41.226 INFO:teuthology.orchestra.run.smithi094.stdout:generic/062 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:41.237 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/062 2024-04-25T17:00:41.237 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/062 2024-04-25T17:00:41.237 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:41.243 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:41.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:41.268 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2024-04-25T17:00:41.692 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:41.697 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:41.698 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:43.464 INFO:teuthology.orchestra.run.smithi094.stdout:generic/063 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-25T17:00:43.470 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/063 2024-04-25T17:00:43.470 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/063 2024-04-25T17:00:43.470 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:43.474 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:43.710 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:43.711 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2024-04-25T17:00:44.071 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:44.076 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:44.292 INFO:teuthology.orchestra.run.smithi094.stdout:generic/064 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:44.298 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/064 2024-04-25T17:00:44.298 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/064 2024-04-25T17:00:44.298 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:44.302 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:44.312 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:44.312 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2024-04-25T17:00:44.694 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:44.703 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:44.703 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:44.904 INFO:teuthology.orchestra.run.smithi094.stdout:generic/065 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:44.914 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/065 2024-04-25T17:00:44.914 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/065 2024-04-25T17:00:44.915 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:44.918 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:44.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:44.932 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2024-04-25T17:00:45.323 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:45.328 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:45.328 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:45.517 INFO:teuthology.orchestra.run.smithi094.stdout:generic/066 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:45.522 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/066 2024-04-25T17:00:45.522 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/066 2024-04-25T17:00:45.522 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:45.524 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:45.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:45.543 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2024-04-25T17:00:46.001 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:46.006 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:46.006 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:generic/067 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/067 2024-04-25T17:00:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/067 2024-04-25T17:00:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:46.264 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:46.275 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:46.275 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2024-04-25T17:00:46.673 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:46.678 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:46.678 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:47.161 INFO:teuthology.orchestra.run.smithi094.stdout:generic/068 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:47.165 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/068 2024-04-25T17:00:47.165 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/068 2024-04-25T17:00:47.165 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:47.168 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:47.181 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:47.182 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2024-04-25T17:00:47.554 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:47.558 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:47.558 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:47.750 INFO:teuthology.orchestra.run.smithi094.stdout:generic/069 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:00:47.757 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/069 2024-04-25T17:00:47.757 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/069 2024-04-25T17:00:47.757 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:00:47.758 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:00:47.772 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:00:47.772 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2024-04-25T17:00:48.217 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:00:48.228 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:00:48.228 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:02:03.467 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:02:03.470 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:02:33.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:02:33.708 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:02:39.484 INFO:teuthology.orchestra.run.smithi094.stdout:generic/070 111s 2024-04-25T17:02:39.523 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/070 2024-04-25T17:02:39.523 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:02:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:02:39.534 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:02:39.535 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2024-04-25T17:02:40.159 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:02:40.170 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:02:40.170 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:02:40.344 INFO:teuthology.orchestra.run.smithi094.stdout:generic/071 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:02:40.350 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/071 2024-04-25T17:02:40.350 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/071 2024-04-25T17:02:40.350 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:02:40.354 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:02:40.374 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:02:40.374 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2024-04-25T17:02:40.712 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:02:40.718 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:02:40.719 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:02:42.132 INFO:teuthology.orchestra.run.smithi094.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-04-25T17:02:42.136 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/072 2024-04-25T17:02:42.136 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/072 2024-04-25T17:02:42.136 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:02:42.139 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:02:42.148 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:02:42.149 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2024-04-25T17:02:42.412 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:02:42.420 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:02:42.420 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:02:42.583 INFO:teuthology.orchestra.run.smithi094.stdout:generic/073 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:02:42.588 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/073 2024-04-25T17:02:42.588 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/073 2024-04-25T17:02:42.588 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:02:42.590 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:02:42.610 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:02:42.610 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2024-04-25T17:02:42.874 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:02:42.879 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:02:42.879 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:03:03.841 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:03:03.843 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:03:34.101 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:03:34.104 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:04:04.391 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:04:04.394 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:04:34.691 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:04:34.693 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:05:04.944 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:05:04.946 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:05:35.341 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:05:35.344 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:06:05.525 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:06:05.528 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:06:35.923 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:06:35.926 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:07:07.612 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:07:07.614 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:07:40.161 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:07:40.232 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:08:10.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:08:10.856 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:08:41.199 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:08:41.201 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:09:11.481 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:09:11.483 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:09:41.802 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:09:41.805 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:10:11.862 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:10:11.864 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:10:42.189 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:10:42.192 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:11:12.287 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:11:12.431 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:11:42.506 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:11:42.509 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:12:12.624 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:12:12.627 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:12:42.908 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:12:42.911 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:12:42.961 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T17:12:42.962+0000 7f2c3013d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34000) UID: 0 2024-04-25T17:12:42.961 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T17:12:42.962+0000 7f0f14e8e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34000) UID: 0 2024-04-25T17:12:42.961 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T17:12:42.962+0000 7f7fd7c37640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34000) UID: 0 2024-04-25T17:12:42.962 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T17:12:42.962+0000 7f55ebc07640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34000) UID: 0 2024-04-25T17:17:24.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:17:24.229 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:17:24.250 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T17:17:24.251+0000 7f2c3013d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34027) UID: 0 2024-04-25T17:17:24.250 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T17:17:24.251+0000 7f55ebc07640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34027) UID: 0 2024-04-25T17:17:24.251 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T17:17:24.251+0000 7f0f14e8e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34027) UID: 0 2024-04-25T17:17:24.254 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T17:17:24.251+0000 7f7fd7c37640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34027) UID: 0 2024-04-25T17:17:24.574 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T17:17:24.571+0000 7f837496c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 116007) UID: 0 2024-04-25T17:17:24.574 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T17:17:24.571+0000 7efca5c83640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 116007) UID: 0 2024-04-25T17:17:24.575 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T17:17:24.571+0000 7f9a177b0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 116007) UID: 0 2024-04-25T17:17:24.586 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T17:17:24.571+0000 7f3801006640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 116007) UID: 0 2024-04-25T17:27:07.278 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:27:07.280 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:27:07.387 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T17:27:07.381+0000 7f837496c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 117003) UID: 0 2024-04-25T17:27:07.387 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T17:27:07.381+0000 7f3801006640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 117003) UID: 0 2024-04-25T17:27:07.387 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T17:27:07.381+0000 7efca5c83640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 117003) UID: 0 2024-04-25T17:27:07.388 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T17:27:07.381+0000 7f9a177b0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 117003) UID: 0 2024-04-25T17:27:07.579 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T17:27:07.575+0000 7f0f14e8e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34131) UID: 0 2024-04-25T17:27:07.579 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T17:27:07.575+0000 7f7fd7c37640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34131) UID: 0 2024-04-25T17:27:07.580 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T17:27:07.575+0000 7f2c3013d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34131) UID: 0 2024-04-25T17:27:07.582 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T17:27:07.575+0000 7f55ebc07640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34131) UID: 0 2024-04-25T17:31:40.465 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T17:31:40.465+0000 7f0ef5c3d640 -1 bdev(0x560cf2982900 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2024-04-25T17:33:18.560 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:33:18.563 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:33:18.760 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T17:33:18.759+0000 7f2c3013d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34320) UID: 0 2024-04-25T17:33:18.761 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T17:33:18.759+0000 7f7fd7c37640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34320) UID: 0 2024-04-25T17:33:18.761 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T17:33:18.759+0000 7f0f14e8e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34320) UID: 0 2024-04-25T17:33:18.761 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T17:33:18.759+0000 7f55ebc07640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34320) UID: 0 2024-04-25T17:38:41.021 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:38:41.024 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:38:41.052 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T17:38:41.045+0000 7f9a177b0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 118040) UID: 0 2024-04-25T17:38:41.052 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T17:38:41.045+0000 7f837496c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 118040) UID: 0 2024-04-25T17:38:41.052 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T17:38:41.045+0000 7efca5c83640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 118040) UID: 0 2024-04-25T17:38:41.053 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T17:38:41.045+0000 7f3801006640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 118040) UID: 0 2024-04-25T17:38:41.073 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T17:38:41.072+0000 7f7fd7c37640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34450) UID: 0 2024-04-25T17:38:41.074 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T17:38:41.072+0000 7f55ebc07640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34450) UID: 0 2024-04-25T17:38:41.074 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T17:38:41.072+0000 7f2c3013d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34450) UID: 0 2024-04-25T17:38:41.074 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T17:38:41.072+0000 7f0f14e8e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 34450) UID: 0 2024-04-25T17:39:40.314 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T17:39:40.311+0000 7f2c11eee640 -1 bdev(0x556c0f264900 /var/lib/ceph/osd/ceph-6/block) aio_submit retries 2 2024-04-25T17:41:14.709 INFO:teuthology.orchestra.run.smithi094.stdout:generic/074 2312s 2024-04-25T17:41:15.200 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/074 2024-04-25T17:41:15.206 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:41:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:41:15.339 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:41:15.339 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2024-04-25T17:41:17.659 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:41:17.665 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:41:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:generic/075 71s 2024-04-25T17:42:28.830 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/075 2024-04-25T17:42:28.830 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:28.833 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:29.006 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:29.006 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2024-04-25T17:42:29.395 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:29.399 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:29.399 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:29.514 INFO:teuthology.orchestra.run.smithi094.stdout:generic/076 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:42:29.517 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/076 2024-04-25T17:42:29.517 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/076 2024-04-25T17:42:29.517 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:29.520 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:29.528 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:29.528 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2024-04-25T17:42:29.781 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:29.786 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:29.786 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:31.212 INFO:teuthology.orchestra.run.smithi094.stdout:generic/077 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:42:31.217 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/077 2024-04-25T17:42:31.217 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/077 2024-04-25T17:42:31.217 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:31.222 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:31.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:31.234 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2024-04-25T17:42:31.442 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:31.446 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:31.446 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:31.607 INFO:teuthology.orchestra.run.smithi094.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2024-04-25T17:42:31.611 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/078 2024-04-25T17:42:31.611 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/078 2024-04-25T17:42:31.611 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:31.614 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:31.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:31.623 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2024-04-25T17:42:32.172 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:32.176 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:32.176 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:32.339 INFO:teuthology.orchestra.run.smithi094.stdout:generic/079 [not run] file system doesn't support chattr +ia 2024-04-25T17:42:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/079 2024-04-25T17:42:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/079 2024-04-25T17:42:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:32.346 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:32.354 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:32.354 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2024-04-25T17:42:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:32.541 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:generic/080 2s 2024-04-25T17:42:34.751 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/080 2024-04-25T17:42:34.751 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:34.804 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:34.804 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2024-04-25T17:42:35.028 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:35.032 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:generic/081 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:42:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/081 2024-04-25T17:42:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/081 2024-04-25T17:42:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:35.169 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:35.169 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2024-04-25T17:42:35.364 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:35.368 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:35.479 INFO:teuthology.orchestra.run.smithi094.stdout:generic/082 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:42:35.482 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/082 2024-04-25T17:42:35.482 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/082 2024-04-25T17:42:35.482 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:35.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:35.493 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2024-04-25T17:42:35.671 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:35.675 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:35.777 INFO:teuthology.orchestra.run.smithi094.stdout:generic/083 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:42:35.780 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/083 2024-04-25T17:42:35.780 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/083 2024-04-25T17:42:35.780 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:35.791 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:35.791 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2024-04-25T17:42:35.970 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:35.974 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:36.075 INFO:teuthology.orchestra.run.smithi094.stdout:generic/084 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:42:36.079 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/084 2024-04-25T17:42:36.079 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/084 2024-04-25T17:42:36.079 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:36.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:36.089 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2024-04-25T17:42:36.265 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:36.269 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:36.407 INFO:teuthology.orchestra.run.smithi094.stdout:generic/085 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:42:36.411 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/085 2024-04-25T17:42:36.411 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/085 2024-04-25T17:42:36.411 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:36.421 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:36.422 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2024-04-25T17:42:36.604 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:36.608 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:36.608 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:41.246 INFO:teuthology.orchestra.run.smithi094.stdout:generic/086 5s 2024-04-25T17:42:41.403 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/086 2024-04-25T17:42:41.403 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:41.409 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:41.506 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:41.507 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2024-04-25T17:42:43.814 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:43.818 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:43.818 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:44.218 INFO:teuthology.orchestra.run.smithi094.stdout:generic/087 1s 2024-04-25T17:42:44.223 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/087 2024-04-25T17:42:44.223 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:44.225 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:44.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:44.234 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2024-04-25T17:42:44.786 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:44.790 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:44.790 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:44.982 INFO:teuthology.orchestra.run.smithi094.stdout:generic/088 0s 2024-04-25T17:42:44.987 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/088 2024-04-25T17:42:44.988 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:42:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:42:44.998 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:42:44.998 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2024-04-25T17:42:45.538 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:42:45.542 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:42:45.542 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:45:29.247 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:45:29.250 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:45:30.052 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T17:45:30.051+0000 7f837496c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125386) UID: 0 2024-04-25T17:45:30.053 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T17:45:30.051+0000 7f9a177b0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125386) UID: 0 2024-04-25T17:45:30.053 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T17:45:30.051+0000 7efca5c83640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125386) UID: 0 2024-04-25T17:45:30.053 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T17:45:30.051+0000 7f3801006640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125386) UID: 0 2024-04-25T17:50:28.131 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:50:28.134 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T17:50:30.690 INFO:tasks.ceph.mds.a.smithi094.stderr:2024-04-25T17:50:30.687+0000 7fb5271b5640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 125613) UID: 0 2024-04-25T17:50:30.691 INFO:tasks.ceph.mds.c.smithi094.stderr:2024-04-25T17:50:30.687+0000 7f3de9731640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 125613) UID: 0 2024-04-25T17:57:59.856 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T17:57:59.855+0000 7f9a177b0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 127640) UID: 0 2024-04-25T17:57:59.857 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T17:57:59.855+0000 7f837496c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 127640) UID: 0 2024-04-25T17:57:59.857 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T17:57:59.855+0000 7efca5c83640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 127640) UID: 0 2024-04-25T17:57:59.857 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T17:57:59.855+0000 7f3801006640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 127640) UID: 0 2024-04-25T17:59:32.238 INFO:teuthology.orchestra.run.smithi094.stdout:generic/089 1006s 2024-04-25T17:59:32.700 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/089 2024-04-25T17:59:32.700 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:59:32.706 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:59:32.722 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:59:32.723 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2024-04-25T17:59:37.618 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:59:37.624 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:59:37.624 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:59:38.755 INFO:teuthology.orchestra.run.smithi094.stdout:generic/090 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T17:59:38.761 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/090 2024-04-25T17:59:38.762 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/090 2024-04-25T17:59:38.762 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T17:59:38.766 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T17:59:38.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T17:59:38.779 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2024-04-25T17:59:39.012 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T17:59:39.017 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T17:59:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:01:45.338 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:01:45.341 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:01:45.358 INFO:tasks.ceph.mds.a.smithi094.stderr:2024-04-25T18:01:45.356+0000 7fb5271b5640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 128526) UID: 0 2024-04-25T18:01:45.358 INFO:tasks.ceph.mds.c.smithi094.stderr:2024-04-25T18:01:45.356+0000 7f3de9731640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 128526) UID: 0 2024-04-25T18:04:13.274 INFO:teuthology.orchestra.run.smithi094.stdout:generic/091 274s 2024-04-25T18:04:13.286 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/091 2024-04-25T18:04:13.286 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:13.298 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:13.299 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2024-04-25T18:04:13.896 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:13.900 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:13.900 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:14.131 INFO:teuthology.orchestra.run.smithi094.stdout:generic/092 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-25T18:04:14.135 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/092 2024-04-25T18:04:14.135 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/092 2024-04-25T18:04:14.135 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:14.137 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:14.146 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:14.146 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2024-04-25T18:04:14.749 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:14.753 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:14.754 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:15.701 INFO:teuthology.orchestra.run.smithi094.stdout:generic/093 1s 2024-04-25T18:04:15.706 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/093 2024-04-25T18:04:15.706 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:15.717 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:15.717 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2024-04-25T18:04:16.281 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:16.285 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:16.388 INFO:teuthology.orchestra.run.smithi094.stdout:generic/094 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:04:16.391 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/094 2024-04-25T18:04:16.392 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/094 2024-04-25T18:04:16.392 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:16.403 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:16.403 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2024-04-25T18:04:16.582 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:16.586 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:16.692 INFO:teuthology.orchestra.run.smithi094.stdout:generic/095 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:04:16.696 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/095 2024-04-25T18:04:16.696 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/095 2024-04-25T18:04:16.696 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:16.705 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:16.706 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2024-04-25T18:04:16.885 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:16.889 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:16.889 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:17.000 INFO:teuthology.orchestra.run.smithi094.stdout:generic/096 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:04:17.003 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/096 2024-04-25T18:04:17.003 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/096 2024-04-25T18:04:17.003 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:17.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:17.014 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2024-04-25T18:04:17.194 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:17.198 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:17.926 INFO:teuthology.orchestra.run.smithi094.stdout:generic/097 0s 2024-04-25T18:04:17.931 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/097 2024-04-25T18:04:17.931 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:17.941 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:17.942 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2024-04-25T18:04:18.127 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:18.131 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:18.131 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:18.244 INFO:teuthology.orchestra.run.smithi094.stdout:generic/098 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:04:18.247 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/098 2024-04-25T18:04:18.247 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/098 2024-04-25T18:04:18.248 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:18.250 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:18.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:18.258 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-04-25T18:04:18.436 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:18.440 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:18.441 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:18.588 INFO:teuthology.orchestra.run.smithi094.stdout:generic/099 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-25T18:04:18.591 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/099 2024-04-25T18:04:18.591 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/099 2024-04-25T18:04:18.591 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:18.594 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:18.601 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:18.601 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2024-04-25T18:04:18.779 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:18.783 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:18.783 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:50.213 INFO:teuthology.orchestra.run.smithi094.stdout:generic/100 31s 2024-04-25T18:04:50.234 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/100 2024-04-25T18:04:50.235 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:50.238 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:50.256 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:50.256 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2024-04-25T18:04:51.002 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:51.008 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:51.008 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:51.934 INFO:teuthology.orchestra.run.smithi094.stdout:generic/101 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:04:51.940 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/101 2024-04-25T18:04:51.940 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/101 2024-04-25T18:04:51.940 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:51.943 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:51.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:51.954 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2024-04-25T18:04:52.154 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:52.158 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:52.158 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:52.509 INFO:teuthology.orchestra.run.smithi094.stdout:generic/102 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:04:52.513 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/102 2024-04-25T18:04:52.513 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/102 2024-04-25T18:04:52.514 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:52.527 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:52.527 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2024-04-25T18:04:52.720 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:52.724 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:52.882 INFO:teuthology.orchestra.run.smithi094.stdout:generic/103 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:04:52.887 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/103 2024-04-25T18:04:52.887 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/103 2024-04-25T18:04:52.887 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:52.898 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:52.899 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2024-04-25T18:04:53.089 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:53.093 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:53.311 INFO:teuthology.orchestra.run.smithi094.stdout:generic/104 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:04:53.315 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/104 2024-04-25T18:04:53.315 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/104 2024-04-25T18:04:53.315 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:53.326 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:53.326 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2024-04-25T18:04:53.523 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:53.527 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:53.641 INFO:teuthology.orchestra.run.smithi094.stdout:generic/105 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:04:53.645 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/105 2024-04-25T18:04:53.645 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/105 2024-04-25T18:04:53.645 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:53.648 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:53.655 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:53.656 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2024-04-25T18:04:53.844 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:53.848 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:53.848 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:53.986 INFO:teuthology.orchestra.run.smithi094.stdout:generic/106 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:04:53.990 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/106 2024-04-25T18:04:53.990 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/106 2024-04-25T18:04:53.990 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:53.992 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:54.000 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:54.000 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2024-04-25T18:04:54.187 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:54.191 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:54.191 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:54.306 INFO:teuthology.orchestra.run.smithi094.stdout:generic/107 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:04:54.310 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/107 2024-04-25T18:04:54.310 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/107 2024-04-25T18:04:54.310 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:54.320 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:54.321 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2024-04-25T18:04:54.504 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:54.508 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:54.508 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:54.864 INFO:teuthology.orchestra.run.smithi094.stdout:generic/108 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:04:54.869 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/108 2024-04-25T18:04:54.869 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/108 2024-04-25T18:04:54.869 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:54.883 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:54.883 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2024-04-25T18:04:55.082 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:55.086 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:55.086 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:55.195 INFO:teuthology.orchestra.run.smithi094.stdout:generic/109 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:04:55.199 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/109 2024-04-25T18:04:55.200 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/109 2024-04-25T18:04:55.200 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:55.210 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:55.211 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2024-04-25T18:04:55.397 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:55.401 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:55.401 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:55.570 INFO:teuthology.orchestra.run.smithi094.stdout:generic/110 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:04:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/110 2024-04-25T18:04:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/110 2024-04-25T18:04:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:55.576 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:55.584 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:55.584 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2024-04-25T18:04:56.140 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:56.144 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:56.144 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:56.299 INFO:teuthology.orchestra.run.smithi094.stdout:generic/111 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:04:56.303 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/111 2024-04-25T18:04:56.303 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/111 2024-04-25T18:04:56.303 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:04:56.305 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:04:56.313 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:04:56.313 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2024-04-25T18:04:56.888 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:04:56.891 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:04:56.892 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:06:19.849 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:06:19.852 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:06:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:generic/112 94s 2024-04-25T18:06:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/112 2024-04-25T18:06:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:06:30.953 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:06:30.964 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:06:30.965 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2024-04-25T18:06:31.563 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:06:31.568 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:06:31.568 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:06:49.918 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:06:49.921 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:07:20.067 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:07:20.069 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:07:20.668 INFO:teuthology.orchestra.run.smithi094.stdout:generic/113 49s 2024-04-25T18:07:20.674 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/113 2024-04-25T18:07:20.674 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:07:20.677 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:20.688 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:07:20.688 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2024-04-25T18:07:21.263 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:07:21.267 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:07:21.267 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:21.425 INFO:teuthology.orchestra.run.smithi094.stdout:generic/114 [not run] Sparse files are not supported or do not work as expected 2024-04-25T18:07:21.428 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/114 2024-04-25T18:07:21.428 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/114 2024-04-25T18:07:21.428 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:07:21.431 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:21.438 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:07:21.439 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2024-04-25T18:07:21.682 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:07:21.686 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:07:21.686 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:21.847 INFO:teuthology.orchestra.run.smithi094.stdout:generic/115 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:07:21.851 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/115 2024-04-25T18:07:21.851 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/115 2024-04-25T18:07:21.851 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:07:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:21.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:07:21.861 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2024-04-25T18:07:22.407 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:07:22.411 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:07:22.411 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:22.566 INFO:teuthology.orchestra.run.smithi094.stdout:generic/116 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:07:22.570 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/116 2024-04-25T18:07:22.570 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/116 2024-04-25T18:07:22.570 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:07:22.573 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:22.581 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:07:22.581 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2024-04-25T18:07:23.106 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:07:23.110 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:07:23.110 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:23.221 INFO:teuthology.orchestra.run.smithi094.stdout:generic/117 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:07:23.225 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/117 2024-04-25T18:07:23.225 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/117 2024-04-25T18:07:23.225 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:07:23.227 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:23.236 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:07:23.236 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2024-04-25T18:07:23.420 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:07:23.424 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:07:23.424 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:23.582 INFO:teuthology.orchestra.run.smithi094.stdout:generic/118 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:07:23.585 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/118 2024-04-25T18:07:23.586 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/118 2024-04-25T18:07:23.586 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:07:23.588 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:23.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:07:23.595 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2024-04-25T18:07:24.143 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:07:24.147 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:07:24.147 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:24.304 INFO:teuthology.orchestra.run.smithi094.stdout:generic/119 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:07:24.308 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/119 2024-04-25T18:07:24.308 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/119 2024-04-25T18:07:24.308 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:07:24.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:24.318 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:07:24.319 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2024-04-25T18:07:24.873 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:07:24.877 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:07:24.877 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:24.977 INFO:teuthology.orchestra.run.smithi094.stdout:generic/120 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:07:24.980 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/120 2024-04-25T18:07:24.981 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/120 2024-04-25T18:07:24.981 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:07:24.983 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:24.991 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:07:24.991 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2024-04-25T18:07:25.177 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:07:25.181 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:07:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:25.336 INFO:teuthology.orchestra.run.smithi094.stdout:generic/121 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-25T18:07:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/121 2024-04-25T18:07:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/121 2024-04-25T18:07:25.341 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:07:25.343 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:25.351 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:07:25.351 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2024-04-25T18:07:25.898 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:07:25.902 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:07:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:26.061 INFO:teuthology.orchestra.run.smithi094.stdout:generic/122 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-25T18:07:26.065 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/122 2024-04-25T18:07:26.065 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/122 2024-04-25T18:07:26.065 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:07:26.067 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:26.075 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:07:26.075 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2024-04-25T18:07:26.616 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:07:26.620 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:07:26.620 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:27.231 INFO:teuthology.orchestra.run.smithi094.stdout:generic/123 1s 2024-04-25T18:07:27.236 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/123 2024-04-25T18:07:27.236 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:07:27.239 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:27.247 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:07:27.248 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2024-04-25T18:07:27.793 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:07:27.797 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:07:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:27.901 INFO:teuthology.orchestra.run.smithi094.stdout:generic/124 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:07:27.905 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/124 2024-04-25T18:07:27.905 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/124 2024-04-25T18:07:27.905 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:07:27.907 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:27.915 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:07:27.915 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2024-04-25T18:07:28.098 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:07:28.103 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:07:28.103 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:07:50.205 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:07:50.209 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:08:20.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:08:20.367 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:08:29.688 INFO:teuthology.orchestra.run.smithi094.stdout:generic/125 61s 2024-04-25T18:08:29.698 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/125 2024-04-25T18:08:29.698 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:08:29.703 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:08:29.716 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:08:29.716 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2024-04-25T18:08:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:08:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:08:30.307 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:08:31.132 INFO:teuthology.orchestra.run.smithi094.stdout:generic/126 - output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/126.out.bad) 2024-04-25T18:08:31.134 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/126.out 2024-04-25 16:52:50.075099990 +0000 2024-04-25T18:08:31.134 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/126.out.bad 2024-04-25 18:08:31.126041225 +0000 2024-04-25T18:08:31.134 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -1,7 +1,7 @@ 2024-04-25T18:08:31.134 INFO:teuthology.orchestra.run.smithi094.stdout: QA output created by 126 2024-04-25T18:08:31.134 INFO:teuthology.orchestra.run.smithi094.stdout: -x a 001 file owned by (99/99) as user/group(12/100) PASS 2024-04-25T18:08:31.134 INFO:teuthology.orchestra.run.smithi094.stdout: -x a 010 file owned by (99/99) as user/group(200/99) PASS 2024-04-25T18:08:31.134 INFO:teuthology.orchestra.run.smithi094.stdout: -x a 100 file owned by (99/99) as user/group(99/500) PASS 2024-04-25T18:08:31.134 INFO:teuthology.orchestra.run.smithi094.stdout: +x a 001 file owned by (99/99) as user/group(12/100) FAIL 2024-04-25T18:08:31.134 INFO:teuthology.orchestra.run.smithi094.stdout: +x a 010 file owned by (99/99) as user/group(200/99) FAIL 2024-04-25T18:08:31.134 INFO:teuthology.orchestra.run.smithi094.stdout: +x a 100 file owned by (99/99) as user/group(99/500) FAIL 2024-04-25T18:08:31.134 INFO:teuthology.orchestra.run.smithi094.stdout: w a 002 file owned by (99/99) as user/group(12/100) PASS 2024-04-25T18:08:31.134 INFO:teuthology.orchestra.run.smithi094.stdout: w a 020 file owned by (99/99) as user/group(200/99) PASS 2024-04-25T18:08:31.135 INFO:teuthology.orchestra.run.smithi094.stdout: w a 200 file owned by (99/99) as user/group(99/500) PASS 2024-04-25T18:08:31.136 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/126 2024-04-25T18:08:31.136 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/126 2024-04-25T18:08:31.136 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-25T18:08:31.139 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:08:31.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:08:31.147 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-25T18:08:31.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-25T18:08:31.147 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2024-04-25T18:08:31.720 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:08:31.724 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:08:31.724 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:08:50.438 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:08:50.442 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:09:20.591 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:09:20.598 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:09:50.657 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:09:50.660 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:10:20.747 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:10:20.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:10:50.803 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:10:50.806 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:11:20.864 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:11:20.868 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:11:50.946 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:11:50.949 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:12:21.019 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:12:21.022 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:12:51.249 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:12:51.252 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:13:21.315 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:13:21.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:13:51.697 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:13:51.699 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:14:21.760 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:14:21.764 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:14:51.828 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:14:51.831 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:15:21.889 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:15:21.892 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:15:52.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:15:52.027 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:16:22.076 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:16:22.080 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:16:52.173 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:16:52.176 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:17:22.314 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:17:22.317 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:17:52.689 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:17:52.693 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:18:22.752 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:18:22.755 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:18:52.803 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:18:52.806 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:19:22.852 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:19:22.855 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:19:52.995 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:19:52.998 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:20:23.076 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:20:23.079 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:20:53.324 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:20:53.327 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:21:23.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:21:23.374 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:21:53.486 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:21:53.490 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:22:23.746 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:22:23.750 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:22:53.827 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:22:53.830 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:23:23.938 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:23:23.941 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:23:53.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:23:53.997 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:24:24.100 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:24:24.104 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:24:54.207 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:24:54.210 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:25:24.265 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:25:24.268 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:25:54.385 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:25:54.388 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:26:24.439 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:26:24.443 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:26:54.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:26:54.859 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:27:24.941 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:27:24.944 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:27:55.034 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:27:55.038 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:28:25.223 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:28:25.226 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:28:55.555 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:28:55.558 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:29:25.660 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:29:25.663 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:29:55.724 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:29:55.727 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:30:25.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:30:25.808 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:30:56.028 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:30:56.031 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:31:26.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:31:26.146 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:31:56.264 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:31:56.268 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:32:26.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:32:26.341 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:32:56.385 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:32:56.389 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:33:26.524 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:33:26.527 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:33:56.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:33:56.609 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:34:26.704 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:34:26.707 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:34:56.828 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:34:56.831 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:35:26.885 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:35:26.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:35:56.943 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:35:56.947 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:36:27.269 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:36:27.272 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:36:57.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:36:57.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:37:27.398 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:37:27.402 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:37:57.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:37:57.575 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:38:27.645 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:38:27.648 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:38:57.758 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:38:57.762 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:39:27.890 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:39:27.893 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:39:57.936 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:39:57.940 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:40:28.101 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:40:28.105 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:40:58.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:40:58.169 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:41:28.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:41:28.360 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:41:58.402 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:41:58.406 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:42:28.643 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:42:28.647 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:42:58.682 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:42:58.686 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:43:05.697 INFO:teuthology.orchestra.run.smithi094.stdout:generic/127 2074s 2024-04-25T18:43:05.705 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/127 2024-04-25T18:43:05.705 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:43:05.708 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:43:05.721 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:43:05.721 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2024-04-25T18:43:06.309 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:43:06.313 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:43:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:43:06.429 INFO:teuthology.orchestra.run.smithi094.stdout:generic/128 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:43:06.434 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/128 2024-04-25T18:43:06.434 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/128 2024-04-25T18:43:06.434 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:43:06.437 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:43:06.446 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:43:06.446 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2024-04-25T18:43:06.652 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:43:06.657 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:43:06.657 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:43:06.764 INFO:teuthology.orchestra.run.smithi094.stdout:generic/129 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:43:06.768 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/129 2024-04-25T18:43:06.768 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/129 2024-04-25T18:43:06.768 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:43:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:43:06.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:43:06.778 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2024-04-25T18:43:06.960 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:43:06.964 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:43:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:43:07.067 INFO:teuthology.orchestra.run.smithi094.stdout:generic/130 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:43:07.071 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/130 2024-04-25T18:43:07.071 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/130 2024-04-25T18:43:07.071 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:43:07.073 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:43:07.081 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:43:07.082 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2024-04-25T18:43:07.265 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:43:07.269 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:43:07.269 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:43:08.623 INFO:teuthology.orchestra.run.smithi094.stdout:generic/131 1s 2024-04-25T18:43:08.629 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/131 2024-04-25T18:43:08.629 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:43:08.632 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:43:08.641 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:43:08.642 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2024-04-25T18:43:09.184 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:43:09.188 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:43:09.188 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:43:09.290 INFO:teuthology.orchestra.run.smithi094.stdout:generic/132 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:43:09.293 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/132 2024-04-25T18:43:09.293 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/132 2024-04-25T18:43:09.293 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:43:09.295 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:43:09.304 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:43:09.304 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2024-04-25T18:43:09.481 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:43:09.485 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:43:09.486 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:43:28.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:43:28.796 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:43:58.878 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:43:58.881 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:44:28.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:44:28.996 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:44:59.092 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:44:59.095 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:45:29.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:45:29.410 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:45:29.446 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T18:45:29.446+0000 7f2c3013d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35050) UID: 0 2024-04-25T18:45:29.446 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T18:45:29.446+0000 7f55ebc07640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35050) UID: 0 2024-04-25T18:45:29.446 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T18:45:29.446+0000 7f7fd7c37640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35050) UID: 0 2024-04-25T18:45:29.447 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T18:45:29.446+0000 7f0f14e8e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35050) UID: 0 2024-04-25T18:48:20.221 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:48:20.225 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:48:20.501 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T18:48:20.498+0000 7f2c3013d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35061) UID: 0 2024-04-25T18:48:20.501 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T18:48:20.498+0000 7f7fd7c37640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35061) UID: 0 2024-04-25T18:48:20.501 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T18:48:20.498+0000 7f55ebc07640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35061) UID: 0 2024-04-25T18:48:20.502 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T18:48:20.498+0000 7f0f14e8e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35061) UID: 0 2024-04-25T18:51:12.153 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:51:12.157 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:51:42.289 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:51:42.292 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:52:12.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:52:12.400 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:52:30.571 INFO:teuthology.orchestra.run.smithi094.stdout:generic/133 561s 2024-04-25T18:52:30.577 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/133 2024-04-25T18:52:30.577 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:30.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:30.595 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2024-04-25T18:52:31.319 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:31.323 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:31.323 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:31.479 INFO:teuthology.orchestra.run.smithi094.stdout:generic/134 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:31.482 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/134 2024-04-25T18:52:31.483 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/134 2024-04-25T18:52:31.483 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:31.485 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:31.493 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:31.493 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2024-04-25T18:52:32.008 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:32.012 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:32.012 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:32.208 INFO:teuthology.orchestra.run.smithi094.stdout:generic/135 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:32.213 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/135 2024-04-25T18:52:32.213 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/135 2024-04-25T18:52:32.213 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:32.217 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:32.231 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:32.231 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2024-04-25T18:52:32.420 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:32.424 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:32.424 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:32.582 INFO:teuthology.orchestra.run.smithi094.stdout:generic/136 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/136 2024-04-25T18:52:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/136 2024-04-25T18:52:32.586 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:32.596 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:32.596 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2024-04-25T18:52:33.136 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:33.140 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:33.141 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:33.293 INFO:teuthology.orchestra.run.smithi094.stdout:generic/137 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:33.296 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/137 2024-04-25T18:52:33.296 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/137 2024-04-25T18:52:33.296 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:33.299 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:33.307 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:33.307 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2024-04-25T18:52:33.867 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:33.868 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:33.869 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:34.020 INFO:teuthology.orchestra.run.smithi094.stdout:generic/138 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:34.023 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/138 2024-04-25T18:52:34.023 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/138 2024-04-25T18:52:34.024 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:34.034 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:34.034 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2024-04-25T18:52:34.575 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:34.579 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:34.579 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:34.731 INFO:teuthology.orchestra.run.smithi094.stdout:generic/139 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:34.734 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/139 2024-04-25T18:52:34.734 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/139 2024-04-25T18:52:34.734 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:34.736 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:34.744 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:34.744 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2024-04-25T18:52:35.289 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:35.293 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:generic/140 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:35.442 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/140 2024-04-25T18:52:35.442 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/140 2024-04-25T18:52:35.442 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:35.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:35.452 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2024-04-25T18:52:35.983 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:35.988 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:36.093 INFO:teuthology.orchestra.run.smithi094.stdout:generic/141 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:36.096 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/141 2024-04-25T18:52:36.096 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/141 2024-04-25T18:52:36.097 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:36.107 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:36.107 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2024-04-25T18:52:36.283 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:36.287 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:36.438 INFO:teuthology.orchestra.run.smithi094.stdout:generic/142 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:36.441 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/142 2024-04-25T18:52:36.442 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/142 2024-04-25T18:52:36.442 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:36.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:36.452 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2024-04-25T18:52:36.985 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:36.990 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:36.990 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:37.146 INFO:teuthology.orchestra.run.smithi094.stdout:generic/143 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:37.148 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/143 2024-04-25T18:52:37.149 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/143 2024-04-25T18:52:37.149 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:37.151 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:37.159 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:37.159 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2024-04-25T18:52:37.709 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:37.713 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:37.713 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:37.860 INFO:teuthology.orchestra.run.smithi094.stdout:generic/144 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:37.863 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/144 2024-04-25T18:52:37.863 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/144 2024-04-25T18:52:37.863 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:37.866 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:37.873 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:37.873 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2024-04-25T18:52:38.427 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:38.431 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:38.431 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:38.584 INFO:teuthology.orchestra.run.smithi094.stdout:generic/145 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:38.587 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/145 2024-04-25T18:52:38.587 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/145 2024-04-25T18:52:38.587 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:38.589 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:38.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:38.598 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2024-04-25T18:52:39.123 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:39.127 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:39.127 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:39.275 INFO:teuthology.orchestra.run.smithi094.stdout:generic/146 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:39.279 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/146 2024-04-25T18:52:39.279 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/146 2024-04-25T18:52:39.279 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:39.281 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:39.289 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:39.290 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2024-04-25T18:52:39.818 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:39.822 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:39.823 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:39.973 INFO:teuthology.orchestra.run.smithi094.stdout:generic/147 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:39.977 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/147 2024-04-25T18:52:39.977 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/147 2024-04-25T18:52:39.977 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:39.979 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:39.987 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:39.987 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2024-04-25T18:52:40.603 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:40.607 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:40.607 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:40.791 INFO:teuthology.orchestra.run.smithi094.stdout:generic/148 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:40.794 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/148 2024-04-25T18:52:40.795 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/148 2024-04-25T18:52:40.795 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:40.797 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:40.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:40.805 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2024-04-25T18:52:41.348 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:41.352 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:41.501 INFO:teuthology.orchestra.run.smithi094.stdout:generic/149 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:41.504 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/149 2024-04-25T18:52:41.505 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/149 2024-04-25T18:52:41.505 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:41.507 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:41.514 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:41.515 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2024-04-25T18:52:42.038 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:42.042 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:42.042 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:42.193 INFO:teuthology.orchestra.run.smithi094.stdout:generic/150 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:42.197 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/150 2024-04-25T18:52:42.197 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/150 2024-04-25T18:52:42.197 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:42.199 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:42.207 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:42.208 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2024-04-25T18:52:42.511 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:52:42.518 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:52:42.734 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:42.738 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:42.739 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:42.889 INFO:teuthology.orchestra.run.smithi094.stdout:generic/151 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:42.893 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/151 2024-04-25T18:52:42.893 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/151 2024-04-25T18:52:42.893 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:42.895 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:42.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:42.904 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2024-04-25T18:52:43.454 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:43.458 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:43.458 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:43.612 INFO:teuthology.orchestra.run.smithi094.stdout:generic/152 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:43.616 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/152 2024-04-25T18:52:43.616 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/152 2024-04-25T18:52:43.616 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:43.626 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:43.626 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2024-04-25T18:52:44.161 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:44.165 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:44.165 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:44.312 INFO:teuthology.orchestra.run.smithi094.stdout:generic/153 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:44.316 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/153 2024-04-25T18:52:44.316 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/153 2024-04-25T18:52:44.316 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:44.318 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:44.325 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:44.325 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2024-04-25T18:52:44.863 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:44.867 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:44.867 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:45.016 INFO:teuthology.orchestra.run.smithi094.stdout:generic/154 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:45.019 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/154 2024-04-25T18:52:45.019 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/154 2024-04-25T18:52:45.020 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:45.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:45.030 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2024-04-25T18:52:45.568 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:45.572 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:45.572 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:45.721 INFO:teuthology.orchestra.run.smithi094.stdout:generic/155 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:45.724 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/155 2024-04-25T18:52:45.724 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/155 2024-04-25T18:52:45.725 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:45.727 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:45.735 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:45.735 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2024-04-25T18:52:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:46.259 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:46.403 INFO:teuthology.orchestra.run.smithi094.stdout:generic/156 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:46.406 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/156 2024-04-25T18:52:46.407 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/156 2024-04-25T18:52:46.407 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:46.409 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:46.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:46.417 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2024-04-25T18:52:46.965 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:46.969 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:47.120 INFO:teuthology.orchestra.run.smithi094.stdout:generic/157 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:47.124 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/157 2024-04-25T18:52:47.124 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/157 2024-04-25T18:52:47.124 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:47.126 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:47.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:47.135 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2024-04-25T18:52:47.663 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:47.667 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:47.667 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:47.821 INFO:teuthology.orchestra.run.smithi094.stdout:generic/158 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:47.825 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/158 2024-04-25T18:52:47.825 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/158 2024-04-25T18:52:47.825 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:47.827 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:47.835 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:47.836 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2024-04-25T18:52:48.359 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:48.363 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:48.363 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:48.520 INFO:teuthology.orchestra.run.smithi094.stdout:generic/159 [not run] file system doesn't support chattr +i 2024-04-25T18:52:48.523 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/159 2024-04-25T18:52:48.524 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/159 2024-04-25T18:52:48.524 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:48.534 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:48.534 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2024-04-25T18:52:48.715 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:48.718 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:48.719 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:48.856 INFO:teuthology.orchestra.run.smithi094.stdout:generic/160 [not run] file system doesn't support chattr +i 2024-04-25T18:52:48.860 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/160 2024-04-25T18:52:48.860 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/160 2024-04-25T18:52:48.860 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:48.863 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:48.871 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:48.871 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2024-04-25T18:52:49.047 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:49.051 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:49.051 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:generic/161 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:49.158 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/161 2024-04-25T18:52:49.158 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/161 2024-04-25T18:52:49.158 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:49.160 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:49.168 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:49.168 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2024-04-25T18:52:49.343 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:49.347 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:49.347 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:49.456 INFO:teuthology.orchestra.run.smithi094.stdout:generic/162 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:49.460 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/162 2024-04-25T18:52:49.460 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/162 2024-04-25T18:52:49.460 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:49.462 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:49.470 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:49.470 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2024-04-25T18:52:49.651 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:49.655 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:49.655 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:49.760 INFO:teuthology.orchestra.run.smithi094.stdout:generic/163 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:49.763 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/163 2024-04-25T18:52:49.763 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/163 2024-04-25T18:52:49.763 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:49.766 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:49.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:49.774 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2024-04-25T18:52:49.952 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:49.956 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:49.956 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:50.060 INFO:teuthology.orchestra.run.smithi094.stdout:generic/164 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:50.064 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/164 2024-04-25T18:52:50.064 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/164 2024-04-25T18:52:50.064 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:50.066 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:50.074 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:50.074 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2024-04-25T18:52:50.253 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:50.257 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:50.257 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:50.361 INFO:teuthology.orchestra.run.smithi094.stdout:generic/165 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/165 2024-04-25T18:52:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/165 2024-04-25T18:52:50.365 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:50.367 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:50.375 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:50.375 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2024-04-25T18:52:50.554 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:50.559 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:50.660 INFO:teuthology.orchestra.run.smithi094.stdout:generic/166 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:50.664 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/166 2024-04-25T18:52:50.664 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/166 2024-04-25T18:52:50.664 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:50.666 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:50.675 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:50.675 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2024-04-25T18:52:50.854 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:50.857 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:50.857 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:50.960 INFO:teuthology.orchestra.run.smithi094.stdout:generic/167 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:50.963 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/167 2024-04-25T18:52:50.963 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/167 2024-04-25T18:52:50.963 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:50.966 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:50.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:50.974 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2024-04-25T18:52:51.151 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:51.155 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:51.155 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:51.257 INFO:teuthology.orchestra.run.smithi094.stdout:generic/168 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:51.260 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/168 2024-04-25T18:52:51.260 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/168 2024-04-25T18:52:51.260 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:51.271 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:51.271 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2024-04-25T18:52:51.448 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:51.452 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:51.453 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:51.555 INFO:teuthology.orchestra.run.smithi094.stdout:generic/169 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:51.559 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/169 2024-04-25T18:52:51.559 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/169 2024-04-25T18:52:51.559 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:51.569 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:51.569 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2024-04-25T18:52:51.751 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:51.755 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:51.755 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:51.858 INFO:teuthology.orchestra.run.smithi094.stdout:generic/170 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:51.862 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/170 2024-04-25T18:52:51.862 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/170 2024-04-25T18:52:51.862 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:51.864 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:51.872 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:51.873 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2024-04-25T18:52:52.052 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:52.056 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:52.056 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:52.172 INFO:teuthology.orchestra.run.smithi094.stdout:generic/171 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:52.176 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/171 2024-04-25T18:52:52.176 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/171 2024-04-25T18:52:52.176 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:52.179 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:52.186 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:52.186 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2024-04-25T18:52:52.363 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:52.367 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:52.367 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:52.473 INFO:teuthology.orchestra.run.smithi094.stdout:generic/172 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:52.477 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/172 2024-04-25T18:52:52.477 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/172 2024-04-25T18:52:52.477 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:52.487 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:52.487 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2024-04-25T18:52:52.667 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:52.671 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:52.778 INFO:teuthology.orchestra.run.smithi094.stdout:generic/173 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:52.781 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/173 2024-04-25T18:52:52.782 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/173 2024-04-25T18:52:52.782 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:52.792 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:52.792 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2024-04-25T18:52:52.967 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:52.971 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:53.078 INFO:teuthology.orchestra.run.smithi094.stdout:generic/174 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:53.082 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/174 2024-04-25T18:52:53.082 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/174 2024-04-25T18:52:53.082 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:53.092 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:53.092 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2024-04-25T18:52:53.268 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:53.272 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:53.377 INFO:teuthology.orchestra.run.smithi094.stdout:generic/175 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:53.381 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/175 2024-04-25T18:52:53.381 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/175 2024-04-25T18:52:53.381 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:53.391 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:53.391 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2024-04-25T18:52:53.568 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:53.572 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:53.676 INFO:teuthology.orchestra.run.smithi094.stdout:generic/176 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:53.680 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/176 2024-04-25T18:52:53.680 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/176 2024-04-25T18:52:53.680 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:53.682 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:53.691 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:53.691 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2024-04-25T18:52:53.872 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:53.876 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:53.876 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:54.036 INFO:teuthology.orchestra.run.smithi094.stdout:generic/177 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:54.040 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/177 2024-04-25T18:52:54.040 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/177 2024-04-25T18:52:54.040 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:54.042 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:54.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:54.051 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2024-04-25T18:52:54.232 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:54.236 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:54.236 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:54.390 INFO:teuthology.orchestra.run.smithi094.stdout:generic/178 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:54.393 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/178 2024-04-25T18:52:54.394 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/178 2024-04-25T18:52:54.394 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:54.396 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:54.404 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:54.404 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2024-04-25T18:52:54.944 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:54.949 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:54.949 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:55.099 INFO:teuthology.orchestra.run.smithi094.stdout:generic/179 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:55.102 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/179 2024-04-25T18:52:55.102 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/179 2024-04-25T18:52:55.102 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:55.104 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:55.113 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:55.113 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2024-04-25T18:52:55.664 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:55.668 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:55.668 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:55.818 INFO:teuthology.orchestra.run.smithi094.stdout:generic/180 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:55.821 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/180 2024-04-25T18:52:55.822 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/180 2024-04-25T18:52:55.822 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:55.824 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:55.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:55.832 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2024-04-25T18:52:56.368 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:56.372 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:56.372 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:56.517 INFO:teuthology.orchestra.run.smithi094.stdout:generic/181 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:56.521 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/181 2024-04-25T18:52:56.521 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/181 2024-04-25T18:52:56.521 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:56.523 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:56.531 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:56.531 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2024-04-25T18:52:57.061 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:57.065 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:57.065 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:57.219 INFO:teuthology.orchestra.run.smithi094.stdout:generic/182 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-25T18:52:57.223 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/182 2024-04-25T18:52:57.223 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/182 2024-04-25T18:52:57.223 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:57.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:57.233 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2024-04-25T18:52:57.741 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:57.745 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:57.846 INFO:teuthology.orchestra.run.smithi094.stdout:generic/183 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:57.850 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/183 2024-04-25T18:52:57.850 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/183 2024-04-25T18:52:57.850 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:57.852 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:57.860 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:57.860 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2024-04-25T18:52:58.039 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:58.043 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:58.043 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:58.189 INFO:teuthology.orchestra.run.smithi094.stdout:generic/184 [failed, exit status 1]- output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/184.out.bad) 2024-04-25T18:52:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/184.out 2024-04-25 16:52:50.079099934 +0000 2024-04-25T18:52:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/184.out.bad 2024-04-25 18:52:58.184569711 +0000 2024-04-25T18:52:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -1 +1,2 @@ 2024-04-25T18:52:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: QA output created by 184 2024-04-25T18:52:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: +/tmp/tmp.FooT1Le6w4xfstests-dev/tests/generic/184: line 23: /tmp/tmp.PS50Hvgg8stest/null: Permission denied 2024-04-25T18:52:58.193 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/184 2024-04-25T18:52:58.194 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/184 2024-04-25T18:52:58.194 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-25T18:52:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:58.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:52:58.204 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-25T18:52:58.204 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-25T18:52:58.205 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2024-04-25T18:52:58.386 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:58.390 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:58.390 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:58.491 INFO:teuthology.orchestra.run.smithi094.stdout:generic/185 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:58.495 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/185 2024-04-25T18:52:58.495 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/185 2024-04-25T18:52:58.495 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:58.497 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:58.505 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:58.506 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2024-04-25T18:52:58.682 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:58.687 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:58.797 INFO:teuthology.orchestra.run.smithi094.stdout:generic/186 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:58.801 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/186 2024-04-25T18:52:58.801 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/186 2024-04-25T18:52:58.801 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:58.803 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:58.811 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:58.812 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2024-04-25T18:52:58.990 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:58.994 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:58.994 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:59.097 INFO:teuthology.orchestra.run.smithi094.stdout:generic/187 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:59.101 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/187 2024-04-25T18:52:59.101 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/187 2024-04-25T18:52:59.101 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:59.105 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:59.111 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:59.111 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2024-04-25T18:52:59.293 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:59.298 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:59.399 INFO:teuthology.orchestra.run.smithi094.stdout:generic/188 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:59.403 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/188 2024-04-25T18:52:59.403 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/188 2024-04-25T18:52:59.403 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:59.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:59.413 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2024-04-25T18:52:59.594 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:59.599 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:59.599 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:59.700 INFO:teuthology.orchestra.run.smithi094.stdout:generic/189 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:52:59.704 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/189 2024-04-25T18:52:59.704 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/189 2024-04-25T18:52:59.704 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:52:59.706 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:59.714 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:52:59.715 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2024-04-25T18:52:59.892 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:52:59.896 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:52:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:52:59.998 INFO:teuthology.orchestra.run.smithi094.stdout:generic/190 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:00.001 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/190 2024-04-25T18:53:00.002 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/190 2024-04-25T18:53:00.002 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:00.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:00.013 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2024-04-25T18:53:00.190 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:00.194 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:00.194 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:00.295 INFO:teuthology.orchestra.run.smithi094.stdout:generic/191 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:00.299 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/191 2024-04-25T18:53:00.299 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/191 2024-04-25T18:53:00.299 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:00.301 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:00.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:00.309 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2024-04-25T18:53:00.489 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:00.493 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:00.493 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:00.601 INFO:teuthology.orchestra.run.smithi094.stdout:generic/192 [not run] atime not maintained by ceph-fuse 2024-04-25T18:53:00.605 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/192 2024-04-25T18:53:00.605 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/192 2024-04-25T18:53:00.605 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:00.607 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:00.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:00.615 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2024-04-25T18:53:00.792 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:00.796 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:00.796 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:01.414 INFO:teuthology.orchestra.run.smithi094.stdout:generic/193 - output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/193.out.bad) 2024-04-25T18:53:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/193.out 2024-04-25 16:52:50.079099934 +0000 2024-04-25T18:53:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/193.out.bad 2024-04-25 18:53:01.412520580 +0000 2024-04-25T18:53:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -24,7 +24,6 @@ 2024-04-25T18:53:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: user: chmod a+r on qa_user owned file (should succeed) 2024-04-25T18:53:01.416 INFO:teuthology.orchestra.run.smithi094.stdout: user: chmod a+r on root owned file (should fail) 2024-04-25T18:53:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: -chmod: changing permissions of 'test.root': Operation not permitted 2024-04-25T18:53:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: check that the sgid bit is cleared 2024-04-25T18:53:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: -rw-rw-rw- 2024-04-25T18:53:01.417 INFO:teuthology.orchestra.run.smithi094.stdout: check that suid bit is not cleared 2024-04-25T18:53:01.418 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/193 2024-04-25T18:53:01.418 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/193 2024-04-25T18:53:01.419 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-25T18:53:01.421 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:01.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:53:01.429 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-25T18:53:01.429 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-25T18:53:01.429 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2024-04-25T18:53:01.969 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:01.974 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:01.974 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:02.144 INFO:teuthology.orchestra.run.smithi094.stdout:generic/194 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:02.149 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/194 2024-04-25T18:53:02.149 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/194 2024-04-25T18:53:02.149 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:02.161 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:02.161 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2024-04-25T18:53:02.339 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:02.343 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:02.343 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:02.444 INFO:teuthology.orchestra.run.smithi094.stdout:generic/195 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:02.447 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/195 2024-04-25T18:53:02.448 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/195 2024-04-25T18:53:02.448 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:02.457 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:02.458 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2024-04-25T18:53:02.634 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:02.638 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:02.638 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:generic/196 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:02.752 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/196 2024-04-25T18:53:02.752 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/196 2024-04-25T18:53:02.752 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:02.754 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:02.762 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:02.763 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2024-04-25T18:53:02.940 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:02.944 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:02.944 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:03.045 INFO:teuthology.orchestra.run.smithi094.stdout:generic/197 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:03.049 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/197 2024-04-25T18:53:03.049 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/197 2024-04-25T18:53:03.049 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:03.059 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:03.060 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2024-04-25T18:53:03.238 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:03.242 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:03.242 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:04.245 INFO:teuthology.orchestra.run.smithi094.stdout:generic/198 1s 2024-04-25T18:53:04.252 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/198 2024-04-25T18:53:04.252 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:04.256 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:04.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:04.267 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2024-04-25T18:53:04.817 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:04.821 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:04.821 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:04.926 INFO:teuthology.orchestra.run.smithi094.stdout:generic/199 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:04.930 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/199 2024-04-25T18:53:04.930 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/199 2024-04-25T18:53:04.930 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:04.932 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:04.940 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:04.940 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2024-04-25T18:53:05.122 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:05.126 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:05.126 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:05.267 INFO:teuthology.orchestra.run.smithi094.stdout:generic/200 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:05.271 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/200 2024-04-25T18:53:05.271 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/200 2024-04-25T18:53:05.271 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:05.274 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:05.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:05.282 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2024-04-25T18:53:05.461 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:05.465 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:05.465 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:05.567 INFO:teuthology.orchestra.run.smithi094.stdout:generic/201 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:05.570 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/201 2024-04-25T18:53:05.570 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/201 2024-04-25T18:53:05.570 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:05.573 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:05.580 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:05.581 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2024-04-25T18:53:05.759 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:05.763 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:05.763 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:05.866 INFO:teuthology.orchestra.run.smithi094.stdout:generic/202 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:05.869 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/202 2024-04-25T18:53:05.869 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/202 2024-04-25T18:53:05.869 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:05.871 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:05.879 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:05.879 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2024-04-25T18:53:06.058 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:06.062 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:06.062 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:06.171 INFO:teuthology.orchestra.run.smithi094.stdout:generic/203 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:06.174 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/203 2024-04-25T18:53:06.175 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/203 2024-04-25T18:53:06.175 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:06.177 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:06.185 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:06.185 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2024-04-25T18:53:06.367 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:06.371 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:06.371 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:08.651 INFO:teuthology.orchestra.run.smithi094.stdout:generic/204 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:08.658 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/204 2024-04-25T18:53:08.658 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/204 2024-04-25T18:53:08.658 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:08.663 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:08.725 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:08.726 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2024-04-25T18:53:08.962 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:08.967 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:08.967 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:09.109 INFO:teuthology.orchestra.run.smithi094.stdout:generic/205 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:09.113 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/205 2024-04-25T18:53:09.113 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/205 2024-04-25T18:53:09.113 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:09.116 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:09.127 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:09.127 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2024-04-25T18:53:09.333 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:09.337 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:09.337 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:09.449 INFO:teuthology.orchestra.run.smithi094.stdout:generic/206 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:53:09.452 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/206 2024-04-25T18:53:09.452 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/206 2024-04-25T18:53:09.452 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:09.455 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:09.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:09.462 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2024-04-25T18:53:09.640 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:09.644 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:09.644 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:12.608 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:53:12.610 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:53:37.091 INFO:teuthology.orchestra.run.smithi094.stdout:generic/207 28s 2024-04-25T18:53:37.098 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/207 2024-04-25T18:53:37.098 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:53:37.101 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:37.286 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:53:37.286 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2024-04-25T18:53:37.697 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:53:37.702 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:53:37.702 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:53:42.855 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:53:42.858 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:54:12.905 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:54:12.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:54:42.951 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:54:42.955 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:55:13.148 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:55:13.152 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:55:43.203 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:55:43.207 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:56:13.251 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:56:13.254 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:56:43.341 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:56:43.344 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:56:58.176 INFO:teuthology.orchestra.run.smithi094.stdout:generic/208 201s 2024-04-25T18:56:58.185 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/208 2024-04-25T18:56:58.185 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:56:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:56:58.202 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:56:58.202 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2024-04-25T18:56:58.758 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:56:58.762 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:56:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:13.387 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:57:13.390 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:57:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:generic/209 [failed, exit status 1]- output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/209.out.bad) 2024-04-25T18:57:22.750 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/209.out 2024-04-25 16:52:50.079099934 +0000 2024-04-25T18:57:22.750 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/209.out.bad 2024-04-25 18:57:22.684545670 +0000 2024-04-25T18:57:22.750 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -1,2 +1,2 @@ 2024-04-25T18:57:22.750 INFO:teuthology.orchestra.run.smithi094.stdout: QA output created by 209 2024-04-25T18:57:22.750 INFO:teuthology.orchestra.run.smithi094.stdout: -test ran for 30 seconds without error 2024-04-25T18:57:22.750 INFO:teuthology.orchestra.run.smithi094.stdout: +reader found old byte at pos 4096 2024-04-25T18:57:22.751 INFO:teuthology.orchestra.run.smithi094.stdout: \ No newline at end of file 2024-04-25T18:57:22.756 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/209 2024-04-25T18:57:22.756 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/209 2024-04-25T18:57:22.756 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-25T18:57:22.765 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:22.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:57:22.793 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-25T18:57:22.793 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-25T18:57:22.794 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2024-04-25T18:57:23.068 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:23.073 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:23.073 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:23.455 INFO:teuthology.orchestra.run.smithi094.stdout:generic/210 0s 2024-04-25T18:57:23.461 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/210 2024-04-25T18:57:23.461 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:23.465 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:23.481 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:23.482 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2024-04-25T18:57:24.006 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:24.010 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:24.010 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:24.350 INFO:teuthology.orchestra.run.smithi094.stdout:generic/211 0s 2024-04-25T18:57:24.355 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/211 2024-04-25T18:57:24.355 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:24.358 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:24.367 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:24.367 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2024-04-25T18:57:24.548 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:24.552 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:24.552 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:24.839 INFO:teuthology.orchestra.run.smithi094.stdout:generic/212 0s 2024-04-25T18:57:24.846 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/212 2024-04-25T18:57:24.846 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:24.849 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:24.859 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:24.860 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2024-04-25T18:57:25.058 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:25.062 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:25.062 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:25.453 INFO:teuthology.orchestra.run.smithi094.stdout:generic/213 - output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/213.out.bad) 2024-04-25T18:57:25.455 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/213.out 2024-04-25 16:52:50.079099934 +0000 2024-04-25T18:57:25.455 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/213.out.bad 2024-04-25 18:57:25.448503637 +0000 2024-04-25T18:57:25.456 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -1,4 +1,3 @@ 2024-04-25T18:57:25.456 INFO:teuthology.orchestra.run.smithi094.stdout: QA output created by 213 2024-04-25T18:57:25.456 INFO:teuthology.orchestra.run.smithi094.stdout: We should get: fallocate: No space left on device 2024-04-25T18:57:25.456 INFO:teuthology.orchestra.run.smithi094.stdout: Strangely, xfs_io sometimes says "Success" when something went wrong, FYI 2024-04-25T18:57:25.456 INFO:teuthology.orchestra.run.smithi094.stdout: -fallocate: No space left on device 2024-04-25T18:57:25.458 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/213 2024-04-25T18:57:25.458 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/213 2024-04-25T18:57:25.458 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-25T18:57:25.461 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:25.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:57:25.469 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-25T18:57:25.469 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-25T18:57:25.470 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2024-04-25T18:57:26.003 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:26.007 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:26.346 INFO:teuthology.orchestra.run.smithi094.stdout:generic/214 0s 2024-04-25T18:57:26.351 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/214 2024-04-25T18:57:26.351 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:26.361 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:26.362 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2024-04-25T18:57:26.919 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:26.924 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:26.924 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:30.057 INFO:teuthology.orchestra.run.smithi094.stdout:generic/215 4s 2024-04-25T18:57:30.063 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/215 2024-04-25T18:57:30.063 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:30.074 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:30.074 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2024-04-25T18:57:30.341 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:30.346 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:30.346 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:30.499 INFO:teuthology.orchestra.run.smithi094.stdout:generic/216 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:30.504 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/216 2024-04-25T18:57:30.504 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/216 2024-04-25T18:57:30.504 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:30.507 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:30.522 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:30.522 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2024-04-25T18:57:30.804 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:30.809 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:30.809 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:generic/217 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/217 2024-04-25T18:57:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/217 2024-04-25T18:57:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:30.977 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:30.978 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2024-04-25T18:57:31.250 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:31.255 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:31.255 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi094.stdout:generic/218 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/218 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/218 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:31.451 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:31.452 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi094.stdout:generic/219 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/219 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/219 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:31.934 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:31.935 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:generic/220 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/220 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/220 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:32.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:32.369 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2024-04-25T18:57:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:32.659 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:32.659 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:33.867 INFO:teuthology.orchestra.run.smithi094.stdout:generic/221 1s 2024-04-25T18:57:33.873 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/221 2024-04-25T18:57:33.873 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:33.875 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:33.892 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:33.892 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2024-04-25T18:57:34.161 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:34.166 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:34.166 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:34.316 INFO:teuthology.orchestra.run.smithi094.stdout:generic/222 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:34.320 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/222 2024-04-25T18:57:34.320 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/222 2024-04-25T18:57:34.320 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:34.322 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:34.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:34.334 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2024-04-25T18:57:34.611 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:34.615 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:34.615 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:34.793 INFO:teuthology.orchestra.run.smithi094.stdout:generic/223 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:34.798 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/223 2024-04-25T18:57:34.798 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/223 2024-04-25T18:57:34.798 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:34.801 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:34.817 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:34.817 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2024-04-25T18:57:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:35.110 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:35.269 INFO:teuthology.orchestra.run.smithi094.stdout:generic/224 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:35.273 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/224 2024-04-25T18:57:35.273 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/224 2024-04-25T18:57:35.273 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:35.287 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:35.287 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:35.714 INFO:teuthology.orchestra.run.smithi094.stdout:generic/225 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:35.718 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/225 2024-04-25T18:57:35.718 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/225 2024-04-25T18:57:35.718 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:35.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:35.734 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2024-04-25T18:57:35.994 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:35.999 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:36.166 INFO:teuthology.orchestra.run.smithi094.stdout:generic/226 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:36.170 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/226 2024-04-25T18:57:36.170 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/226 2024-04-25T18:57:36.170 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:36.183 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:36.183 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2024-04-25T18:57:36.485 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:36.492 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:36.638 INFO:teuthology.orchestra.run.smithi094.stdout:generic/227 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:36.643 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/227 2024-04-25T18:57:36.643 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/227 2024-04-25T18:57:36.643 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:36.646 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:36.657 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:36.657 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2024-04-25T18:57:36.942 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:36.948 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:36.948 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:37.244 INFO:teuthology.orchestra.run.smithi094.stdout:generic/228 1s 2024-04-25T18:57:37.252 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/228 2024-04-25T18:57:37.252 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:37.254 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:37.266 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:37.266 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2024-04-25T18:57:37.902 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:37.911 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:37.911 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:38.076 INFO:teuthology.orchestra.run.smithi094.stdout:generic/229 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:38.080 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/229 2024-04-25T18:57:38.081 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/229 2024-04-25T18:57:38.081 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:38.085 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:38.102 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:38.102 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2024-04-25T18:57:38.399 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:38.406 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:38.406 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:38.567 INFO:teuthology.orchestra.run.smithi094.stdout:generic/230 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:38.571 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/230 2024-04-25T18:57:38.572 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/230 2024-04-25T18:57:38.572 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:38.575 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:38.585 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:38.585 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2024-04-25T18:57:38.888 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:38.892 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:38.893 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:39.060 INFO:teuthology.orchestra.run.smithi094.stdout:generic/231 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:39.064 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/231 2024-04-25T18:57:39.065 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/231 2024-04-25T18:57:39.065 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:39.085 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:39.085 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2024-04-25T18:57:39.362 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:39.369 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:39.369 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:39.509 INFO:teuthology.orchestra.run.smithi094.stdout:generic/232 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:39.518 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/232 2024-04-25T18:57:39.518 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/232 2024-04-25T18:57:39.518 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:39.537 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:39.537 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2024-04-25T18:57:39.829 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:39.834 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:39.834 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:39.994 INFO:teuthology.orchestra.run.smithi094.stdout:generic/233 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:39.999 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/233 2024-04-25T18:57:39.999 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/233 2024-04-25T18:57:39.999 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:40.015 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:40.016 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2024-04-25T18:57:40.288 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:40.292 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:40.292 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:40.441 INFO:teuthology.orchestra.run.smithi094.stdout:generic/234 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:40.445 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/234 2024-04-25T18:57:40.445 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/234 2024-04-25T18:57:40.445 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:40.457 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:40.458 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2024-04-25T18:57:40.759 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:40.764 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:40.764 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:40.929 INFO:teuthology.orchestra.run.smithi094.stdout:generic/235 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:40.937 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/235 2024-04-25T18:57:40.937 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/235 2024-04-25T18:57:40.937 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:40.940 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:40.957 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:40.958 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2024-04-25T18:57:41.224 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:41.229 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:41.229 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:42.557 INFO:teuthology.orchestra.run.smithi094.stdout:generic/236 1s 2024-04-25T18:57:42.564 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/236 2024-04-25T18:57:42.564 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:42.573 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:42.682 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:42.682 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2024-04-25T18:57:43.174 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:43.178 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:43.178 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:43.391 INFO:teuthology.orchestra.run.smithi094.stdout:generic/237 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-25T18:57:43.395 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/237 2024-04-25T18:57:43.395 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/237 2024-04-25T18:57:43.396 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:43.398 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:43.407 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:43.407 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2024-04-25T18:57:43.501 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:57:43.503 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:57:43.660 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:43.669 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:43.670 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:43.816 INFO:teuthology.orchestra.run.smithi094.stdout:generic/238 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:57:43.821 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/238 2024-04-25T18:57:43.821 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/238 2024-04-25T18:57:43.821 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:43.827 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:43.837 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:43.838 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2024-04-25T18:57:44.108 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:44.112 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:44.112 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:44.287 INFO:teuthology.orchestra.run.smithi094.stdout:generic/239 [not run] Sparse files are not supported or do not work as expected 2024-04-25T18:57:44.295 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/239 2024-04-25T18:57:44.295 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/239 2024-04-25T18:57:44.295 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:44.295 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:44.311 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:44.311 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2024-04-25T18:57:44.560 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:44.565 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:44.767 INFO:teuthology.orchestra.run.smithi094.stdout:generic/240 [not run] Sparse files are not supported or do not work as expected 2024-04-25T18:57:44.771 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/240 2024-04-25T18:57:44.771 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/240 2024-04-25T18:57:44.771 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:57:44.773 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:57:44.782 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:57:44.783 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2024-04-25T18:57:45.062 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:57:45.070 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:57:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:58:13.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:58:13.617 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:58:43.694 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:58:43.697 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:58:59.078 INFO:teuthology.orchestra.run.smithi094.stdout:generic/241 74s 2024-04-25T18:58:59.084 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/241 2024-04-25T18:58:59.084 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:58:59.087 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:58:59.101 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:58:59.101 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2024-04-25T18:58:59.769 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:58:59.774 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:58:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:58:59.966 INFO:teuthology.orchestra.run.smithi094.stdout:generic/242 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:58:59.970 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/242 2024-04-25T18:58:59.970 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/242 2024-04-25T18:58:59.970 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:58:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:58:59.988 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:58:59.988 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2024-04-25T18:59:00.279 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:59:00.284 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:59:00.284 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:59:00.435 INFO:teuthology.orchestra.run.smithi094.stdout:generic/243 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T18:59:00.438 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/243 2024-04-25T18:59:00.439 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/243 2024-04-25T18:59:00.439 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:59:00.441 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:59:00.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:59:00.454 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2024-04-25T18:59:00.761 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:59:00.766 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:59:00.766 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:59:00.902 INFO:teuthology.orchestra.run.smithi094.stdout:generic/244 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-25T18:59:00.906 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/244 2024-04-25T18:59:00.907 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/244 2024-04-25T18:59:00.907 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:59:00.909 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:59:00.920 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:59:00.921 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2024-04-25T18:59:01.213 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:59:01.220 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:59:01.220 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:59:01.516 INFO:teuthology.orchestra.run.smithi094.stdout:generic/245 0s 2024-04-25T18:59:01.527 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/245 2024-04-25T18:59:01.527 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:59:01.530 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:59:01.759 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:59:01.760 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2024-04-25T18:59:02.190 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:59:02.195 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:59:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:59:02.429 INFO:teuthology.orchestra.run.smithi094.stdout:generic/246 0s 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/246 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T18:59:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:59:02.457 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T18:59:02.457 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2024-04-25T18:59:03.094 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T18:59:03.102 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T18:59:03.102 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T18:59:14.005 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:59:14.008 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:59:44.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T18:59:44.060 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:00:02.021 INFO:teuthology.orchestra.run.smithi094.stdout:generic/247 58s 2024-04-25T19:00:02.067 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/247 2024-04-25T19:00:02.067 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:02.085 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:02.086 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2024-04-25T19:00:02.386 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:02.391 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:02.391 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:02.624 INFO:teuthology.orchestra.run.smithi094.stdout:generic/248 0s 2024-04-25T19:00:02.639 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/248 2024-04-25T19:00:02.639 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:02.642 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:02.651 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:02.651 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2024-04-25T19:00:02.934 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:02.939 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:02.939 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:05.183 INFO:teuthology.orchestra.run.smithi094.stdout:generic/249 3s 2024-04-25T19:00:05.190 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/249 2024-04-25T19:00:05.190 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:05.193 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:05.201 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:05.201 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2024-04-25T19:00:05.813 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:05.818 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:05.818 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:06.037 INFO:teuthology.orchestra.run.smithi094.stdout:generic/250 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:00:06.042 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/250 2024-04-25T19:00:06.042 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/250 2024-04-25T19:00:06.042 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:06.044 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:06.058 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:06.058 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2024-04-25T19:00:06.383 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:06.388 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:06.388 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:06.544 INFO:teuthology.orchestra.run.smithi094.stdout:generic/251 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:00:06.548 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/251 2024-04-25T19:00:06.548 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/251 2024-04-25T19:00:06.548 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:06.550 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:06.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:06.561 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2024-04-25T19:00:06.828 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:06.834 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:06.834 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:07.023 INFO:teuthology.orchestra.run.smithi094.stdout:generic/252 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:00:07.035 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/252 2024-04-25T19:00:07.035 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/252 2024-04-25T19:00:07.035 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:07.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:07.051 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2024-04-25T19:00:07.345 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:07.350 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:07.350 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:07.520 INFO:teuthology.orchestra.run.smithi094.stdout:generic/253 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:00:07.524 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/253 2024-04-25T19:00:07.524 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/253 2024-04-25T19:00:07.525 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:07.538 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:07.538 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2024-04-25T19:00:07.833 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:07.838 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:07.838 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:08.022 INFO:teuthology.orchestra.run.smithi094.stdout:generic/254 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:00:08.034 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/254 2024-04-25T19:00:08.034 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/254 2024-04-25T19:00:08.034 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:08.037 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:08.047 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:08.047 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2024-04-25T19:00:08.340 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:08.345 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:08.345 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:09.839 INFO:teuthology.orchestra.run.smithi094.stdout:generic/255 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-25T19:00:09.843 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/255 2024-04-25T19:00:09.843 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/255 2024-04-25T19:00:09.843 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:09.851 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:09.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:09.861 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2024-04-25T19:00:10.481 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:10.485 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:10.485 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:11.800 INFO:teuthology.orchestra.run.smithi094.stdout:generic/256 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:00:11.805 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/256 2024-04-25T19:00:11.805 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/256 2024-04-25T19:00:11.805 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:11.808 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:11.825 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:11.825 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2024-04-25T19:00:12.107 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:12.112 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:12.112 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:14.004 INFO:teuthology.orchestra.run.smithi094.stdout:generic/257 1s 2024-04-25T19:00:14.015 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/257 2024-04-25T19:00:14.015 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:14.019 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:14.236 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:14.237 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2024-04-25T19:00:14.675 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:14.680 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:14.680 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:14.682 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:00:14.684 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:00:14.696 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T19:00:14.689+0000 7f837496c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 215746) UID: 0 2024-04-25T19:00:14.697 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T19:00:14.689+0000 7f9a177b0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 215746) UID: 0 2024-04-25T19:00:14.697 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T19:00:14.689+0000 7efca5c83640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 215746) UID: 0 2024-04-25T19:00:14.700 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T19:00:14.689+0000 7f3801006640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 215746) UID: 0 2024-04-25T19:00:14.926 INFO:teuthology.orchestra.run.smithi094.stdout:generic/258 [failed, exit status 1]- output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/258.out.bad) 2024-04-25T19:00:14.929 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/258.out 2024-04-25 16:52:50.083099879 +0000 2024-04-25T19:00:14.929 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/258.out.bad 2024-04-25 19:00:14.917927026 +0000 2024-04-25T19:00:14.929 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -1,5 +1,6 @@ 2024-04-25T19:00:14.929 INFO:teuthology.orchestra.run.smithi094.stdout: QA output created by 258 2024-04-25T19:00:14.929 INFO:teuthology.orchestra.run.smithi094.stdout: Creating file with timestamp of Jan 1, 1960 2024-04-25T19:00:14.929 INFO:teuthology.orchestra.run.smithi094.stdout: Testing for negative seconds since epoch 2024-04-25T19:00:14.929 INFO:teuthology.orchestra.run.smithi094.stdout: -Remounting to flush cache 2024-04-25T19:00:14.929 INFO:teuthology.orchestra.run.smithi094.stdout: -Testing for negative seconds since epoch 2024-04-25T19:00:14.929 INFO:teuthology.orchestra.run.smithi094.stdout: +Timestamp wrapped: 3979351756 2024-04-25T19:00:14.929 INFO:teuthology.orchestra.run.smithi094.stdout: +Timestamp wrapped 2024-04-25T19:00:14.929 INFO:teuthology.orchestra.run.smithi094.stdout: +(see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/258.full for details) 2024-04-25T19:00:14.932 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/258 2024-04-25T19:00:14.932 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/258 2024-04-25T19:00:14.932 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-25T19:00:14.936 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:14.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:00:14.946 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-25T19:00:14.946 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-25T19:00:14.947 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2024-04-25T19:00:15.293 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:15.296 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:15.296 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:15.502 INFO:teuthology.orchestra.run.smithi094.stdout:generic/259 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:00:15.508 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/259 2024-04-25T19:00:15.508 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/259 2024-04-25T19:00:15.508 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:15.511 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:15.522 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:15.523 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2024-04-25T19:00:15.897 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:15.902 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:15.902 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:16.082 INFO:teuthology.orchestra.run.smithi094.stdout:generic/260 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:00:16.086 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/260 2024-04-25T19:00:16.086 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/260 2024-04-25T19:00:16.086 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:16.088 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:16.101 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:16.102 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2024-04-25T19:00:16.413 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:16.419 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:16.419 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:16.571 INFO:teuthology.orchestra.run.smithi094.stdout:generic/261 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:00:16.575 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/261 2024-04-25T19:00:16.576 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/261 2024-04-25T19:00:16.576 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:16.603 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:16.603 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2024-04-25T19:00:17.093 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:17.172 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:17.942 INFO:teuthology.orchestra.run.smithi094.stdout:generic/262 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:00:17.954 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/262 2024-04-25T19:00:17.954 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/262 2024-04-25T19:00:17.954 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:00:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:00:17.968 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:00:17.968 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2024-04-25T19:00:21.381 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:00:21.386 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:00:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:18.632 INFO:teuthology.orchestra.run.smithi094.stdout:generic/263 117s 2024-04-25T19:02:18.638 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/263 2024-04-25T19:02:18.638 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:18.642 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:18.652 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:18.652 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2024-04-25T19:02:19.224 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:19.229 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:19.229 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:19.440 INFO:teuthology.orchestra.run.smithi094.stdout:generic/264 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:19.446 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/264 2024-04-25T19:02:19.446 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/264 2024-04-25T19:02:19.446 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:19.450 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:19.464 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:19.464 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2024-04-25T19:02:19.665 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:19.669 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:19.670 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:20.114 INFO:teuthology.orchestra.run.smithi094.stdout:generic/265 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:20.121 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/265 2024-04-25T19:02:20.121 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/265 2024-04-25T19:02:20.121 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:20.126 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:20.138 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:20.138 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2024-04-25T19:02:20.366 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:20.370 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:20.370 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:20.731 INFO:teuthology.orchestra.run.smithi094.stdout:generic/266 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:20.737 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/266 2024-04-25T19:02:20.737 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/266 2024-04-25T19:02:20.737 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:20.740 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:20.751 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:20.751 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2024-04-25T19:02:20.972 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:20.976 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:20.976 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:21.117 INFO:teuthology.orchestra.run.smithi094.stdout:generic/267 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:21.121 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/267 2024-04-25T19:02:21.121 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/267 2024-04-25T19:02:21.121 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:21.132 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:21.132 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2024-04-25T19:02:21.351 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:21.355 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:21.356 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:21.496 INFO:teuthology.orchestra.run.smithi094.stdout:generic/268 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:21.500 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/268 2024-04-25T19:02:21.500 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/268 2024-04-25T19:02:21.500 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:21.511 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:21.511 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2024-04-25T19:02:21.699 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:21.703 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:21.703 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:21.812 INFO:teuthology.orchestra.run.smithi094.stdout:generic/269 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:21.816 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/269 2024-04-25T19:02:21.816 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/269 2024-04-25T19:02:21.816 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:21.827 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:21.827 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2024-04-25T19:02:22.015 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:22.020 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:22.020 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:22.140 INFO:teuthology.orchestra.run.smithi094.stdout:generic/270 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-25T19:02:22.144 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/270 2024-04-25T19:02:22.144 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/270 2024-04-25T19:02:22.144 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:22.147 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:22.155 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:22.155 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2024-04-25T19:02:22.344 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:22.348 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:22.475 INFO:teuthology.orchestra.run.smithi094.stdout:generic/271 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:22.479 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/271 2024-04-25T19:02:22.479 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/271 2024-04-25T19:02:22.479 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:22.491 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:22.491 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2024-04-25T19:02:22.676 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:22.680 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:22.680 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:22.808 INFO:teuthology.orchestra.run.smithi094.stdout:generic/272 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:22.812 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/272 2024-04-25T19:02:22.812 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/272 2024-04-25T19:02:22.812 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:22.814 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:22.823 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:22.823 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2024-04-25T19:02:23.007 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:23.012 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:23.012 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:23.116 INFO:teuthology.orchestra.run.smithi094.stdout:generic/273 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:23.120 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/273 2024-04-25T19:02:23.120 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/273 2024-04-25T19:02:23.120 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:23.122 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:23.131 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:23.131 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2024-04-25T19:02:23.321 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:23.325 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:23.325 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:23.432 INFO:teuthology.orchestra.run.smithi094.stdout:generic/274 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:23.436 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/274 2024-04-25T19:02:23.436 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/274 2024-04-25T19:02:23.436 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:23.438 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:23.446 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:23.447 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2024-04-25T19:02:23.631 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:23.635 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:23.635 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:23.738 INFO:teuthology.orchestra.run.smithi094.stdout:generic/275 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:23.741 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/275 2024-04-25T19:02:23.741 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/275 2024-04-25T19:02:23.741 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:23.743 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:23.751 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:23.751 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2024-04-25T19:02:23.932 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:23.936 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:23.936 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:24.188 INFO:teuthology.orchestra.run.smithi094.stdout:generic/276 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:24.191 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/276 2024-04-25T19:02:24.192 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/276 2024-04-25T19:02:24.192 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:24.203 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:24.203 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2024-04-25T19:02:24.388 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:24.392 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:24.392 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:24.543 INFO:teuthology.orchestra.run.smithi094.stdout:generic/277 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:24.546 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/277 2024-04-25T19:02:24.546 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/277 2024-04-25T19:02:24.546 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:24.549 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:24.556 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:24.556 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2024-04-25T19:02:24.742 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:24.746 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:24.746 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:24.871 INFO:teuthology.orchestra.run.smithi094.stdout:generic/278 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:24.875 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/278 2024-04-25T19:02:24.875 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/278 2024-04-25T19:02:24.875 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:24.877 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:24.886 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:24.886 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2024-04-25T19:02:25.069 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:25.073 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:25.073 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:25.228 INFO:teuthology.orchestra.run.smithi094.stdout:generic/279 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:25.232 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/279 2024-04-25T19:02:25.232 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/279 2024-04-25T19:02:25.232 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:25.242 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:25.242 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2024-04-25T19:02:25.431 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:25.434 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:25.435 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:25.684 INFO:teuthology.orchestra.run.smithi094.stdout:generic/280 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:25.689 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/280 2024-04-25T19:02:25.689 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/280 2024-04-25T19:02:25.689 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:25.693 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:25.703 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:25.703 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2024-04-25T19:02:25.916 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:25.920 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:26.065 INFO:teuthology.orchestra.run.smithi094.stdout:generic/281 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:26.069 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/281 2024-04-25T19:02:26.069 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/281 2024-04-25T19:02:26.069 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:26.080 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:26.080 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2024-04-25T19:02:26.267 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:26.271 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:26.392 INFO:teuthology.orchestra.run.smithi094.stdout:generic/282 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:26.396 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/282 2024-04-25T19:02:26.396 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/282 2024-04-25T19:02:26.396 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:26.406 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:26.406 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2024-04-25T19:02:26.596 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:26.600 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:26.600 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:26.769 INFO:teuthology.orchestra.run.smithi094.stdout:generic/283 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:26.772 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/283 2024-04-25T19:02:26.773 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/283 2024-04-25T19:02:26.773 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:26.775 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:26.783 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:26.784 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2024-04-25T19:02:26.971 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:26.975 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:26.975 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:27.080 INFO:teuthology.orchestra.run.smithi094.stdout:generic/284 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:02:27.084 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/284 2024-04-25T19:02:27.084 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/284 2024-04-25T19:02:27.084 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:27.086 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:27.094 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:27.094 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2024-04-25T19:02:27.279 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:27.284 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:27.284 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:30.296 INFO:teuthology.orchestra.run.smithi094.stdout:generic/285 3s 2024-04-25T19:02:30.301 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/285 2024-04-25T19:02:30.301 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:02:30.304 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:02:30.312 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:02:30.312 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2024-04-25T19:02:31.009 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:02:31.013 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:02:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:07.769 INFO:teuthology.orchestra.run.smithi094.stdout:generic/286 36s 2024-04-25T19:03:07.774 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/286 2024-04-25T19:03:07.774 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:07.777 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:07.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:07.810 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2024-04-25T19:03:08.410 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:08.414 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:08.414 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:08.544 INFO:teuthology.orchestra.run.smithi094.stdout:generic/287 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:08.547 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/287 2024-04-25T19:03:08.547 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/287 2024-04-25T19:03:08.547 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:08.550 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:08.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:08.557 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2024-04-25T19:03:08.741 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:08.745 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:08.745 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:08.931 INFO:teuthology.orchestra.run.smithi094.stdout:generic/288 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:08.935 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/288 2024-04-25T19:03:08.935 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/288 2024-04-25T19:03:08.935 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:08.942 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:08.952 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:08.952 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2024-04-25T19:03:09.193 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:09.197 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:09.198 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:09.301 INFO:teuthology.orchestra.run.smithi094.stdout:generic/289 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:09.304 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/289 2024-04-25T19:03:09.305 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/289 2024-04-25T19:03:09.305 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:09.307 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:09.315 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:09.315 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2024-04-25T19:03:09.496 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:09.500 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:09.501 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:09.602 INFO:teuthology.orchestra.run.smithi094.stdout:generic/290 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:09.606 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/290 2024-04-25T19:03:09.606 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/290 2024-04-25T19:03:09.606 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:09.608 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:09.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:09.616 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2024-04-25T19:03:09.795 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:09.799 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:09.799 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:09.956 INFO:teuthology.orchestra.run.smithi094.stdout:generic/291 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:09.959 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/291 2024-04-25T19:03:09.959 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/291 2024-04-25T19:03:09.959 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:09.962 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:09.970 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:09.970 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2024-04-25T19:03:10.155 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:10.159 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:10.159 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:10.263 INFO:teuthology.orchestra.run.smithi094.stdout:generic/292 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:10.267 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/292 2024-04-25T19:03:10.267 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/292 2024-04-25T19:03:10.267 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:10.269 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:10.277 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:10.277 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2024-04-25T19:03:10.456 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:10.460 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:10.460 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:10.564 INFO:teuthology.orchestra.run.smithi094.stdout:generic/293 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:10.568 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/293 2024-04-25T19:03:10.568 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/293 2024-04-25T19:03:10.568 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:10.570 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:10.578 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:10.578 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2024-04-25T19:03:10.760 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:10.764 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:10.764 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:10.867 INFO:teuthology.orchestra.run.smithi094.stdout:generic/294 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:10.871 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/294 2024-04-25T19:03:10.871 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/294 2024-04-25T19:03:10.871 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:10.873 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:10.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:10.881 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2024-04-25T19:03:11.063 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:11.067 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:11.067 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:11.171 INFO:teuthology.orchestra.run.smithi094.stdout:generic/295 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:11.174 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/295 2024-04-25T19:03:11.175 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/295 2024-04-25T19:03:11.175 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:11.186 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:11.186 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2024-04-25T19:03:11.365 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:11.369 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:11.369 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:11.471 INFO:teuthology.orchestra.run.smithi094.stdout:generic/296 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:11.475 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/296 2024-04-25T19:03:11.475 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/296 2024-04-25T19:03:11.475 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:11.477 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:11.485 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:11.485 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2024-04-25T19:03:11.666 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:11.670 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:11.789 INFO:teuthology.orchestra.run.smithi094.stdout:generic/297 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:11.792 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/297 2024-04-25T19:03:11.793 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/297 2024-04-25T19:03:11.793 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:11.795 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:11.803 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:11.803 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2024-04-25T19:03:11.984 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:11.988 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:11.988 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:12.094 INFO:teuthology.orchestra.run.smithi094.stdout:generic/298 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/298 2024-04-25T19:03:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/298 2024-04-25T19:03:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:12.100 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:12.108 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:12.108 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2024-04-25T19:03:12.289 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:12.293 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:12.397 INFO:teuthology.orchestra.run.smithi094.stdout:generic/299 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:12.401 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/299 2024-04-25T19:03:12.401 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/299 2024-04-25T19:03:12.401 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:12.403 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:12.411 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:12.411 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2024-04-25T19:03:12.591 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:12.595 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:12.697 INFO:teuthology.orchestra.run.smithi094.stdout:generic/300 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:12.700 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/300 2024-04-25T19:03:12.700 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/300 2024-04-25T19:03:12.700 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:12.710 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:12.711 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2024-04-25T19:03:12.894 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:12.899 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:12.899 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:13.002 INFO:teuthology.orchestra.run.smithi094.stdout:generic/301 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:13.005 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/301 2024-04-25T19:03:13.006 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/301 2024-04-25T19:03:13.006 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:13.008 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:13.016 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:13.016 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2024-04-25T19:03:13.196 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:13.202 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:13.202 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:13.318 INFO:teuthology.orchestra.run.smithi094.stdout:generic/302 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:13.321 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/302 2024-04-25T19:03:13.321 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/302 2024-04-25T19:03:13.321 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:13.324 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:13.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:13.332 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2024-04-25T19:03:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:13.516 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:13.516 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:13.743 INFO:teuthology.orchestra.run.smithi094.stdout:generic/303 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T19:03:13.747 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/303 2024-04-25T19:03:13.747 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/303 2024-04-25T19:03:13.747 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:13.749 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:13.757 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:13.758 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2024-04-25T19:03:14.323 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:14.327 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:14.327 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:14.653 INFO:teuthology.orchestra.run.smithi094.stdout:generic/304 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-25T19:03:14.657 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/304 2024-04-25T19:03:14.657 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/304 2024-04-25T19:03:14.657 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:14.660 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:14.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:14.669 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2024-04-25T19:03:15.236 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:15.240 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:15.240 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:15.407 INFO:teuthology.orchestra.run.smithi094.stdout:generic/305 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:15.410 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/305 2024-04-25T19:03:15.410 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/305 2024-04-25T19:03:15.411 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:15.413 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:15.422 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:15.422 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2024-04-25T19:03:15.607 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:15.611 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:15.611 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:15.844 INFO:teuthology.orchestra.run.smithi094.stdout:generic/306 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:15.848 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/306 2024-04-25T19:03:15.848 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/306 2024-04-25T19:03:15.848 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:15.850 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:15.859 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:15.859 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2024-04-25T19:03:16.051 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:16.056 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:16.056 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:16.298 INFO:teuthology.orchestra.run.smithi094.stdout:generic/307 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:03:16.303 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/307 2024-04-25T19:03:16.303 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/307 2024-04-25T19:03:16.303 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:16.307 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:16.358 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:16.358 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2024-04-25T19:03:16.551 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:16.555 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:16.807 INFO:teuthology.orchestra.run.smithi094.stdout:generic/308 0s 2024-04-25T19:03:16.878 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/308 2024-04-25T19:03:16.878 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:16.882 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:16.891 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:16.891 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2024-04-25T19:03:17.086 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:17.090 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:17.091 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:18.533 INFO:teuthology.orchestra.run.smithi094.stdout:generic/309 1s 2024-04-25T19:03:18.591 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/309 2024-04-25T19:03:18.591 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:03:18.595 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:03:18.605 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:03:18.605 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2024-04-25T19:03:19.165 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:03:19.169 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:03:19.169 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:04:00.183 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:04:00.186 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:04:30.626 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:04:30.629 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:04:46.957 INFO:teuthology.orchestra.run.smithi094.stdout:generic/310 87s 2024-04-25T19:04:46.963 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/310 2024-04-25T19:04:46.963 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:04:46.966 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:04:47.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:04:47.029 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2024-04-25T19:04:47.402 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:04:47.457 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:04:47.457 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:04:47.920 INFO:teuthology.orchestra.run.smithi094.stdout:generic/311 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:04:47.923 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/311 2024-04-25T19:04:47.923 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/311 2024-04-25T19:04:47.923 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:04:47.926 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:04:47.934 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:04:47.934 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2024-04-25T19:04:48.323 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:04:48.326 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:04:48.326 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:04:48.462 INFO:teuthology.orchestra.run.smithi094.stdout:generic/312 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:04:48.466 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/312 2024-04-25T19:04:48.466 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/312 2024-04-25T19:04:48.466 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:04:48.468 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:04:48.476 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:04:48.476 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2024-04-25T19:04:48.654 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:04:48.658 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:04:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:04:52.894 INFO:teuthology.orchestra.run.smithi094.stdout:generic/313 4s 2024-04-25T19:04:52.930 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/313 2024-04-25T19:04:52.930 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:04:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:04:52.946 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:04:52.947 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2024-04-25T19:04:53.165 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:04:53.170 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:04:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:04:57.560 INFO:teuthology.orchestra.run.smithi094.stdout:generic/314 4s 2024-04-25T19:04:57.573 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/314 2024-04-25T19:04:57.573 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:04:57.577 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:04:57.589 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:04:57.590 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2024-04-25T19:04:57.811 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:04:57.815 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:04:57.815 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:00.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:05:00.672 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:05:01.419 INFO:teuthology.orchestra.run.smithi094.stdout:generic/315 4s 2024-04-25T19:05:01.426 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/315 2024-04-25T19:05:01.426 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:05:01.429 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:01.441 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:05:01.441 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2024-04-25T19:05:01.990 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:05:01.994 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:05:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:03.046 INFO:teuthology.orchestra.run.smithi094.stdout:generic/316 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-25T19:05:03.051 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/316 2024-04-25T19:05:03.051 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/316 2024-04-25T19:05:03.051 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:05:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:03.063 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:05:03.064 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2024-04-25T19:05:03.600 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:05:03.604 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:05:03.604 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:03.710 INFO:teuthology.orchestra.run.smithi094.stdout:generic/317 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:05:03.712 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/317 2024-04-25T19:05:03.712 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/317 2024-04-25T19:05:03.712 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:05:03.714 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:03.722 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:05:03.723 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2024-04-25T19:05:03.907 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:05:03.911 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:05:03.911 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:04.021 INFO:teuthology.orchestra.run.smithi094.stdout:generic/318 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:05:04.024 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/318 2024-04-25T19:05:04.025 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/318 2024-04-25T19:05:04.025 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:05:04.027 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:04.035 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:05:04.035 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2024-04-25T19:05:04.216 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:05:04.220 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:05:04.220 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:04.356 INFO:teuthology.orchestra.run.smithi094.stdout:generic/319 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-25T19:05:04.360 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/319 2024-04-25T19:05:04.360 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/319 2024-04-25T19:05:04.360 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:05:04.363 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:04.370 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:05:04.371 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2024-04-25T19:05:04.555 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:05:04.559 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:05:04.559 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:04.662 INFO:teuthology.orchestra.run.smithi094.stdout:generic/320 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:05:04.665 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/320 2024-04-25T19:05:04.665 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/320 2024-04-25T19:05:04.666 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:05:04.667 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:04.675 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:05:04.676 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2024-04-25T19:05:04.855 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:05:04.859 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:05:04.859 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:04.988 INFO:teuthology.orchestra.run.smithi094.stdout:generic/321 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:05:04.992 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/321 2024-04-25T19:05:04.992 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/321 2024-04-25T19:05:04.992 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:05:04.995 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:05.004 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:05:05.004 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2024-04-25T19:05:05.197 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:05:05.201 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:05:05.201 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:05.316 INFO:teuthology.orchestra.run.smithi094.stdout:generic/322 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:05:05.320 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/322 2024-04-25T19:05:05.320 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/322 2024-04-25T19:05:05.320 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:05:05.322 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:05.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:05:05.330 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2024-04-25T19:05:05.512 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:05:05.516 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:05:05.516 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:05:31.168 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:05:31.170 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:06:01.250 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:06:01.252 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:06:31.361 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:06:31.364 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:06:31.379 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T19:06:31.376+0000 7f837496c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 272927) UID: 0 2024-04-25T19:06:31.379 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T19:06:31.376+0000 7efca5c83640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 272927) UID: 0 2024-04-25T19:06:31.380 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T19:06:31.376+0000 7f9a177b0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 272927) UID: 0 2024-04-25T19:06:31.380 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T19:06:31.376+0000 7f3801006640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 272927) UID: 0 2024-04-25T19:07:07.836 INFO:teuthology.orchestra.run.smithi094.stdout:generic/323 122s 2024-04-25T19:07:07.862 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/323 2024-04-25T19:07:07.862 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:07.867 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:07.893 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:07.894 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2024-04-25T19:07:08.510 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:08.514 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:08.515 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:08.999 INFO:teuthology.orchestra.run.smithi094.stdout:generic/324 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:09.005 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/324 2024-04-25T19:07:09.005 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/324 2024-04-25T19:07:09.005 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:09.009 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:09.020 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:09.021 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2024-04-25T19:07:09.243 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:09.247 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:09.247 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:09.662 INFO:teuthology.orchestra.run.smithi094.stdout:generic/325 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:09.814 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/325 2024-04-25T19:07:09.814 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/325 2024-04-25T19:07:09.814 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:09.815 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:09.815 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:09.815 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2024-04-25T19:07:10.054 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:10.059 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:10.059 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:10.217 INFO:teuthology.orchestra.run.smithi094.stdout:generic/326 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:10.221 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/326 2024-04-25T19:07:10.221 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/326 2024-04-25T19:07:10.221 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:10.224 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:10.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:10.234 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2024-04-25T19:07:10.438 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:10.442 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:10.442 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:10.578 INFO:teuthology.orchestra.run.smithi094.stdout:generic/327 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:10.582 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/327 2024-04-25T19:07:10.582 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/327 2024-04-25T19:07:10.582 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:10.585 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:10.594 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:10.594 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2024-04-25T19:07:10.782 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:10.787 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:10.787 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:11.109 INFO:teuthology.orchestra.run.smithi094.stdout:generic/328 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:11.114 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/328 2024-04-25T19:07:11.115 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/328 2024-04-25T19:07:11.115 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:11.128 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:11.128 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2024-04-25T19:07:11.327 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:11.331 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:11.331 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:11.668 INFO:teuthology.orchestra.run.smithi094.stdout:generic/329 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:11.674 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/329 2024-04-25T19:07:11.674 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/329 2024-04-25T19:07:11.674 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:11.689 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:11.689 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2024-04-25T19:07:11.901 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:11.905 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:11.905 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:12.039 INFO:teuthology.orchestra.run.smithi094.stdout:generic/330 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:12.043 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/330 2024-04-25T19:07:12.043 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/330 2024-04-25T19:07:12.043 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:12.045 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:12.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:12.054 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2024-04-25T19:07:12.252 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:12.257 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:12.413 INFO:teuthology.orchestra.run.smithi094.stdout:generic/331 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:12.417 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/331 2024-04-25T19:07:12.417 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/331 2024-04-25T19:07:12.417 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:12.420 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:12.429 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:12.429 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2024-04-25T19:07:12.622 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:12.626 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:12.735 INFO:teuthology.orchestra.run.smithi094.stdout:generic/332 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:12.738 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/332 2024-04-25T19:07:12.739 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/332 2024-04-25T19:07:12.739 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:12.748 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:12.749 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2024-04-25T19:07:12.928 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:12.932 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:12.932 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:13.032 INFO:teuthology.orchestra.run.smithi094.stdout:generic/333 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:13.036 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/333 2024-04-25T19:07:13.036 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/333 2024-04-25T19:07:13.036 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:13.038 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:13.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:13.046 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2024-04-25T19:07:13.225 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:13.229 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:13.514 INFO:teuthology.orchestra.run.smithi094.stdout:generic/334 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:13.519 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/334 2024-04-25T19:07:13.519 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/334 2024-04-25T19:07:13.519 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:13.523 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:13.532 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:13.533 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2024-04-25T19:07:13.731 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:13.735 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:13.735 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:13.938 INFO:teuthology.orchestra.run.smithi094.stdout:generic/335 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:13.942 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/335 2024-04-25T19:07:13.943 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/335 2024-04-25T19:07:13.943 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:13.945 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:13.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:13.954 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2024-04-25T19:07:14.139 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:14.143 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:14.144 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:14.333 INFO:teuthology.orchestra.run.smithi094.stdout:generic/336 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:14.337 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/336 2024-04-25T19:07:14.337 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/336 2024-04-25T19:07:14.337 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:14.339 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:14.348 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:14.348 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2024-04-25T19:07:14.535 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:14.539 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:14.539 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:14.748 INFO:teuthology.orchestra.run.smithi094.stdout:generic/337 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:14.753 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/337 2024-04-25T19:07:14.753 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/337 2024-04-25T19:07:14.753 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:14.757 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:14.768 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:14.768 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2024-04-25T19:07:14.980 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:14.983 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:14.984 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:15.217 INFO:teuthology.orchestra.run.smithi094.stdout:generic/338 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:15.222 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/338 2024-04-25T19:07:15.222 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/338 2024-04-25T19:07:15.222 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:15.234 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:15.234 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2024-04-25T19:07:15.456 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:15.461 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:15.461 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:15.586 INFO:teuthology.orchestra.run.smithi094.stdout:generic/339 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:15.591 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/339 2024-04-25T19:07:15.591 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/339 2024-04-25T19:07:15.591 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:15.594 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:15.604 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:15.604 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2024-04-25T19:07:15.816 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:15.820 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:15.820 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:16.092 INFO:teuthology.orchestra.run.smithi094.stdout:generic/340 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:16.097 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/340 2024-04-25T19:07:16.097 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/340 2024-04-25T19:07:16.097 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:16.100 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:16.109 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:16.109 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2024-04-25T19:07:16.311 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:16.315 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:16.315 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:16.545 INFO:teuthology.orchestra.run.smithi094.stdout:generic/341 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:16.550 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/341 2024-04-25T19:07:16.550 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/341 2024-04-25T19:07:16.550 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:16.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:16.562 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2024-04-25T19:07:16.758 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:16.762 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:16.957 INFO:teuthology.orchestra.run.smithi094.stdout:generic/342 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:16.961 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/342 2024-04-25T19:07:16.961 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/342 2024-04-25T19:07:16.961 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:16.964 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:16.972 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:16.973 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2024-04-25T19:07:17.171 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:17.175 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:17.301 INFO:teuthology.orchestra.run.smithi094.stdout:generic/343 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:17.305 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/343 2024-04-25T19:07:17.305 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/343 2024-04-25T19:07:17.305 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:17.315 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:17.315 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2024-04-25T19:07:17.500 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:17.504 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:17.504 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:17.745 INFO:teuthology.orchestra.run.smithi094.stdout:generic/344 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:17.749 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/344 2024-04-25T19:07:17.750 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/344 2024-04-25T19:07:17.750 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:17.761 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:17.761 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2024-04-25T19:07:17.952 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:17.956 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:18.068 INFO:teuthology.orchestra.run.smithi094.stdout:generic/345 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:18.071 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/345 2024-04-25T19:07:18.071 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/345 2024-04-25T19:07:18.071 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:18.074 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:18.081 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:18.082 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2024-04-25T19:07:18.265 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:18.269 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:18.269 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:18.451 INFO:teuthology.orchestra.run.smithi094.stdout:generic/346 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:18.455 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/346 2024-04-25T19:07:18.455 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/346 2024-04-25T19:07:18.455 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:18.458 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:18.467 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:18.467 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2024-04-25T19:07:18.654 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:18.658 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:18.658 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:18.937 INFO:teuthology.orchestra.run.smithi094.stdout:generic/347 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:18.941 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/347 2024-04-25T19:07:18.941 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/347 2024-04-25T19:07:18.941 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:18.943 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:18.952 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:18.952 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2024-04-25T19:07:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:19.141 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:19.142 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:19.765 INFO:teuthology.orchestra.run.smithi094.stdout:generic/348 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:19.771 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/348 2024-04-25T19:07:19.771 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/348 2024-04-25T19:07:19.772 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:19.775 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:19.788 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:19.789 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2024-04-25T19:07:20.004 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:20.008 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:20.008 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:22.274 INFO:teuthology.orchestra.run.smithi094.stdout:generic/349 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-25T19:07:22.279 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/349 2024-04-25T19:07:22.279 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/349 2024-04-25T19:07:22.279 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:22.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:22.293 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2024-04-25T19:07:22.506 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:22.511 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:25.568 INFO:teuthology.orchestra.run.smithi094.stdout:generic/350 3s 2024-04-25T19:07:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/350 2024-04-25T19:07:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:25.582 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:25.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:25.596 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2024-04-25T19:07:25.832 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:25.836 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:25.836 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:26.905 INFO:teuthology.orchestra.run.smithi094.stdout:generic/351 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-25T19:07:26.909 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/351 2024-04-25T19:07:26.909 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/351 2024-04-25T19:07:26.909 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:26.912 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:26.920 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:26.921 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2024-04-25T19:07:27.502 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:27.506 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:27.506 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:27.633 INFO:teuthology.orchestra.run.smithi094.stdout:generic/352 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:27.636 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/352 2024-04-25T19:07:27.636 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/352 2024-04-25T19:07:27.636 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:27.639 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:27.647 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:27.647 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2024-04-25T19:07:27.831 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:27.835 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:27.835 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:27.937 INFO:teuthology.orchestra.run.smithi094.stdout:generic/353 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:27.941 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/353 2024-04-25T19:07:27.941 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/353 2024-04-25T19:07:27.941 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:27.943 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:27.951 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:27.951 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2024-04-25T19:07:28.130 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:28.135 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:28.135 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:28.236 INFO:teuthology.orchestra.run.smithi094.stdout:generic/354 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:28.240 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/354 2024-04-25T19:07:28.240 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/354 2024-04-25T19:07:28.240 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:28.242 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:28.250 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:28.250 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2024-04-25T19:07:28.435 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:28.439 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:28.439 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:30.223 INFO:teuthology.orchestra.run.smithi094.stdout:generic/355 2s 2024-04-25T19:07:30.229 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/355 2024-04-25T19:07:30.229 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:30.241 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:30.241 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/356) 2024-04-25T19:07:30.811 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:30.815 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:30.815 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:31.159 INFO:teuthology.orchestra.run.smithi094.stdout:generic/356 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:31.163 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/356 2024-04-25T19:07:31.163 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/356 2024-04-25T19:07:31.163 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:31.166 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:31.175 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:31.176 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/357) 2024-04-25T19:07:31.362 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:31.367 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:31.367 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:31.471 INFO:teuthology.orchestra.run.smithi094.stdout:generic/357 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/357 2024-04-25T19:07:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/357 2024-04-25T19:07:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:31.477 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:31.484 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:31.485 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/358) 2024-04-25T19:07:31.677 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:31.681 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:31.681 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:31.792 INFO:teuthology.orchestra.run.smithi094.stdout:generic/358 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:31.796 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/358 2024-04-25T19:07:31.796 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/358 2024-04-25T19:07:31.796 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:31.807 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:31.807 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/359) 2024-04-25T19:07:31.985 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:31.989 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:31.989 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:32.107 INFO:teuthology.orchestra.run.smithi094.stdout:generic/359 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:32.111 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/359 2024-04-25T19:07:32.111 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/359 2024-04-25T19:07:32.111 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:32.114 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:32.121 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:32.121 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/360) 2024-04-25T19:07:32.309 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:32.313 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:32.313 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:32.751 INFO:teuthology.orchestra.run.smithi094.stdout:generic/360 0s 2024-04-25T19:07:32.757 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/360 2024-04-25T19:07:32.757 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:32.760 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:32.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:32.774 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/361) 2024-04-25T19:07:33.008 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:33.012 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:33.012 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:33.276 INFO:teuthology.orchestra.run.smithi094.stdout:generic/361 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:33.284 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/361 2024-04-25T19:07:33.284 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/361 2024-04-25T19:07:33.284 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:33.287 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:33.298 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:33.298 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/362) 2024-04-25T19:07:33.491 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:33.495 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:33.495 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:33.615 INFO:teuthology.orchestra.run.smithi094.stdout:generic/362 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:33.619 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/362 2024-04-25T19:07:33.619 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/362 2024-04-25T19:07:33.619 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:33.621 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:33.629 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:33.630 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/363) 2024-04-25T19:07:33.816 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:33.819 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:33.819 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:33.920 INFO:teuthology.orchestra.run.smithi094.stdout:generic/363 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:33.924 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/363 2024-04-25T19:07:33.924 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/363 2024-04-25T19:07:33.924 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:33.926 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:33.934 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:33.934 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/364) 2024-04-25T19:07:34.113 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:34.117 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:34.238 INFO:teuthology.orchestra.run.smithi094.stdout:generic/364 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:34.242 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/364 2024-04-25T19:07:34.242 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/364 2024-04-25T19:07:34.242 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:34.244 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:34.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:34.252 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/365) 2024-04-25T19:07:34.435 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:34.439 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:34.555 INFO:teuthology.orchestra.run.smithi094.stdout:generic/365 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:34.558 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/365 2024-04-25T19:07:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/365 2024-04-25T19:07:34.559 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:34.569 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:34.570 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/366) 2024-04-25T19:07:34.753 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:34.757 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:34.757 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout:generic/366 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/366 2024-04-25T19:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/366 2024-04-25T19:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:35.353 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:35.353 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/367) 2024-04-25T19:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout:generic/367 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/367 2024-04-25T19:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/367 2024-04-25T19:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:35.700 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:35.700 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/368) 2024-04-25T19:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout:generic/368 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/368 2024-04-25T19:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/368 2024-04-25T19:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:36.014 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:36.015 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/369) 2024-04-25T19:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout:generic/369 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/369 2024-04-25T19:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/369 2024-04-25T19:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:36.321 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:36.321 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/370) 2024-04-25T19:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:36.614 INFO:teuthology.orchestra.run.smithi094.stdout:generic/370 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:36.618 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/370 2024-04-25T19:07:36.618 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/370 2024-04-25T19:07:36.618 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:36.620 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:36.628 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:36.628 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/371) 2024-04-25T19:07:36.809 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:36.813 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:36.813 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:36.939 INFO:teuthology.orchestra.run.smithi094.stdout:generic/371 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:36.943 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/371 2024-04-25T19:07:36.943 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/371 2024-04-25T19:07:36.943 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:36.945 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:36.953 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:36.953 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/372) 2024-04-25T19:07:37.135 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:37.139 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:37.139 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:37.243 INFO:teuthology.orchestra.run.smithi094.stdout:generic/372 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:37.246 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/372 2024-04-25T19:07:37.246 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/372 2024-04-25T19:07:37.246 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:37.248 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:37.256 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:37.256 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/373) 2024-04-25T19:07:37.438 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:37.442 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:37.442 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:37.572 INFO:teuthology.orchestra.run.smithi094.stdout:generic/373 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:37.576 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/373 2024-04-25T19:07:37.576 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/373 2024-04-25T19:07:37.576 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:37.578 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:37.586 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:37.587 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/374) 2024-04-25T19:07:37.770 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:37.774 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:37.774 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:37.879 INFO:teuthology.orchestra.run.smithi094.stdout:generic/374 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:37.882 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/374 2024-04-25T19:07:37.883 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/374 2024-04-25T19:07:37.883 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:37.885 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:37.893 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:37.893 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/375) 2024-04-25T19:07:38.074 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:38.079 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:38.079 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:38.312 INFO:teuthology.orchestra.run.smithi094.stdout:generic/375 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-25T19:07:38.316 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/375 2024-04-25T19:07:38.316 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/375 2024-04-25T19:07:38.316 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:38.319 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:38.327 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:38.327 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/376) 2024-04-25T19:07:38.516 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:38.520 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:38.520 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:generic/376 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:38.729 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/376 2024-04-25T19:07:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/376 2024-04-25T19:07:38.730 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:38.732 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:38.740 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:38.740 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/377) 2024-04-25T19:07:38.923 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:38.927 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:38.928 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:39.044 INFO:teuthology.orchestra.run.smithi094.stdout:generic/377 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:39.048 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/377 2024-04-25T19:07:39.048 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/377 2024-04-25T19:07:39.048 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:39.058 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:39.059 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/378) 2024-04-25T19:07:39.242 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:39.247 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:39.247 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:39.665 INFO:teuthology.orchestra.run.smithi094.stdout:generic/378 0s 2024-04-25T19:07:39.671 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/378 2024-04-25T19:07:39.671 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:39.674 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:39.682 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:39.682 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/379) 2024-04-25T19:07:40.238 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:40.242 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:40.243 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:40.421 INFO:teuthology.orchestra.run.smithi094.stdout:generic/379 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:40.425 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/379 2024-04-25T19:07:40.426 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/379 2024-04-25T19:07:40.426 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:40.437 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:40.437 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/380) 2024-04-25T19:07:40.628 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:40.633 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:40.633 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:40.744 INFO:teuthology.orchestra.run.smithi094.stdout:generic/380 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:40.748 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/380 2024-04-25T19:07:40.748 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/380 2024-04-25T19:07:40.748 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:40.750 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:40.758 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:40.758 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/381) 2024-04-25T19:07:40.940 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:40.944 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:40.944 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:41.050 INFO:teuthology.orchestra.run.smithi094.stdout:generic/381 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:41.053 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/381 2024-04-25T19:07:41.054 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/381 2024-04-25T19:07:41.054 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:41.056 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:41.064 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:41.064 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/382) 2024-04-25T19:07:41.258 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:41.262 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:41.262 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:41.389 INFO:teuthology.orchestra.run.smithi094.stdout:generic/382 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:41.393 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/382 2024-04-25T19:07:41.393 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/382 2024-04-25T19:07:41.393 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:41.395 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:41.403 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:41.403 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/383) 2024-04-25T19:07:41.595 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:41.599 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:41.599 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:41.707 INFO:teuthology.orchestra.run.smithi094.stdout:generic/383 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:41.711 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/383 2024-04-25T19:07:41.711 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/383 2024-04-25T19:07:41.711 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:41.722 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:41.731 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:41.731 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/384) 2024-04-25T19:07:41.916 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:41.920 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:41.920 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:generic/384 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-25T19:07:42.040 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/384 2024-04-25T19:07:42.041 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/384 2024-04-25T19:07:42.041 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:42.043 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:42.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:42.051 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/385) 2024-04-25T19:07:42.234 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:42.238 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:42.238 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:42.361 INFO:teuthology.orchestra.run.smithi094.stdout:generic/385 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:42.364 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/385 2024-04-25T19:07:42.364 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/385 2024-04-25T19:07:42.364 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:42.367 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:42.375 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:42.375 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/386) 2024-04-25T19:07:42.561 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:42.565 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:42.565 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:42.675 INFO:teuthology.orchestra.run.smithi094.stdout:generic/386 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-25T19:07:42.679 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/386 2024-04-25T19:07:42.679 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/386 2024-04-25T19:07:42.679 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:42.681 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:42.689 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:42.690 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/387) 2024-04-25T19:07:42.874 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:42.878 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:42.878 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:42.986 INFO:teuthology.orchestra.run.smithi094.stdout:generic/387 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:42.990 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/387 2024-04-25T19:07:42.990 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/387 2024-04-25T19:07:42.990 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:42.992 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:43.000 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:43.001 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/388) 2024-04-25T19:07:43.185 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:43.189 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:43.190 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:43.324 INFO:teuthology.orchestra.run.smithi094.stdout:generic/388 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:43.327 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/388 2024-04-25T19:07:43.327 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/388 2024-04-25T19:07:43.327 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:43.329 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:43.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:43.337 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/389) 2024-04-25T19:07:43.517 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:43.521 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:43.521 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:43.675 INFO:teuthology.orchestra.run.smithi094.stdout:generic/389 [not run] O_TMPFILE is not supported 2024-04-25T19:07:43.679 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/389 2024-04-25T19:07:43.679 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/389 2024-04-25T19:07:43.679 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:43.681 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:43.689 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:43.689 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/390) 2024-04-25T19:07:43.879 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:43.884 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:43.884 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:44.049 INFO:teuthology.orchestra.run.smithi094.stdout:generic/390 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:07:44.054 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/390 2024-04-25T19:07:44.054 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/390 2024-04-25T19:07:44.054 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:07:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:07:44.064 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:07:44.064 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/391) 2024-04-25T19:07:44.243 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:07:44.247 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:07:44.248 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:24.775 INFO:teuthology.orchestra.run.smithi094.stdout:generic/391 40s 2024-04-25T19:08:25.305 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/391 2024-04-25T19:08:25.306 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:25.311 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:25.403 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:25.403 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/392) 2024-04-25T19:08:28.464 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:28.472 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:28.472 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:28.755 INFO:teuthology.orchestra.run.smithi094.stdout:generic/392 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:28.759 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/392 2024-04-25T19:08:28.759 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/392 2024-04-25T19:08:28.759 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:28.762 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:28.776 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:28.776 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/393) 2024-04-25T19:08:29.123 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:29.131 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:29.131 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:29.281 INFO:teuthology.orchestra.run.smithi094.stdout:generic/393 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:29.292 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/393 2024-04-25T19:08:29.292 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/393 2024-04-25T19:08:29.292 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:29.297 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:29.307 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:29.308 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/394) 2024-04-25T19:08:29.693 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:29.711 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:29.711 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:30.006 INFO:teuthology.orchestra.run.smithi094.stdout:generic/394 1s 2024-04-25T19:08:30.014 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/394 2024-04-25T19:08:30.014 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:30.016 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:30.048 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:30.049 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/395) 2024-04-25T19:08:30.726 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:30.732 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:30.732 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:generic/395 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/395 2024-04-25T19:08:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/395 2024-04-25T19:08:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:30.982 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:30.999 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:30.999 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/396) 2024-04-25T19:08:31.447 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:31.451 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:31.451 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:31.659 INFO:teuthology.orchestra.run.smithi094.stdout:generic/396 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:31.663 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/396 2024-04-25T19:08:31.663 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/396 2024-04-25T19:08:31.663 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:31.666 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:31.685 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:31.685 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/397) 2024-04-25T19:08:32.085 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:32.090 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:32.090 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:generic/397 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/397 2024-04-25T19:08:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/397 2024-04-25T19:08:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:32.357 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:32.366 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:32.366 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/398) 2024-04-25T19:08:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:32.552 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:32.683 INFO:teuthology.orchestra.run.smithi094.stdout:generic/398 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:32.686 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/398 2024-04-25T19:08:32.686 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/398 2024-04-25T19:08:32.686 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:32.689 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:32.696 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:32.697 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/399) 2024-04-25T19:08:32.876 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:32.880 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:32.881 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:33.013 INFO:teuthology.orchestra.run.smithi094.stdout:generic/399 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:33.016 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/399 2024-04-25T19:08:33.016 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/399 2024-04-25T19:08:33.016 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:33.018 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:33.026 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:33.026 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/400) 2024-04-25T19:08:33.219 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:33.223 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:33.223 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:33.330 INFO:teuthology.orchestra.run.smithi094.stdout:generic/400 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-25T19:08:33.333 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/400 2024-04-25T19:08:33.333 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/400 2024-04-25T19:08:33.333 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:33.336 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:33.343 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:33.343 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/401) 2024-04-25T19:08:33.578 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:33.582 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:33.587 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:33.687 INFO:teuthology.orchestra.run.smithi094.stdout:generic/401 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:33.691 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/401 2024-04-25T19:08:33.691 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/401 2024-04-25T19:08:33.691 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:33.693 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:33.701 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:33.701 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/402) 2024-04-25T19:08:33.885 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:33.889 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:33.889 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:34.014 INFO:teuthology.orchestra.run.smithi094.stdout:generic/402 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:34.017 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/402 2024-04-25T19:08:34.018 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/402 2024-04-25T19:08:34.018 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:34.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:34.028 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/403) 2024-04-25T19:08:34.211 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:34.214 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:34.214 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:34.319 INFO:teuthology.orchestra.run.smithi094.stdout:generic/403 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:34.322 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/403 2024-04-25T19:08:34.322 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/403 2024-04-25T19:08:34.322 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:34.325 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:34.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:34.332 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/404) 2024-04-25T19:08:34.524 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:34.528 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:35.534 INFO:teuthology.orchestra.run.smithi094.stdout:generic/404 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-25T19:08:35.538 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/404 2024-04-25T19:08:35.538 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/404 2024-04-25T19:08:35.538 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:35.549 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:35.550 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/405) 2024-04-25T19:08:36.108 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:36.112 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:36.391 INFO:teuthology.orchestra.run.smithi094.stdout:generic/405 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:36.395 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/405 2024-04-25T19:08:36.395 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/405 2024-04-25T19:08:36.395 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:36.408 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:36.408 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/406) 2024-04-25T19:08:36.595 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:36.599 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:36.599 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:36.703 INFO:teuthology.orchestra.run.smithi094.stdout:generic/406 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:36.707 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/406 2024-04-25T19:08:36.707 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/406 2024-04-25T19:08:36.707 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:36.710 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:36.718 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:36.718 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/407) 2024-04-25T19:08:36.901 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:36.905 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:36.905 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:37.072 INFO:teuthology.orchestra.run.smithi094.stdout:generic/407 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T19:08:37.076 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/407 2024-04-25T19:08:37.076 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/407 2024-04-25T19:08:37.076 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:37.078 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:37.086 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:37.086 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/408) 2024-04-25T19:08:37.640 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:37.644 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:37.644 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:37.808 INFO:teuthology.orchestra.run.smithi094.stdout:generic/408 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-25T19:08:37.811 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/408 2024-04-25T19:08:37.811 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/408 2024-04-25T19:08:37.811 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:37.814 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:37.821 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:37.822 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/409) 2024-04-25T19:08:38.384 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:38.389 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:38.389 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:38.501 INFO:teuthology.orchestra.run.smithi094.stdout:generic/409 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:38.505 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/409 2024-04-25T19:08:38.505 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/409 2024-04-25T19:08:38.505 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:38.508 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:38.516 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:38.516 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/410) 2024-04-25T19:08:38.701 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:38.705 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:38.705 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:38.812 INFO:teuthology.orchestra.run.smithi094.stdout:generic/410 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:38.816 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/410 2024-04-25T19:08:38.816 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/410 2024-04-25T19:08:38.816 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:38.826 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:38.826 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/411) 2024-04-25T19:08:39.007 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:39.011 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:39.117 INFO:teuthology.orchestra.run.smithi094.stdout:generic/411 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:39.121 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/411 2024-04-25T19:08:39.121 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/411 2024-04-25T19:08:39.121 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:39.123 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:39.131 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:39.131 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/412) 2024-04-25T19:08:39.313 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:39.317 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:39.317 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:39.427 INFO:teuthology.orchestra.run.smithi094.stdout:generic/412 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:39.431 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/412 2024-04-25T19:08:39.431 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/412 2024-04-25T19:08:39.431 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:39.433 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:39.441 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:39.441 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/413) 2024-04-25T19:08:39.621 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:39.625 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:39.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:39.759 INFO:teuthology.orchestra.run.smithi094.stdout:generic/413 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:39.762 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/413 2024-04-25T19:08:39.762 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/413 2024-04-25T19:08:39.763 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:39.765 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:39.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:39.773 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/414) 2024-04-25T19:08:39.955 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:39.959 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:40.063 INFO:teuthology.orchestra.run.smithi094.stdout:generic/414 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:40.066 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/414 2024-04-25T19:08:40.067 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/414 2024-04-25T19:08:40.067 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:40.076 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:40.076 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/415) 2024-04-25T19:08:40.272 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:40.276 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:40.276 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:40.381 INFO:teuthology.orchestra.run.smithi094.stdout:generic/415 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:40.384 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/415 2024-04-25T19:08:40.385 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/415 2024-04-25T19:08:40.385 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:40.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:40.395 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/416) 2024-04-25T19:08:40.584 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:40.588 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:40.589 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:40.695 INFO:teuthology.orchestra.run.smithi094.stdout:generic/416 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:40.698 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/416 2024-04-25T19:08:40.698 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/416 2024-04-25T19:08:40.698 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:40.701 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:40.709 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:40.709 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/417) 2024-04-25T19:08:40.892 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:40.896 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:40.999 INFO:teuthology.orchestra.run.smithi094.stdout:generic/417 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:41.003 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/417 2024-04-25T19:08:41.003 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/417 2024-04-25T19:08:41.003 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:41.006 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:41.015 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:41.015 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/418) 2024-04-25T19:08:41.195 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:41.199 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:41.200 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:41.397 INFO:teuthology.orchestra.run.smithi094.stdout:generic/418 [not run] require ceph-fuse to be valid block disk 2024-04-25T19:08:41.403 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/418 2024-04-25T19:08:41.403 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/418 2024-04-25T19:08:41.403 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:41.406 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:41.415 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:41.415 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/419) 2024-04-25T19:08:41.620 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:41.624 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:41.624 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:41.758 INFO:teuthology.orchestra.run.smithi094.stdout:generic/419 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:41.761 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/419 2024-04-25T19:08:41.761 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/419 2024-04-25T19:08:41.761 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:41.763 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:41.771 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:41.772 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/420) 2024-04-25T19:08:41.963 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:41.967 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:41.967 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:43.004 INFO:teuthology.orchestra.run.smithi094.stdout:generic/420 2s 2024-04-25T19:08:43.010 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/420 2024-04-25T19:08:43.010 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:43.012 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:43.268 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:43.269 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/421) 2024-04-25T19:08:43.596 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:43.602 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:43.729 INFO:teuthology.orchestra.run.smithi094.stdout:generic/421 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:43.732 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/421 2024-04-25T19:08:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/421 2024-04-25T19:08:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:43.735 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:43.743 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:43.744 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/422) 2024-04-25T19:08:43.930 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:43.934 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:43.935 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:44.042 INFO:teuthology.orchestra.run.smithi094.stdout:generic/422 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:44.045 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/422 2024-04-25T19:08:44.046 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/422 2024-04-25T19:08:44.046 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:44.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:44.056 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/423) 2024-04-25T19:08:44.236 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:44.240 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:44.240 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:44.567 INFO:teuthology.orchestra.run.smithi094.stdout:generic/423 0s 2024-04-25T19:08:44.572 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/423 2024-04-25T19:08:44.572 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:44.582 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:44.583 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/424) 2024-04-25T19:08:45.134 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:45.138 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:45.138 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:45.313 INFO:teuthology.orchestra.run.smithi094.stdout:generic/424 [not run] file system doesn't support any of /usr/bin/chattr +a/+c/+d/+i 2024-04-25T19:08:45.317 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/424 2024-04-25T19:08:45.317 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/424 2024-04-25T19:08:45.317 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:45.319 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:45.327 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:45.327 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/425) 2024-04-25T19:08:45.507 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:45.511 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:45.614 INFO:teuthology.orchestra.run.smithi094.stdout:generic/425 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:08:45.618 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/425 2024-04-25T19:08:45.618 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/425 2024-04-25T19:08:45.618 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:08:45.620 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:08:45.628 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:08:45.628 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/426) 2024-04-25T19:08:45.811 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:08:45.815 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:08:45.816 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:32.509 INFO:teuthology.orchestra.run.smithi094.stdout:generic/426 - output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/426.out.bad) 2024-04-25T19:09:32.573 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/426.out 2024-04-25 16:52:50.095099710 +0000 2024-04-25T19:09:32.573 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/426.out.bad 2024-04-25 19:09:32.461456771 +0000 2024-04-25T19:09:32.573 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -1,5 +1,397 @@ 2024-04-25T19:09:32.573 INFO:teuthology.orchestra.run.smithi094.stdout: QA output created by 426 2024-04-25T19:09:32.573 INFO:teuthology.orchestra.run.smithi094.stdout: test_file_handles TEST_DIR/426-dir -d 2024-04-25T19:09:32.573 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000632) opened an unlinked file! 2024-04-25T19:09:32.573 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000633) opened an unlinked file! 2024-04-25T19:09:32.573 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000634) opened an unlinked file! 2024-04-25T19:09:32.574 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000635) opened an unlinked file! 2024-04-25T19:09:32.574 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000636) opened an unlinked file! 2024-04-25T19:09:32.574 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000637) opened an unlinked file! 2024-04-25T19:09:32.574 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000638) opened an unlinked file! 2024-04-25T19:09:32.574 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000639) opened an unlinked file! 2024-04-25T19:09:32.574 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000640) opened an unlinked file! 2024-04-25T19:09:32.574 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000641) opened an unlinked file! 2024-04-25T19:09:32.574 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000642) opened an unlinked file! 2024-04-25T19:09:32.574 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000643) opened an unlinked file! 2024-04-25T19:09:32.574 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000644) opened an unlinked file! 2024-04-25T19:09:32.574 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000645) opened an unlinked file! 2024-04-25T19:09:32.574 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000646) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000647) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000648) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000649) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000650) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000651) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000652) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000653) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000654) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000655) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000656) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000657) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000658) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000659) opened an unlinked file! 2024-04-25T19:09:32.575 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000660) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000661) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000662) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000663) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000664) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000665) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000666) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000667) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000668) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000669) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000670) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000671) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000672) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000673) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000674) opened an unlinked file! 2024-04-25T19:09:32.576 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000675) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000676) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000677) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000678) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000679) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000680) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000681) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000682) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000683) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000684) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000685) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000686) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000687) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000688) opened an unlinked file! 2024-04-25T19:09:32.577 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000689) opened an unlinked file! 2024-04-25T19:09:32.578 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000690) opened an unlinked file! 2024-04-25T19:09:32.578 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000691) opened an unlinked file! 2024-04-25T19:09:32.578 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000692) opened an unlinked file! 2024-04-25T19:09:32.578 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000693) opened an unlinked file! 2024-04-25T19:09:32.578 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000694) opened an unlinked file! 2024-04-25T19:09:32.578 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000695) opened an unlinked file! 2024-04-25T19:09:32.578 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000696) opened an unlinked file! 2024-04-25T19:09:32.578 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000697) opened an unlinked file! 2024-04-25T19:09:32.578 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000698) opened an unlinked file! 2024-04-25T19:09:32.578 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000699) opened an unlinked file! 2024-04-25T19:09:32.578 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000700) opened an unlinked file! 2024-04-25T19:09:32.578 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000701) opened an unlinked file! 2024-04-25T19:09:32.578 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000702) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000703) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000704) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000705) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000706) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000707) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000708) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000709) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000710) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000711) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000712) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000713) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000714) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000715) opened an unlinked file! 2024-04-25T19:09:32.579 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000716) opened an unlinked file! 2024-04-25T19:09:32.580 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000717) opened an unlinked file! 2024-04-25T19:09:32.580 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000718) opened an unlinked file! 2024-04-25T19:09:32.580 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000719) opened an unlinked file! 2024-04-25T19:09:32.580 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000720) opened an unlinked file! 2024-04-25T19:09:32.580 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000721) opened an unlinked file! 2024-04-25T19:09:32.580 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000722) opened an unlinked file! 2024-04-25T19:09:32.580 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000723) opened an unlinked file! 2024-04-25T19:09:32.580 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000724) opened an unlinked file! 2024-04-25T19:09:32.580 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000725) opened an unlinked file! 2024-04-25T19:09:32.580 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000726) opened an unlinked file! 2024-04-25T19:09:32.580 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000727) opened an unlinked file! 2024-04-25T19:09:32.580 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000728) opened an unlinked file! 2024-04-25T19:09:32.580 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000729) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000730) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000731) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000732) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000733) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000734) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000735) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000736) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000737) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000738) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000739) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000740) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000741) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000742) opened an unlinked file! 2024-04-25T19:09:32.581 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000743) opened an unlinked file! 2024-04-25T19:09:32.582 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000744) opened an unlinked file! 2024-04-25T19:09:32.582 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000745) opened an unlinked file! 2024-04-25T19:09:32.582 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000746) opened an unlinked file! 2024-04-25T19:09:32.582 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000747) opened an unlinked file! 2024-04-25T19:09:32.582 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000748) opened an unlinked file! 2024-04-25T19:09:32.582 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000749) opened an unlinked file! 2024-04-25T19:09:32.582 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000750) opened an unlinked file! 2024-04-25T19:09:32.582 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000751) opened an unlinked file! 2024-04-25T19:09:32.582 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000752) opened an unlinked file! 2024-04-25T19:09:32.582 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000753) opened an unlinked file! 2024-04-25T19:09:32.582 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000754) opened an unlinked file! 2024-04-25T19:09:32.582 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000755) opened an unlinked file! 2024-04-25T19:09:32.582 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000756) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000757) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000758) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000759) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000760) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000761) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000762) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000763) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000764) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000765) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000766) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000767) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000768) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000769) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000770) opened an unlinked file! 2024-04-25T19:09:32.583 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000771) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000772) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000773) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000774) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000775) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000776) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000777) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000778) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000779) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000780) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000781) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000782) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000783) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000784) opened an unlinked file! 2024-04-25T19:09:32.584 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000785) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000786) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000787) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000788) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000789) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000790) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000791) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000792) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000793) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000794) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000795) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000796) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000797) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000798) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000799) opened an unlinked file! 2024-04-25T19:09:32.585 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000800) opened an unlinked file! 2024-04-25T19:09:32.586 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000801) opened an unlinked file! 2024-04-25T19:09:32.586 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000802) opened an unlinked file! 2024-04-25T19:09:32.586 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000803) opened an unlinked file! 2024-04-25T19:09:32.586 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000804) opened an unlinked file! 2024-04-25T19:09:32.586 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000805) opened an unlinked file! 2024-04-25T19:09:32.586 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000806) opened an unlinked file! 2024-04-25T19:09:32.586 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000807) opened an unlinked file! 2024-04-25T19:09:32.586 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000808) opened an unlinked file! 2024-04-25T19:09:32.586 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000809) opened an unlinked file! 2024-04-25T19:09:32.586 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000810) opened an unlinked file! 2024-04-25T19:09:32.586 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000811) opened an unlinked file! 2024-04-25T19:09:32.586 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000812) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000813) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000814) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000815) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000816) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000817) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000818) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000819) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000820) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000821) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000822) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000823) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000824) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000825) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000826) opened an unlinked file! 2024-04-25T19:09:32.587 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000827) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000828) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000829) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000830) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000831) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000832) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000833) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000834) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000835) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000836) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000837) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000838) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000839) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000840) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000841) opened an unlinked file! 2024-04-25T19:09:32.588 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000842) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000843) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000844) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000845) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000846) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000847) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000848) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000849) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000850) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000851) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000852) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000853) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000854) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000855) opened an unlinked file! 2024-04-25T19:09:32.589 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000856) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000857) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000858) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000859) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000860) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000861) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000862) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000863) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000864) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000865) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000866) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000867) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000868) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000869) opened an unlinked file! 2024-04-25T19:09:32.590 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000870) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000871) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000872) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000873) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000874) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000875) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000876) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000877) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000878) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000879) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000880) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000881) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000882) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000883) opened an unlinked file! 2024-04-25T19:09:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000884) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000885) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000886) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000887) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000888) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000889) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000890) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000891) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000892) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000893) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000894) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000895) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000896) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000897) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000898) opened an unlinked file! 2024-04-25T19:09:32.592 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000899) opened an unlinked file! 2024-04-25T19:09:32.593 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000900) opened an unlinked file! 2024-04-25T19:09:32.593 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000901) opened an unlinked file! 2024-04-25T19:09:32.593 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000902) opened an unlinked file! 2024-04-25T19:09:32.593 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000903) opened an unlinked file! 2024-04-25T19:09:32.593 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000904) opened an unlinked file! 2024-04-25T19:09:32.593 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000905) opened an unlinked file! 2024-04-25T19:09:32.593 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000906) opened an unlinked file! 2024-04-25T19:09:32.593 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000907) opened an unlinked file! 2024-04-25T19:09:32.593 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000908) opened an unlinked file! 2024-04-25T19:09:32.593 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000909) opened an unlinked file! 2024-04-25T19:09:32.593 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000910) opened an unlinked file! 2024-04-25T19:09:32.593 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000911) opened an unlinked file! 2024-04-25T19:09:32.593 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000912) opened an unlinked file! 2024-04-25T19:09:32.594 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000913) opened an unlinked file! 2024-04-25T19:09:32.594 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000914) opened an unlinked file! 2024-04-25T19:09:32.594 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000915) opened an unlinked file! 2024-04-25T19:09:32.594 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000916) opened an unlinked file! 2024-04-25T19:09:32.594 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000917) opened an unlinked file! 2024-04-25T19:09:32.594 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000918) opened an unlinked file! 2024-04-25T19:09:32.594 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000919) opened an unlinked file! 2024-04-25T19:09:32.594 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000920) opened an unlinked file! 2024-04-25T19:09:32.594 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000921) opened an unlinked file! 2024-04-25T19:09:32.594 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000922) opened an unlinked file! 2024-04-25T19:09:32.594 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000923) opened an unlinked file! 2024-04-25T19:09:32.594 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000924) opened an unlinked file! 2024-04-25T19:09:32.594 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000925) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000926) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000927) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000928) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000929) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000930) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000931) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000932) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000933) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000934) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000935) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000936) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000937) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000938) opened an unlinked file! 2024-04-25T19:09:32.595 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000939) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000940) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000941) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000942) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000943) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000944) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000945) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000946) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000947) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000948) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000949) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000950) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000951) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000952) opened an unlinked file! 2024-04-25T19:09:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000953) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000954) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000955) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000956) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000957) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000958) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000959) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000960) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000961) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000962) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000963) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000964) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000965) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000966) opened an unlinked file! 2024-04-25T19:09:32.597 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000967) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000968) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000969) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000970) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000971) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000972) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000973) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000974) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000975) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000976) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000977) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000978) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000979) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000980) opened an unlinked file! 2024-04-25T19:09:32.598 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000981) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000982) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000983) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000984) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000985) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000986) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000987) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000988) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000989) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000990) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000991) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000992) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000993) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000994) opened an unlinked file! 2024-04-25T19:09:32.599 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000995) opened an unlinked file! 2024-04-25T19:09:32.600 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000996) opened an unlinked file! 2024-04-25T19:09:32.600 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000997) opened an unlinked file! 2024-04-25T19:09:32.600 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000998) opened an unlinked file! 2024-04-25T19:09:32.600 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file000999) opened an unlinked file! 2024-04-25T19:09:32.600 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001000) opened an unlinked file! 2024-04-25T19:09:32.600 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001001) opened an unlinked file! 2024-04-25T19:09:32.600 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001002) opened an unlinked file! 2024-04-25T19:09:32.600 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001003) opened an unlinked file! 2024-04-25T19:09:32.600 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001004) opened an unlinked file! 2024-04-25T19:09:32.600 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001005) opened an unlinked file! 2024-04-25T19:09:32.600 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001006) opened an unlinked file! 2024-04-25T19:09:32.600 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001007) opened an unlinked file! 2024-04-25T19:09:32.600 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001008) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001009) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001010) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001011) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001012) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001013) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001014) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001015) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001016) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001017) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001018) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001019) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001020) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001021) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001022) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/426-dir/file001023) opened an unlinked file! 2024-04-25T19:09:32.601 INFO:teuthology.orchestra.run.smithi094.stdout: test_file_handles TEST_DIR/426-dir 2024-04-25T19:09:32.602 INFO:teuthology.orchestra.run.smithi094.stdout: test_file_handles TEST_DIR/426-dir -l 2024-04-25T19:09:32.602 INFO:teuthology.orchestra.run.smithi094.stdout: test_file_handles TEST_DIR/426-dir -u 2024-04-25T19:09:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/426 2024-04-25T19:09:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/426 2024-04-25T19:09:32.602 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-25T19:09:32.707 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:32.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:09:32.778 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-25T19:09:32.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-25T19:09:32.778 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/427) 2024-04-25T19:09:36.704 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:09:36.710 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:09:36.710 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:37.062 INFO:teuthology.orchestra.run.smithi094.stdout:generic/427 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:09:37.066 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/427 2024-04-25T19:09:37.066 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/427 2024-04-25T19:09:37.066 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:09:37.069 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:37.079 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:09:37.079 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/428) 2024-04-25T19:09:37.317 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:09:37.321 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:09:37.321 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:37.606 INFO:teuthology.orchestra.run.smithi094.stdout:generic/428 0s 2024-04-25T19:09:37.614 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/428 2024-04-25T19:09:37.614 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:09:37.617 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:37.627 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:09:37.627 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/429) 2024-04-25T19:09:37.879 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:09:37.884 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:09:37.884 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:38.055 INFO:teuthology.orchestra.run.smithi094.stdout:generic/429 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:09:38.060 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/429 2024-04-25T19:09:38.060 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/429 2024-04-25T19:09:38.060 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:09:38.062 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:38.072 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:09:38.072 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/430) 2024-04-25T19:09:38.272 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:09:38.276 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:09:38.277 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:38.602 INFO:teuthology.orchestra.run.smithi094.stdout:generic/430 0s 2024-04-25T19:09:38.608 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/430 2024-04-25T19:09:38.608 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:09:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:38.618 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:09:38.619 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/431) 2024-04-25T19:09:39.202 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:09:39.206 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:09:39.206 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:39.480 INFO:teuthology.orchestra.run.smithi094.stdout:generic/431 0s 2024-04-25T19:09:39.485 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/431 2024-04-25T19:09:39.485 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:09:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:39.496 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:09:39.496 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/432) 2024-04-25T19:09:40.074 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:09:40.078 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:09:40.078 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:40.329 INFO:teuthology.orchestra.run.smithi094.stdout:generic/432 0s 2024-04-25T19:09:40.333 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/432 2024-04-25T19:09:40.334 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:09:40.336 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:40.344 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:09:40.344 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/433) 2024-04-25T19:09:40.916 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:09:40.920 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:09:40.920 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:41.170 INFO:teuthology.orchestra.run.smithi094.stdout:generic/433 1s 2024-04-25T19:09:41.176 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/433 2024-04-25T19:09:41.176 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:09:41.178 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:41.187 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:09:41.187 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/434) 2024-04-25T19:09:41.741 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:09:41.745 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:09:41.745 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:42.013 INFO:teuthology.orchestra.run.smithi094.stdout:generic/434 - output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/434.out.bad) 2024-04-25T19:09:42.014 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/434.out 2024-04-25 16:52:50.095099710 +0000 2024-04-25T19:09:42.015 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/434.out.bad 2024-04-25 19:09:42.009311791 +0000 2024-04-25T19:09:42.015 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -9,6 +9,6 @@ 2024-04-25T19:09:42.015 INFO:teuthology.orchestra.run.smithi094.stdout: copy_range: Bad file descriptor 2024-04-25T19:09:42.015 INFO:teuthology.orchestra.run.smithi094.stdout: d41d8cd98f00b204e9800998ecf8427e TEST_DIR/test-434/copy 2024-04-25T19:09:42.015 INFO:teuthology.orchestra.run.smithi094.stdout: Try to copy to a device 2024-04-25T19:09:42.015 INFO:teuthology.orchestra.run.smithi094.stdout: -copy_range: Invalid argument 2024-04-25T19:09:42.015 INFO:teuthology.orchestra.run.smithi094.stdout: +/tmp/tmp.PS50Hvgg8stest/test-434/dev1: Permission denied 2024-04-25T19:09:42.015 INFO:teuthology.orchestra.run.smithi094.stdout: Try to copy to a fifo 2024-04-25T19:09:42.015 INFO:teuthology.orchestra.run.smithi094.stdout: copy_range: Invalid argument 2024-04-25T19:09:42.017 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/434 2024-04-25T19:09:42.017 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/434 2024-04-25T19:09:42.017 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-25T19:09:42.019 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:42.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:09:42.027 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-25T19:09:42.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-25T19:09:42.028 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/435) 2024-04-25T19:09:42.586 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:09:42.590 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:09:42.590 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:42.718 INFO:teuthology.orchestra.run.smithi094.stdout:generic/435 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:09:42.721 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/435 2024-04-25T19:09:42.722 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/435 2024-04-25T19:09:42.722 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:09:42.724 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:42.732 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:09:42.732 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/436) 2024-04-25T19:09:42.923 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:09:42.927 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:09:42.927 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:45.045 INFO:teuthology.orchestra.run.smithi094.stdout:generic/436 3s 2024-04-25T19:09:45.051 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/436 2024-04-25T19:09:45.051 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:09:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:45.109 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:09:45.110 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/437) 2024-04-25T19:09:45.655 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:09:45.659 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:09:45.659 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:46.493 INFO:teuthology.orchestra.run.smithi094.stdout:generic/437 1s 2024-04-25T19:09:46.498 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/437 2024-04-25T19:09:46.499 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:09:46.501 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:09:46.511 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:09:46.512 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/438) 2024-04-25T19:09:46.700 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:09:46.704 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:09:46.704 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:10:24.209 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:10:24.212 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:10:54.344 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:10:54.346 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:11:04.385 INFO:teuthology.orchestra.run.smithi094.stdout:generic/438 78s 2024-04-25T19:11:04.390 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/438 2024-04-25T19:11:04.390 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:04.392 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:04.444 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:04.444 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/439) 2024-04-25T19:11:04.632 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:04.636 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:04.636 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:04.741 INFO:teuthology.orchestra.run.smithi094.stdout:generic/439 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:04.745 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/439 2024-04-25T19:11:04.745 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/439 2024-04-25T19:11:04.745 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:04.747 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:04.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:04.755 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/440) 2024-04-25T19:11:04.948 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:04.953 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:04.953 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:05.084 INFO:teuthology.orchestra.run.smithi094.stdout:generic/440 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:05.087 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/440 2024-04-25T19:11:05.087 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/440 2024-04-25T19:11:05.087 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:05.090 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:05.098 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:05.098 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/441) 2024-04-25T19:11:05.281 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:05.285 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:05.285 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:05.449 INFO:teuthology.orchestra.run.smithi094.stdout:generic/441 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:05.453 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/441 2024-04-25T19:11:05.453 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/441 2024-04-25T19:11:05.453 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:05.455 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:05.464 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:05.464 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/442) 2024-04-25T19:11:05.657 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:05.661 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:05.779 INFO:teuthology.orchestra.run.smithi094.stdout:generic/442 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:05.783 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/442 2024-04-25T19:11:05.783 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/442 2024-04-25T19:11:05.783 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:05.785 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:05.793 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:05.793 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/443) 2024-04-25T19:11:05.977 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:05.981 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:06.121 INFO:teuthology.orchestra.run.smithi094.stdout:generic/443 1s 2024-04-25T19:11:06.126 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/443 2024-04-25T19:11:06.127 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:06.129 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:06.137 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:06.137 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/444) 2024-04-25T19:11:06.320 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:06.324 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:06.462 INFO:teuthology.orchestra.run.smithi094.stdout:generic/444 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-25T19:11:06.466 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/444 2024-04-25T19:11:06.466 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/444 2024-04-25T19:11:06.466 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:06.468 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:06.476 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:06.476 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/445) 2024-04-25T19:11:06.662 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:06.665 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:06.665 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:08.738 INFO:teuthology.orchestra.run.smithi094.stdout:generic/445 2s 2024-04-25T19:11:08.816 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/445 2024-04-25T19:11:08.816 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:08.820 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:08.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:08.833 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/446) 2024-04-25T19:11:09.431 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:09.435 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:09.435 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:09.551 INFO:teuthology.orchestra.run.smithi094.stdout:generic/446 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:09.555 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/446 2024-04-25T19:11:09.555 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/446 2024-04-25T19:11:09.555 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:09.558 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:09.567 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:09.567 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/447) 2024-04-25T19:11:09.766 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:09.770 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:09.770 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:09.885 INFO:teuthology.orchestra.run.smithi094.stdout:generic/447 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:09.888 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/447 2024-04-25T19:11:09.888 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/447 2024-04-25T19:11:09.888 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:09.891 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:09.899 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:09.899 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/448) 2024-04-25T19:11:10.085 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:10.089 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:10.089 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:12.162 INFO:teuthology.orchestra.run.smithi094.stdout:generic/448 2s 2024-04-25T19:11:12.169 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/448 2024-04-25T19:11:12.169 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:12.172 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:12.183 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:12.183 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/449) 2024-04-25T19:11:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:12.749 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:12.866 INFO:teuthology.orchestra.run.smithi094.stdout:generic/449 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:12.870 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/449 2024-04-25T19:11:12.870 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/449 2024-04-25T19:11:12.871 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:12.873 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:12.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:12.882 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/450) 2024-04-25T19:11:13.076 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:13.080 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:13.080 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:15.214 INFO:teuthology.orchestra.run.smithi094.stdout:generic/450 2s 2024-04-25T19:11:15.221 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/450 2024-04-25T19:11:15.221 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:15.225 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:15.237 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:15.237 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/451) 2024-04-25T19:11:15.795 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:15.799 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:15.799 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:16.334 INFO:teuthology.orchestra.run.smithi094.stdout:generic/451 - output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/451.out.bad) 2024-04-25T19:11:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/451.out 2024-04-25 16:52:50.099099653 +0000 2024-04-25T19:11:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/451.out.bad 2024-04-25 19:11:16.331879612 +0000 2024-04-25T19:11:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -1,2 +1,5 @@ 2024-04-25T19:11:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: QA output created by 451 2024-04-25T19:11:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: +get stale data from buffer read 2024-04-25T19:11:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: +00000000 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 UUUUUUUUUUUUUUUU 2024-04-25T19:11:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: +000a0000 2024-04-25T19:11:16.337 INFO:teuthology.orchestra.run.smithi094.stdout: Silence is golden 2024-04-25T19:11:16.338 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/451 2024-04-25T19:11:16.339 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/451 2024-04-25T19:11:16.339 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-25T19:11:16.341 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:16.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:11:16.349 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-25T19:11:16.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-25T19:11:16.349 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/452) 2024-04-25T19:11:16.869 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:16.873 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:16.873 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:16.975 INFO:teuthology.orchestra.run.smithi094.stdout:generic/452 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:16.979 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/452 2024-04-25T19:11:16.979 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/452 2024-04-25T19:11:16.979 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:16.981 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:16.990 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:16.990 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/453) 2024-04-25T19:11:17.174 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:17.178 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:17.281 INFO:teuthology.orchestra.run.smithi094.stdout:generic/453 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:17.284 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/453 2024-04-25T19:11:17.284 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/453 2024-04-25T19:11:17.284 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:17.294 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:17.295 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/454) 2024-04-25T19:11:17.477 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:17.482 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:17.482 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:17.583 INFO:teuthology.orchestra.run.smithi094.stdout:generic/454 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:17.587 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/454 2024-04-25T19:11:17.587 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/454 2024-04-25T19:11:17.587 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:17.589 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:17.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:17.598 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/455) 2024-04-25T19:11:17.782 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:17.786 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:18.005 INFO:teuthology.orchestra.run.smithi094.stdout:generic/455 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:18.009 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/455 2024-04-25T19:11:18.010 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/455 2024-04-25T19:11:18.010 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:18.012 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:18.021 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:18.021 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/456) 2024-04-25T19:11:18.211 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:18.215 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:18.215 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:18.336 INFO:teuthology.orchestra.run.smithi094.stdout:generic/456 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:18.340 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/456 2024-04-25T19:11:18.340 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/456 2024-04-25T19:11:18.340 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:18.350 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:18.350 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/457) 2024-04-25T19:11:18.533 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:18.537 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:18.537 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:18.727 INFO:teuthology.orchestra.run.smithi094.stdout:generic/457 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:18.731 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/457 2024-04-25T19:11:18.731 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/457 2024-04-25T19:11:18.731 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:18.733 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:18.741 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:18.742 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/458) 2024-04-25T19:11:18.925 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:18.929 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:18.930 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:19.031 INFO:teuthology.orchestra.run.smithi094.stdout:generic/458 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:19.035 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/458 2024-04-25T19:11:19.035 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/458 2024-04-25T19:11:19.035 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:19.037 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:19.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:19.047 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/459) 2024-04-25T19:11:19.229 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:19.233 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:19.233 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:19.561 INFO:teuthology.orchestra.run.smithi094.stdout:generic/459 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:19.567 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/459 2024-04-25T19:11:19.567 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/459 2024-04-25T19:11:19.568 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:19.571 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:19.583 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:19.583 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/460) 2024-04-25T19:11:19.788 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:19.792 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:19.792 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:19.894 INFO:teuthology.orchestra.run.smithi094.stdout:generic/460 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:19.898 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/460 2024-04-25T19:11:19.898 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/460 2024-04-25T19:11:19.898 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:19.900 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:19.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:19.908 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/461) 2024-04-25T19:11:20.091 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:20.095 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:20.095 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:20.199 INFO:teuthology.orchestra.run.smithi094.stdout:generic/461 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:20.202 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/461 2024-04-25T19:11:20.203 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/461 2024-04-25T19:11:20.203 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:20.205 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:20.212 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:20.213 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/462) 2024-04-25T19:11:20.396 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:20.400 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:20.400 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:20.505 INFO:teuthology.orchestra.run.smithi094.stdout:generic/462 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:20.509 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/462 2024-04-25T19:11:20.509 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/462 2024-04-25T19:11:20.510 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:20.520 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:20.520 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/463) 2024-04-25T19:11:20.705 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:20.709 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:20.709 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:20.883 INFO:teuthology.orchestra.run.smithi094.stdout:generic/463 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-25T19:11:20.887 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/463 2024-04-25T19:11:20.887 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/463 2024-04-25T19:11:20.887 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:20.897 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:20.898 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/464) 2024-04-25T19:11:21.436 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:21.441 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:21.441 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:21.542 INFO:teuthology.orchestra.run.smithi094.stdout:generic/464 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:21.545 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/464 2024-04-25T19:11:21.546 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/464 2024-04-25T19:11:21.546 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:21.556 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:21.556 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/465) 2024-04-25T19:11:21.740 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:21.744 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:21.744 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:24.457 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:11:24.461 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:11:47.496 INFO:teuthology.orchestra.run.smithi094.stdout:generic/465 26s 2024-04-25T19:11:47.507 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/465 2024-04-25T19:11:47.507 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:47.511 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:47.643 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:47.643 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/466) 2024-04-25T19:11:48.067 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:48.071 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:48.175 INFO:teuthology.orchestra.run.smithi094.stdout:generic/466 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:48.179 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/466 2024-04-25T19:11:48.179 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/466 2024-04-25T19:11:48.179 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:48.181 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:48.189 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:48.189 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/467) 2024-04-25T19:11:48.373 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:48.377 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:48.377 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:54.579 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:11:54.585 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:11:56.547 INFO:teuthology.orchestra.run.smithi094.stdout:generic/467 - output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/467.out.bad) 2024-04-25T19:11:56.571 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/467.out 2024-04-25 16:52:50.099099653 +0000 2024-04-25T19:11:56.571 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/467.out.bad 2024-04-25 19:11:56.531246640 +0000 2024-04-25T19:11:56.571 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -1,5 +1,15 @@ 2024-04-25T19:11:56.571 INFO:teuthology.orchestra.run.smithi094.stdout: QA output created by 467 2024-04-25T19:11:56.571 INFO:teuthology.orchestra.run.smithi094.stdout: test_file_handles TEST_DIR/467-dir -dp 2024-04-25T19:11:56.571 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/467-dir/file000000) opened an unlinked file! 2024-04-25T19:11:56.571 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/467-dir/file000001) opened an unlinked file! 2024-04-25T19:11:56.571 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/467-dir/file000002) opened an unlinked file! 2024-04-25T19:11:56.571 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/467-dir/file000003) opened an unlinked file! 2024-04-25T19:11:56.571 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/467-dir/file000004) opened an unlinked file! 2024-04-25T19:11:56.571 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/467-dir/file000005) opened an unlinked file! 2024-04-25T19:11:56.571 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/467-dir/file000006) opened an unlinked file! 2024-04-25T19:11:56.572 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/467-dir/file000007) opened an unlinked file! 2024-04-25T19:11:56.572 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/467-dir/file000008) opened an unlinked file! 2024-04-25T19:11:56.572 INFO:teuthology.orchestra.run.smithi094.stdout: +open_by_handle(/tmp/tmp.PS50Hvgg8stest/467-dir/file000009) opened an unlinked file! 2024-04-25T19:11:56.572 INFO:teuthology.orchestra.run.smithi094.stdout: test_file_handles TEST_DIR/467-dir -rp 2024-04-25T19:11:56.572 INFO:teuthology.orchestra.run.smithi094.stdout: test_file_handles TEST_DIR/467-dir -dkr 2024-04-25T19:11:56.572 INFO:teuthology.orchestra.run.smithi094.stdout: test_file_handles TEST_DIR/467-dir -lr 2024-04-25T19:11:56.587 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/467 2024-04-25T19:11:56.587 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/467 2024-04-25T19:11:56.587 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-25T19:11:56.607 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:56.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:11:56.631 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-25T19:11:56.632 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-25T19:11:56.632 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/468) 2024-04-25T19:11:58.237 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:58.242 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:58.453 INFO:teuthology.orchestra.run.smithi094.stdout:generic/468 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:11:58.458 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/468 2024-04-25T19:11:58.458 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/468 2024-04-25T19:11:58.458 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:11:58.461 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:11:58.475 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:11:58.476 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/469) 2024-04-25T19:11:58.696 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:11:58.701 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:11:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:12:00.738 INFO:teuthology.orchestra.run.smithi094.stdout:generic/469 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-25T19:12:00.743 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/469 2024-04-25T19:12:00.743 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/469 2024-04-25T19:12:00.743 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:12:00.746 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:12:00.757 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:12:00.757 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/470) 2024-04-25T19:12:01.333 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:12:01.334 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:12:01.334 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:12:01.515 INFO:teuthology.orchestra.run.smithi094.stdout:generic/470 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:12:01.519 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/470 2024-04-25T19:12:01.519 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/470 2024-04-25T19:12:01.519 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:12:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:12:01.531 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:12:01.532 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/471) 2024-04-25T19:12:01.721 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:12:01.725 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:12:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:12:26.488 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:12:26.491 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:12:56.620 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:12:56.623 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:13:26.715 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:13:26.719 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:13:26.779 INFO:tasks.ceph.mds.a.smithi094.stderr:2024-04-25T19:13:26.773+0000 7fb5271b5640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 357875) UID: 0 2024-04-25T19:13:26.781 INFO:tasks.ceph.mds.c.smithi094.stderr:2024-04-25T19:13:26.773+0000 7f3de9731640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 357875) UID: 0 2024-04-25T19:13:26.936 INFO:teuthology.orchestra.run.smithi094.stdout:generic/471 85s 2024-04-25T19:13:27.029 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/471 2024-04-25T19:13:27.029 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:13:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:13:27.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:13:27.046 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/472) 2024-04-25T19:13:27.605 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:13:27.609 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:13:27.609 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:13:28.095 INFO:teuthology.orchestra.run.smithi094.stdout:generic/472 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:13:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/472 2024-04-25T19:13:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/472 2024-04-25T19:13:28.100 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:13:28.103 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:13:28.127 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:13:28.127 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/473) 2024-04-25T19:13:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:13:28.776 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:13:28.776 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:13:29.117 INFO:teuthology.orchestra.run.smithi094.stdout:generic/473 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-25T19:13:29.122 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/473 2024-04-25T19:13:29.122 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/473 2024-04-25T19:13:29.122 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:13:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:13:29.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:13:29.135 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/474) 2024-04-25T19:13:29.319 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:13:29.323 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:13:29.323 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:13:29.507 INFO:teuthology.orchestra.run.smithi094.stdout:generic/474 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:13:29.511 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/474 2024-04-25T19:13:29.511 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/474 2024-04-25T19:13:29.511 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:13:29.513 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:13:29.522 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:13:29.522 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/475) 2024-04-25T19:13:29.708 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:13:29.711 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:13:29.712 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:13:29.863 INFO:teuthology.orchestra.run.smithi094.stdout:generic/475 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:13:29.867 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/475 2024-04-25T19:13:29.867 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/475 2024-04-25T19:13:29.867 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:13:29.870 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:13:29.878 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:13:29.878 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/476) 2024-04-25T19:13:30.060 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:13:30.063 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:13:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:13:30.170 INFO:teuthology.orchestra.run.smithi094.stdout:generic/476 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:13:30.174 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/476 2024-04-25T19:13:30.174 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/476 2024-04-25T19:13:30.174 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:13:30.176 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:13:30.184 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:13:30.185 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/477) 2024-04-25T19:13:30.368 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:13:30.372 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:13:30.372 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:13:45.857 INFO:teuthology.orchestra.run.smithi094.stdout:generic/477 15s 2024-04-25T19:13:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/477 2024-04-25T19:13:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:13:46.213 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:13:46.289 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:13:46.289 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/478) 2024-04-25T19:13:49.340 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:13:49.346 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:13:49.346 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:00.181 INFO:teuthology.orchestra.run.smithi094.stdout:generic/478 - output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/478.out.bad) 2024-04-25T19:14:00.183 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/478.out 2024-04-25 16:52:50.099099653 +0000 2024-04-25T19:14:00.183 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/478.out.bad 2024-04-25 19:14:00.153234652 +0000 2024-04-25T19:14:00.183 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -9,8 +9,8 @@ 2024-04-25T19:14:00.183 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.184 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.184 INFO:teuthology.orchestra.run.smithi094.stdout: get wrlck 2024-04-25T19:14:00.184 INFO:teuthology.orchestra.run.smithi094.stdout: -lock could be placed 2024-04-25T19:14:00.184 INFO:teuthology.orchestra.run.smithi094.stdout: -lock could be placed 2024-04-25T19:14:00.184 INFO:teuthology.orchestra.run.smithi094.stdout: +get wrlck 2024-04-25T19:14:00.184 INFO:teuthology.orchestra.run.smithi094.stdout: +get wrlck 2024-04-25T19:14:00.184 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.184 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.184 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.184 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -25,7 +25,7 @@ 2024-04-25T19:14:00.184 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.185 INFO:teuthology.orchestra.run.smithi094.stdout: get wrlck 2024-04-25T19:14:00.185 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.185 INFO:teuthology.orchestra.run.smithi094.stdout: -lock could be placed 2024-04-25T19:14:00.185 INFO:teuthology.orchestra.run.smithi094.stdout: +get wrlck 2024-04-25T19:14:00.185 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.185 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.185 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.185 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -37,7 +37,7 @@ 2024-04-25T19:14:00.185 INFO:teuthology.orchestra.run.smithi094.stdout: get rdlck 2024-04-25T19:14:00.185 INFO:teuthology.orchestra.run.smithi094.stdout: get rdlck 2024-04-25T19:14:00.185 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.186 INFO:teuthology.orchestra.run.smithi094.stdout: -lock could be placed 2024-04-25T19:14:00.186 INFO:teuthology.orchestra.run.smithi094.stdout: +get rdlck 2024-04-25T19:14:00.186 INFO:teuthology.orchestra.run.smithi094.stdout: get rdlck 2024-04-25T19:14:00.186 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.186 INFO:teuthology.orchestra.run.smithi094.stdout: get rdlck 2024-04-25T19:14:00.186 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -45,8 +45,8 @@ 2024-04-25T19:14:00.186 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.186 INFO:teuthology.orchestra.run.smithi094.stdout: get rdlck 2024-04-25T19:14:00.186 INFO:teuthology.orchestra.run.smithi094.stdout: get rdlck 2024-04-25T19:14:00.186 INFO:teuthology.orchestra.run.smithi094.stdout: -lock could be placed 2024-04-25T19:14:00.186 INFO:teuthology.orchestra.run.smithi094.stdout: -lock could be placed 2024-04-25T19:14:00.186 INFO:teuthology.orchestra.run.smithi094.stdout: +get rdlck 2024-04-25T19:14:00.187 INFO:teuthology.orchestra.run.smithi094.stdout: +get rdlck 2024-04-25T19:14:00.187 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.187 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.187 INFO:teuthology.orchestra.run.smithi094.stdout: lock could be placed 2024-04-25T19:14:00.187 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/478 2024-04-25T19:14:00.187 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/478 2024-04-25T19:14:00.187 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-25T19:14:00.189 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:00.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:14:00.200 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-25T19:14:00.200 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-25T19:14:00.200 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/479) 2024-04-25T19:14:00.398 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:00.592 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:00.592 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:00.662 INFO:teuthology.orchestra.run.smithi094.stdout:generic/479 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:00.664 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/479 2024-04-25T19:14:00.664 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/479 2024-04-25T19:14:00.664 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:00.668 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:00.679 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:00.679 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/480) 2024-04-25T19:14:00.888 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:00.892 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:00.892 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:01.025 INFO:teuthology.orchestra.run.smithi094.stdout:generic/480 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:01.028 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/480 2024-04-25T19:14:01.028 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/480 2024-04-25T19:14:01.028 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:01.031 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:01.039 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:01.039 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/481) 2024-04-25T19:14:01.230 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:01.234 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:01.234 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:01.365 INFO:teuthology.orchestra.run.smithi094.stdout:generic/481 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:01.369 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/481 2024-04-25T19:14:01.369 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/481 2024-04-25T19:14:01.369 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:01.372 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:01.381 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:01.381 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/482) 2024-04-25T19:14:01.566 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:01.570 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:01.571 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:01.756 INFO:teuthology.orchestra.run.smithi094.stdout:generic/482 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:01.761 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/482 2024-04-25T19:14:01.761 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/482 2024-04-25T19:14:01.761 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:01.764 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:01.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:01.774 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/483) 2024-04-25T19:14:01.958 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:01.962 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:01.962 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:02.092 INFO:teuthology.orchestra.run.smithi094.stdout:generic/483 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:02.096 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/483 2024-04-25T19:14:02.096 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/483 2024-04-25T19:14:02.096 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:02.098 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:02.106 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:02.107 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/484) 2024-04-25T19:14:02.309 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:02.313 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:02.314 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:02.444 INFO:teuthology.orchestra.run.smithi094.stdout:generic/484 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:02.448 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/484 2024-04-25T19:14:02.448 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/484 2024-04-25T19:14:02.448 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:02.450 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:02.459 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:02.459 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/485) 2024-04-25T19:14:02.642 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:02.646 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:02.647 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:03.635 INFO:teuthology.orchestra.run.smithi094.stdout:generic/485 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-25T19:14:03.640 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/485 2024-04-25T19:14:03.640 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/485 2024-04-25T19:14:03.640 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:03.643 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:03.654 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:03.654 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/486) 2024-04-25T19:14:04.197 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:04.202 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:04.202 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:04.363 INFO:teuthology.orchestra.run.smithi094.stdout:generic/486 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:04.366 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/486 2024-04-25T19:14:04.367 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/486 2024-04-25T19:14:04.367 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:04.369 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:04.377 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:04.377 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/487) 2024-04-25T19:14:04.560 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:04.564 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:04.565 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:04.712 INFO:teuthology.orchestra.run.smithi094.stdout:generic/487 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:04.716 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/487 2024-04-25T19:14:04.716 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/487 2024-04-25T19:14:04.716 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:04.718 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:04.727 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:04.727 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/488) 2024-04-25T19:14:04.916 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:04.920 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:04.920 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:05.026 INFO:teuthology.orchestra.run.smithi094.stdout:generic/488 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:05.029 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/488 2024-04-25T19:14:05.029 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/488 2024-04-25T19:14:05.030 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:05.032 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:05.040 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:05.041 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/489) 2024-04-25T19:14:05.219 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:05.223 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:05.352 INFO:teuthology.orchestra.run.smithi094.stdout:generic/489 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:05.355 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/489 2024-04-25T19:14:05.356 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/489 2024-04-25T19:14:05.356 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:05.358 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:05.366 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:05.366 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/490) 2024-04-25T19:14:05.546 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:05.550 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:05.550 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:06.776 INFO:teuthology.orchestra.run.smithi094.stdout:generic/490 1s 2024-04-25T19:14:06.781 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/490 2024-04-25T19:14:06.781 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:06.792 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:06.793 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/491) 2024-04-25T19:14:07.346 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:07.350 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:07.350 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:07.485 INFO:teuthology.orchestra.run.smithi094.stdout:generic/491 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:07.488 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/491 2024-04-25T19:14:07.488 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/491 2024-04-25T19:14:07.488 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:07.499 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:07.500 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/492) 2024-04-25T19:14:07.678 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:07.683 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:07.683 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:07.811 INFO:teuthology.orchestra.run.smithi094.stdout:generic/492 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:07.815 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/492 2024-04-25T19:14:07.815 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/492 2024-04-25T19:14:07.815 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:07.817 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:07.825 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:07.826 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/493) 2024-04-25T19:14:08.011 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:08.015 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:08.130 INFO:teuthology.orchestra.run.smithi094.stdout:generic/493 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:08.134 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/493 2024-04-25T19:14:08.134 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/493 2024-04-25T19:14:08.134 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:08.144 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:08.144 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/494) 2024-04-25T19:14:08.322 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:08.326 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:08.326 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:08.428 INFO:teuthology.orchestra.run.smithi094.stdout:generic/494 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:08.432 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/494 2024-04-25T19:14:08.432 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/494 2024-04-25T19:14:08.432 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:08.434 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:08.442 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:08.442 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/495) 2024-04-25T19:14:08.622 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:08.625 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:08.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:08.727 INFO:teuthology.orchestra.run.smithi094.stdout:generic/495 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:08.731 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/495 2024-04-25T19:14:08.731 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/495 2024-04-25T19:14:08.731 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:08.734 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:08.742 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:08.742 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/496) 2024-04-25T19:14:08.923 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:08.927 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:08.927 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:09.037 INFO:teuthology.orchestra.run.smithi094.stdout:generic/496 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:09.040 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/496 2024-04-25T19:14:09.041 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/496 2024-04-25T19:14:09.041 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:09.043 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:09.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:09.051 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/497) 2024-04-25T19:14:09.233 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:09.237 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:09.237 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:09.341 INFO:teuthology.orchestra.run.smithi094.stdout:generic/497 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:09.344 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/497 2024-04-25T19:14:09.344 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/497 2024-04-25T19:14:09.344 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:09.346 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:09.354 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:09.354 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/498) 2024-04-25T19:14:09.534 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:09.538 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:09.538 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:09.653 INFO:teuthology.orchestra.run.smithi094.stdout:generic/498 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:09.657 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/498 2024-04-25T19:14:09.657 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/498 2024-04-25T19:14:09.657 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:09.659 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:09.667 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:09.668 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/499) 2024-04-25T19:14:09.850 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:09.854 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:09.854 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:09.956 INFO:teuthology.orchestra.run.smithi094.stdout:generic/499 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:09.960 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/499 2024-04-25T19:14:09.960 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/499 2024-04-25T19:14:09.960 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:09.962 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:09.971 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:09.971 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/500) 2024-04-25T19:14:10.149 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:10.153 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:10.153 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:10.316 INFO:teuthology.orchestra.run.smithi094.stdout:generic/500 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:10.319 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/500 2024-04-25T19:14:10.319 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/500 2024-04-25T19:14:10.319 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:10.322 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:10.329 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:10.330 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/501) 2024-04-25T19:14:10.510 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:10.514 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:10.514 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:10.641 INFO:teuthology.orchestra.run.smithi094.stdout:generic/501 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:10.644 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/501 2024-04-25T19:14:10.644 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/501 2024-04-25T19:14:10.644 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:10.647 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:10.655 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:10.655 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/502) 2024-04-25T19:14:10.834 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:10.838 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:10.838 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:10.952 INFO:teuthology.orchestra.run.smithi094.stdout:generic/502 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:10.956 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/502 2024-04-25T19:14:10.956 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/502 2024-04-25T19:14:10.956 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:10.958 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:10.967 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:10.967 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/503) 2024-04-25T19:14:11.147 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:11.151 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:11.258 INFO:teuthology.orchestra.run.smithi094.stdout:generic/503 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:11.262 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/503 2024-04-25T19:14:11.262 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/503 2024-04-25T19:14:11.262 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:11.264 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:11.272 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:11.272 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/504) 2024-04-25T19:14:11.450 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:11.454 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:11.455 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:11.657 INFO:teuthology.orchestra.run.smithi094.stdout:generic/504 - output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/504.out.bad) 2024-04-25T19:14:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/504.out 2024-04-25 16:52:50.103099598 +0000 2024-04-25T19:14:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/504.out.bad 2024-04-25 19:14:11.641048332 +0000 2024-04-25T19:14:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -1,2 +1,3 @@ 2024-04-25T19:14:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: QA output created by 504 2024-04-25T19:14:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: +lock info not found 2024-04-25T19:14:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: Silence is golden 2024-04-25T19:14:11.661 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/504 2024-04-25T19:14:11.661 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/504 2024-04-25T19:14:11.661 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-25T19:14:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:11.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:14:11.672 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-25T19:14:11.672 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-25T19:14:11.672 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/505) 2024-04-25T19:14:11.851 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:11.855 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:11.856 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:11.958 INFO:teuthology.orchestra.run.smithi094.stdout:generic/505 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:11.962 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/505 2024-04-25T19:14:11.962 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/505 2024-04-25T19:14:11.962 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:11.964 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:11.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:11.973 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/506) 2024-04-25T19:14:12.162 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:12.166 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:12.167 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:12.279 INFO:teuthology.orchestra.run.smithi094.stdout:generic/506 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:12.283 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/506 2024-04-25T19:14:12.283 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/506 2024-04-25T19:14:12.283 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:12.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:12.294 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/507) 2024-04-25T19:14:12.474 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:12.478 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:12.478 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:12.660 INFO:teuthology.orchestra.run.smithi094.stdout:generic/507 [not run] file system doesn't support chattr +ASai 2024-04-25T19:14:12.664 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/507 2024-04-25T19:14:12.664 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/507 2024-04-25T19:14:12.664 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:12.674 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:12.675 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/508) 2024-04-25T19:14:12.856 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:12.860 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:12.860 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:12.967 INFO:teuthology.orchestra.run.smithi094.stdout:generic/508 [not run] lsattr not supported by test filesystem type: ceph-fuse 2024-04-25T19:14:12.971 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/508 2024-04-25T19:14:12.971 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/508 2024-04-25T19:14:12.971 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:12.973 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:12.981 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:12.981 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/509) 2024-04-25T19:14:13.161 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:13.165 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:13.165 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:13.285 INFO:teuthology.orchestra.run.smithi094.stdout:generic/509 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:13.288 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/509 2024-04-25T19:14:13.288 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/509 2024-04-25T19:14:13.288 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:13.299 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:13.299 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/510) 2024-04-25T19:14:13.479 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:13.484 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:13.484 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:13.656 INFO:teuthology.orchestra.run.smithi094.stdout:generic/510 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:13.660 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/510 2024-04-25T19:14:13.660 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/510 2024-04-25T19:14:13.660 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:13.662 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:13.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:13.670 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/511) 2024-04-25T19:14:13.859 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:13.863 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:13.864 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:13.974 INFO:teuthology.orchestra.run.smithi094.stdout:generic/511 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:13.977 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/511 2024-04-25T19:14:13.978 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/511 2024-04-25T19:14:13.978 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:13.980 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:13.989 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:13.990 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/512) 2024-04-25T19:14:14.176 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:14.180 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:14.180 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:14.316 INFO:teuthology.orchestra.run.smithi094.stdout:generic/512 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:14.320 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/512 2024-04-25T19:14:14.321 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/512 2024-04-25T19:14:14.321 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:14.323 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:14.331 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:14.332 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/513) 2024-04-25T19:14:14.511 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:14.515 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:14.515 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:14.621 INFO:teuthology.orchestra.run.smithi094.stdout:generic/513 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:14.625 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/513 2024-04-25T19:14:14.625 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/513 2024-04-25T19:14:14.625 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:14.627 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:14.635 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:14.635 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/514) 2024-04-25T19:14:14.814 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:14.818 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:14.818 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:14.923 INFO:teuthology.orchestra.run.smithi094.stdout:generic/514 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:14.927 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/514 2024-04-25T19:14:14.927 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/514 2024-04-25T19:14:14.927 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:14.929 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:14.937 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:14.938 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/515) 2024-04-25T19:14:15.121 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:15.125 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:15.125 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:15.228 INFO:teuthology.orchestra.run.smithi094.stdout:generic/515 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:15.232 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/515 2024-04-25T19:14:15.232 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/515 2024-04-25T19:14:15.232 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:15.234 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:15.243 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:15.243 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/516) 2024-04-25T19:14:15.423 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:15.427 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:15.427 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:15.584 INFO:teuthology.orchestra.run.smithi094.stdout:generic/516 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-25T19:14:15.587 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/516 2024-04-25T19:14:15.587 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/516 2024-04-25T19:14:15.587 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:15.590 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:15.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:15.598 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/517) 2024-04-25T19:14:16.093 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:16.097 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:16.097 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:16.214 INFO:teuthology.orchestra.run.smithi094.stdout:generic/517 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:16.218 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/517 2024-04-25T19:14:16.218 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/517 2024-04-25T19:14:16.218 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:16.228 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:16.228 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/518) 2024-04-25T19:14:16.405 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:16.409 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:16.409 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:16.522 INFO:teuthology.orchestra.run.smithi094.stdout:generic/518 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:16.525 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/518 2024-04-25T19:14:16.526 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/518 2024-04-25T19:14:16.526 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:16.528 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:16.536 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:16.536 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/519) 2024-04-25T19:14:16.713 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:16.717 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:16.978 INFO:teuthology.orchestra.run.smithi094.stdout:generic/519 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:16.983 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/519 2024-04-25T19:14:16.983 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/519 2024-04-25T19:14:16.983 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:16.986 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:16.996 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:16.997 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/520) 2024-04-25T19:14:17.194 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:17.198 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:17.324 INFO:teuthology.orchestra.run.smithi094.stdout:generic/520 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T19:14:17.328 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/520 2024-04-25T19:14:17.328 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/520 2024-04-25T19:14:17.328 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T19:14:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:14:17.339 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T19:14:17.339 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/521) 2024-04-25T19:14:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T19:14:17.525 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T19:14:17.525 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T19:16:40.532 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:16:40.536 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:16:40.559 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T19:16:40.558+0000 7f2c3013d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35259) UID: 0 2024-04-25T19:16:40.559 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T19:16:40.558+0000 7f55ebc07640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35259) UID: 0 2024-04-25T19:16:40.559 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T19:16:40.558+0000 7f7fd7c37640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35259) UID: 0 2024-04-25T19:16:40.559 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T19:16:40.558+0000 7f0f14e8e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35259) UID: 0 2024-04-25T19:19:27.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:19:27.755 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:19:57.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:19:57.841 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:20:27.951 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:20:27.954 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:20:58.075 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:20:58.079 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:21:28.216 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:21:28.219 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:21:58.314 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:21:58.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:22:28.458 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:22:28.462 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:22:58.512 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:22:58.515 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:23:28.627 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:23:28.631 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:23:58.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:23:58.755 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:24:28.850 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:24:28.854 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:24:58.915 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:24:58.919 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:25:29.041 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:25:29.045 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:25:59.164 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:25:59.167 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:26:29.212 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:26:29.216 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:26:59.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:26:59.299 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:27:29.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:27:29.374 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:27:59.447 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:27:59.451 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:28:29.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:28:29.499 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:28:59.677 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:28:59.681 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:29:29.732 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:29:29.736 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:29:59.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:29:59.833 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:30:29.878 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:30:29.883 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:30:59.983 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:30:59.986 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:31:30.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:31:30.102 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:32:00.161 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:32:00.164 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:32:30.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:32:30.302 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:33:00.346 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:33:00.349 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:33:30.421 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:33:30.424 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:34:00.536 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:34:00.539 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:34:30.582 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:34:30.585 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:35:00.688 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:35:00.691 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:35:30.811 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:35:30.814 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:36:00.866 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:36:00.870 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:36:30.914 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:36:30.917 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:37:01.038 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:37:01.041 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:37:31.170 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:37:31.174 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:38:01.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:38:01.247 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:38:31.626 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:38:31.630 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:39:01.718 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:39:01.722 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:39:31.764 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:39:31.768 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:40:01.855 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:40:01.859 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:40:31.902 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:40:31.905 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:41:01.966 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:41:01.969 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:41:32.005 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:41:32.009 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:42:02.051 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:42:02.054 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:42:32.126 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:42:32.130 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:43:02.232 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:43:02.236 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:43:32.280 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:43:32.283 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:44:02.480 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:44:02.484 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:44:32.725 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:44:32.728 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:45:02.786 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:45:02.790 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:45:32.894 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:45:32.897 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:46:03.085 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:46:03.089 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:46:33.199 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:46:33.203 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:47:03.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:47:03.301 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:47:33.368 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:47:33.372 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:48:03.416 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:48:03.420 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:48:33.690 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:48:33.694 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:49:03.769 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:49:03.772 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:49:33.818 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:49:33.822 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:49:33.891 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T19:49:33.887+0000 7f837496c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 380133) UID: 0 2024-04-25T19:49:33.892 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T19:49:33.887+0000 7f9a177b0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 380133) UID: 0 2024-04-25T19:49:33.892 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T19:49:33.887+0000 7efca5c83640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 380133) UID: 0 2024-04-25T19:49:33.892 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T19:49:33.887+0000 7f3801006640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 380133) UID: 0 2024-04-25T19:52:27.126 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:52:27.129 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:52:57.267 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:52:57.271 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:53:27.314 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:53:27.318 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:53:57.406 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:53:57.411 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:54:27.470 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:54:27.473 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:54:57.559 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:54:57.563 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:55:27.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:55:27.653 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:55:57.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:55:57.709 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:56:27.796 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:56:27.799 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:56:57.920 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:56:57.924 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:57:28.052 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:57:28.056 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:57:58.152 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:57:58.156 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:58:28.250 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:58:28.254 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:58:58.349 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:58:58.353 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:59:28.391 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:59:28.394 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:59:58.498 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:59:58.501 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:00:28.695 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:00:28.699 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:00:58.770 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:00:58.774 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:01:28.841 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:01:28.845 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:01:58.942 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:01:58.945 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:02:29.003 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:02:29.007 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:02:59.077 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:02:59.082 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:03:29.164 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:03:29.168 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:03:59.212 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:03:59.216 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:04:29.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:04:29.332 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:04:59.385 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:04:59.389 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:05:29.482 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:05:29.486 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:05:59.567 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:05:59.570 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:06:29.628 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:06:29.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:06:59.672 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:06:59.676 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:07:29.800 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:07:29.803 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:07:59.890 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:07:59.894 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:08:29.971 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:08:29.974 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:09:00.085 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:09:00.089 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:09:30.325 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:09:30.328 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:10:00.441 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:10:00.444 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:10:30.506 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:10:30.509 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:11:00.621 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:11:00.624 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:11:30.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:11:30.727 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:12:00.865 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:12:00.868 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:12:00.907 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T20:12:00.904+0000 7f2c3013d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35578) UID: 0 2024-04-25T20:12:00.907 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T20:12:00.904+0000 7f7fd7c37640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35578) UID: 0 2024-04-25T20:12:00.908 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T20:12:00.904+0000 7f0f14e8e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35578) UID: 0 2024-04-25T20:12:00.908 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T20:12:00.904+0000 7f55ebc07640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35578) UID: 0 2024-04-25T20:14:48.697 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:14:48.701 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:15:18.752 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:15:18.755 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:15:48.816 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:15:48.821 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:16:18.886 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:16:18.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:16:48.940 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:16:48.944 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:17:18.984 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:17:18.987 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:17:49.031 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:17:49.035 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:18:19.122 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:18:19.125 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:18:49.220 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:18:49.224 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:19:19.434 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:19:19.437 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:19:49.738 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:19:49.741 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:20:19.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:20:19.858 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:20:50.003 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:20:50.006 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:21:20.059 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:21:20.062 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:21:50.144 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:21:50.147 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:22:20.225 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:22:20.228 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:22:50.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:22:50.346 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:23:20.390 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:23:20.393 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:23:50.530 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:23:50.533 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:24:20.594 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:24:20.598 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:24:50.673 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:24:50.676 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:25:20.797 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:25:20.801 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:25:50.865 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:25:50.868 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:26:20.911 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:26:20.915 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:26:50.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:26:50.996 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:27:21.039 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:27:21.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:27:51.087 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:27:51.091 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:28:21.162 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:28:21.166 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:28:51.376 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:28:51.382 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:29:21.556 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:29:21.560 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:29:51.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:29:51.833 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:30:21.877 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:30:21.880 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:30:51.943 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:30:51.946 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:31:21.981 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:31:21.984 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:31:52.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:31:52.036 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:32:22.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:32:22.144 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:32:52.242 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:32:52.245 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:33:22.288 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:33:22.292 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:33:52.389 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:33:52.392 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:34:22.489 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:34:22.493 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:34:52.721 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:34:52.724 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:35:22.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:35:22.808 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:35:52.863 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:35:52.866 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:36:22.963 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:36:22.966 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:36:53.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:36:53.082 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:37:23.213 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:37:23.216 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:37:53.261 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:37:53.264 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:38:23.308 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:38:23.312 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:38:53.399 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:38:53.402 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:39:23.446 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:39:23.449 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:39:53.511 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:39:53.514 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:40:23.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:40:23.721 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:40:53.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:40:53.791 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:41:23.983 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:41:23.986 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:41:54.056 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:41:54.060 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:42:24.306 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:42:24.309 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:42:54.423 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:42:54.427 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:43:24.665 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:43:24.668 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:43:54.722 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:43:54.726 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:44:24.966 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:44:24.969 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:44:55.189 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:44:55.193 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:45:25.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:45:25.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:45:55.362 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:45:55.368 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:46:25.428 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:46:25.431 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:46:55.608 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:46:55.612 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:47:25.730 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:47:25.733 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:47:55.819 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:47:55.822 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:48:25.892 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:48:25.895 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:48:55.974 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:48:55.977 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:49:26.304 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:49:26.308 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:49:56.369 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:49:56.373 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:50:26.417 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:50:26.420 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:50:56.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:50:56.493 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:51:26.731 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:51:26.735 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:51:56.780 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:51:56.783 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:52:26.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:52:26.929 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:52:56.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:52:56.976 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:53:27.096 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:53:27.099 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:53:57.211 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:53:57.214 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:54:27.277 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:54:27.280 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:54:57.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:54:57.359 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:55:27.426 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:55:27.429 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:55:57.493 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:55:57.496 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:56:27.585 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:56:27.588 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:56:57.676 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:56:57.679 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:57:28.026 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:57:28.030 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:57:58.225 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:57:58.228 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:58:28.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:58:28.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:58:58.446 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:58:58.450 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:59:28.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:59:28.664 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:59:58.891 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T20:59:58.894 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:00:28.939 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:00:28.942 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:00:59.182 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:00:59.185 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:01:29.264 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:01:29.267 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:01:59.405 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:01:59.408 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:02:29.522 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:02:29.526 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:02:59.612 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:02:59.615 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:03:29.658 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:03:29.662 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:03:59.772 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:03:59.775 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:04:29.878 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:04:29.881 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:04:59.977 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:04:59.981 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:05:30.018 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:05:30.022 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:06:00.102 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:06:00.105 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:06:30.160 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:06:30.163 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:07:00.259 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:07:00.262 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:07:00.312 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T21:07:00.311+0000 7f2c3013d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35900) UID: 0 2024-04-25T21:07:00.313 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T21:07:00.311+0000 7f55ebc07640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35900) UID: 0 2024-04-25T21:07:00.313 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T21:07:00.311+0000 7f7fd7c37640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35900) UID: 0 2024-04-25T21:07:00.314 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T21:07:00.311+0000 7f0f14e8e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 35900) UID: 0 2024-04-25T21:07:21.369 INFO:teuthology.orchestra.run.smithi094.stdout:generic/521 6784s 2024-04-25T21:07:21.432 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/521 2024-04-25T21:07:21.432 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T21:07:21.436 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T21:07:21.450 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T21:07:21.450 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/522) 2024-04-25T21:07:22.430 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T21:07:22.434 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T21:07:22.434 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T21:09:41.042 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:09:41.046 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:10:11.130 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:10:11.133 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:10:41.186 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:10:41.190 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:11:11.332 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:11:11.335 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:11:41.479 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:11:41.482 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:12:11.588 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:12:11.591 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:12:41.719 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:12:41.722 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:13:11.785 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:13:11.788 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:13:41.918 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:13:41.921 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:14:12.025 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:14:12.029 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:14:42.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:14:42.083 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:15:12.142 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:15:12.146 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:15:42.232 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:15:42.235 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:16:12.302 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:16:12.309 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:16:42.412 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:16:42.415 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:17:12.467 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:17:12.471 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:17:42.639 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:17:42.643 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:18:12.738 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:18:12.742 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:18:42.953 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:18:42.956 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:19:13.012 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:19:13.016 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:19:43.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:19:43.083 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:20:13.146 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:20:13.149 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:20:43.215 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:20:43.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:21:13.418 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:21:13.421 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:21:43.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:21:43.467 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:22:13.520 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:22:13.523 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:22:43.651 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:22:43.654 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:23:13.700 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:23:13.703 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:23:43.909 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:23:43.913 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:24:14.050 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:24:14.053 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:24:44.093 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:24:44.099 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:25:14.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:25:14.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:25:44.374 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:25:44.377 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:26:14.488 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:26:14.491 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:26:14.520 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T21:26:14.519+0000 7f837496c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 381159) UID: 0 2024-04-25T21:26:14.521 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T21:26:14.519+0000 7efca5c83640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 381159) UID: 0 2024-04-25T21:26:14.521 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T21:26:14.519+0000 7f9a177b0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 381159) UID: 0 2024-04-25T21:26:14.521 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T21:26:14.519+0000 7f3801006640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 381159) UID: 0 2024-04-25T21:29:06.167 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:29:06.171 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:29:36.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:29:36.229 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:30:06.292 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:30:06.295 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:30:36.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:30:36.652 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:31:06.722 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:31:06.726 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:31:37.075 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:31:37.082 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:32:07.125 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:32:07.128 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:32:37.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:32:37.250 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:33:07.361 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:33:07.364 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:33:37.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:33:37.507 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:34:07.560 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:34:07.563 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:34:37.706 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:34:37.710 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:35:07.761 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:35:07.764 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:35:37.894 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:35:37.898 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:36:08.060 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:36:08.063 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:36:38.140 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:36:38.143 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:37:08.209 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:37:08.212 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:37:08.231 INFO:tasks.ceph.mds.a.smithi094.stderr:2024-04-25T21:37:08.229+0000 7fb5271b5640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 381218) UID: 0 2024-04-25T21:37:08.232 INFO:tasks.ceph.mds.c.smithi094.stderr:2024-04-25T21:37:08.229+0000 7f3de9731640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 381218) UID: 0 2024-04-25T21:39:48.746 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:39:48.750 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:40:18.789 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:40:18.792 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:40:48.845 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:40:48.848 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:41:19.130 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:41:19.132 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:41:49.192 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:41:49.196 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:42:19.334 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:42:19.337 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:42:49.442 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:42:49.445 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:43:19.552 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:43:19.555 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:43:49.683 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:43:49.686 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:44:19.746 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:44:19.750 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:44:49.865 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:44:49.868 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:45:19.974 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:45:19.977 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:45:50.087 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:45:50.091 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:46:20.136 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:46:20.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:46:50.212 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:46:50.215 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:47:20.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:47:20.345 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:47:50.497 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:47:50.501 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:48:20.628 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:48:20.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:48:50.742 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:48:50.746 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:49:20.796 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:49:20.800 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:49:50.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:49:50.974 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:50:21.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:50:21.036 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:50:51.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:50:51.101 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:51:21.172 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:51:21.175 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:51:51.232 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:51:51.235 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:52:21.350 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:52:21.354 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:52:51.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:52:51.468 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:53:21.571 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:53:21.574 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:53:51.653 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:53:51.656 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:54:21.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:54:21.709 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:54:51.825 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:54:51.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:55:22.006 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:55:22.010 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:55:52.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:55:52.301 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:56:22.346 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:56:22.349 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:56:52.404 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:56:52.406 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:57:22.518 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:57:22.521 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:57:52.648 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:57:52.651 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:58:22.767 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:58:22.771 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:58:52.876 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:58:52.880 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:59:22.948 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:59:22.951 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:59:53.072 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T21:59:53.076 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:00:23.201 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:00:23.204 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:00:53.266 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:00:53.269 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:01:23.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:01:23.487 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:01:53.550 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:01:53.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:02:23.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:02:23.653 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:02:53.839 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:02:53.842 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:03:24.053 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:03:24.056 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:03:54.159 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:03:54.162 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:04:24.209 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:04:24.213 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:04:54.347 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:04:54.351 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:05:24.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:05:24.496 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:05:54.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:05:54.610 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:06:24.747 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:06:24.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:06:54.812 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:06:54.816 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:07:24.951 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:07:24.954 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:07:55.140 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:07:55.144 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:08:25.192 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:08:25.196 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:08:55.449 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:08:55.452 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:09:25.497 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:09:25.502 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:09:55.545 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:09:55.548 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:10:25.694 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:10:25.698 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:10:55.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:10:55.769 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:11:25.897 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:11:25.901 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:11:55.966 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:11:55.968 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:12:26.083 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:12:26.087 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:12:56.231 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:12:56.234 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:13:26.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:13:26.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:13:56.436 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:13:56.440 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:14:26.552 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:14:26.555 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:14:56.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:14:56.691 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:15:26.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:15:26.754 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:15:56.839 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:15:56.845 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:16:27.031 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:16:27.035 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:16:57.104 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:16:57.107 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:17:27.149 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:17:27.153 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:17:57.627 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:17:57.631 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:18:27.694 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:18:27.697 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:18:57.824 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:18:57.827 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:19:27.889 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:19:27.893 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:19:27.929 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T22:19:27.928+0000 7efca5c83640 -1 Fail to open '/proc/381510/cmdline' error = (2) No such file or directory 2024-04-25T22:19:27.930 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T22:19:27.928+0000 7efca5c83640 -1 received signal: Hangup from (PID: 381510) UID: 0 2024-04-25T22:19:27.930 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T22:19:27.928+0000 7f9a177b0640 -1 Fail to open '/proc/381510/cmdline' error = (2) No such file or directory 2024-04-25T22:19:27.930 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T22:19:27.928+0000 7f9a177b0640 -1 received signal: Hangup from (PID: 381510) UID: 0 2024-04-25T22:19:27.931 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T22:19:27.928+0000 7f837496c640 -1 Fail to open '/proc/381510/cmdline' error = (2) No such file or directory 2024-04-25T22:19:27.931 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T22:19:27.928+0000 7f837496c640 -1 received signal: Hangup from (PID: 381510) UID: 0 2024-04-25T22:19:27.931 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T22:19:27.928+0000 7f3801006640 -1 Fail to open '/proc/381510/cmdline' error = (2) No such file or directory 2024-04-25T22:19:27.931 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T22:19:27.928+0000 7f3801006640 -1 received signal: Hangup from (PID: 381510) UID: 0 2024-04-25T22:22:19.772 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:22:19.776 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:22:49.928 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:22:49.932 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:23:04.485 INFO:teuthology.orchestra.run.smithi094.stdout:generic/522 4542s 2024-04-25T22:23:04.591 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/522 2024-04-25T22:23:04.592 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:04.597 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:04.621 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:04.622 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/523) 2024-04-25T22:23:05.650 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:05.655 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:05.655 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:05.858 INFO:teuthology.orchestra.run.smithi094.stdout:generic/523 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T22:23:05.864 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/523 2024-04-25T22:23:05.864 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/523 2024-04-25T22:23:05.864 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:05.866 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:05.877 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:05.878 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/524) 2024-04-25T22:23:06.105 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:06.109 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:06.110 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:06.229 INFO:teuthology.orchestra.run.smithi094.stdout:generic/524 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T22:23:06.234 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/524 2024-04-25T22:23:06.234 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/524 2024-04-25T22:23:06.234 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:06.237 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:06.247 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:06.247 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/525) 2024-04-25T22:23:06.446 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:06.450 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:06.450 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:06.555 INFO:teuthology.orchestra.run.smithi094.stdout:generic/525 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T22:23:06.558 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/525 2024-04-25T22:23:06.558 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/525 2024-04-25T22:23:06.558 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:06.561 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:06.568 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:06.568 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/526) 2024-04-25T22:23:06.755 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:06.759 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:06.925 INFO:teuthology.orchestra.run.smithi094.stdout:generic/526 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T22:23:06.928 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/526 2024-04-25T22:23:06.929 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/526 2024-04-25T22:23:06.929 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:06.939 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:06.939 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/527) 2024-04-25T22:23:07.128 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:07.132 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:07.132 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:07.248 INFO:teuthology.orchestra.run.smithi094.stdout:generic/527 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T22:23:07.252 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/527 2024-04-25T22:23:07.252 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/527 2024-04-25T22:23:07.252 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:07.254 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:07.262 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:07.262 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/528) 2024-04-25T22:23:07.446 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:07.450 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:07.590 INFO:teuthology.orchestra.run.smithi094.stdout:generic/528 [not run] inode creation time not supported by this filesystem 2024-04-25T22:23:07.593 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/528 2024-04-25T22:23:07.594 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/528 2024-04-25T22:23:07.594 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:07.596 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:07.604 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:07.604 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/529) 2024-04-25T22:23:07.788 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:07.792 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:07.793 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:07.953 INFO:teuthology.orchestra.run.smithi094.stdout:generic/529 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-25T22:23:07.956 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/529 2024-04-25T22:23:07.956 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/529 2024-04-25T22:23:07.956 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:07.966 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:07.966 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/530) 2024-04-25T22:23:08.153 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:08.157 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:08.257 INFO:teuthology.orchestra.run.smithi094.stdout:generic/530 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T22:23:08.261 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/530 2024-04-25T22:23:08.261 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/530 2024-04-25T22:23:08.261 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:08.263 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:08.271 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:08.271 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/531) 2024-04-25T22:23:08.456 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:08.460 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:08.460 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:08.559 INFO:teuthology.orchestra.run.smithi094.stdout:generic/531 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T22:23:08.563 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/531 2024-04-25T22:23:08.563 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/531 2024-04-25T22:23:08.563 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:08.565 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:08.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:08.574 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/532) 2024-04-25T22:23:08.756 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:08.760 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:08.760 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:08.927 INFO:teuthology.orchestra.run.smithi094.stdout:generic/532 0s 2024-04-25T22:23:08.932 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/532 2024-04-25T22:23:08.932 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:08.934 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:08.943 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:08.943 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/533) 2024-04-25T22:23:09.128 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:09.132 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:09.132 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:09.607 INFO:teuthology.orchestra.run.smithi094.stdout:generic/533 0s 2024-04-25T22:23:09.612 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/533 2024-04-25T22:23:09.612 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:09.615 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:09.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:09.622 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/534) 2024-04-25T22:23:09.805 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:09.809 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:09.809 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:09.926 INFO:teuthology.orchestra.run.smithi094.stdout:generic/534 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T22:23:09.930 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/534 2024-04-25T22:23:09.930 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/534 2024-04-25T22:23:09.930 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:09.932 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:09.940 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:09.940 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/535) 2024-04-25T22:23:10.129 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:10.133 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:10.133 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:10.248 INFO:teuthology.orchestra.run.smithi094.stdout:generic/535 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T22:23:10.252 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/535 2024-04-25T22:23:10.252 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/535 2024-04-25T22:23:10.252 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:10.254 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:10.262 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:10.263 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/536) 2024-04-25T22:23:10.443 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:10.447 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:10.447 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:10.546 INFO:teuthology.orchestra.run.smithi094.stdout:generic/536 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T22:23:10.550 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/536 2024-04-25T22:23:10.550 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/536 2024-04-25T22:23:10.550 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:10.552 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:10.560 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:10.560 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/537) 2024-04-25T22:23:10.744 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:10.748 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:10.748 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:10.851 INFO:teuthology.orchestra.run.smithi094.stdout:generic/537 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T22:23:10.855 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/537 2024-04-25T22:23:10.855 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/537 2024-04-25T22:23:10.855 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T22:23:10.857 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:10.865 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T22:23:10.865 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/538) 2024-04-25T22:23:11.049 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T22:23:11.053 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T22:23:11.053 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T22:23:20.074 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:23:20.077 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:23:50.148 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:23:50.152 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:24:20.208 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:24:20.212 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:24:50.333 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:24:50.336 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:25:20.381 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:25:20.384 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:25:50.488 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:25:50.492 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:26:20.528 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:26:20.532 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:26:50.585 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:26:50.589 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:27:20.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:27:20.658 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:27:50.864 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:27:50.868 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:28:20.932 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:28:20.936 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:28:50.981 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:28:50.985 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:29:21.112 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:29:21.115 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:29:51.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:29:51.170 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:30:21.290 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:30:21.293 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:30:51.412 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:30:51.416 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:31:21.602 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:31:21.606 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:31:51.756 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:31:51.759 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:32:21.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:32:21.834 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:32:51.919 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:32:51.922 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:33:22.016 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:33:22.020 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:33:52.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:33:52.146 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:34:22.416 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:34:22.420 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:34:52.458 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:34:52.462 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:35:22.580 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:35:22.584 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:35:52.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:35:52.708 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:36:22.759 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:36:22.762 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:36:22.792 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T22:36:22.788+0000 7f9a177b0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 389311) UID: 0 2024-04-25T22:36:22.792 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T22:36:22.788+0000 7f837496c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 389311) UID: 0 2024-04-25T22:36:22.793 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T22:36:22.788+0000 7efca5c83640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 389311) UID: 0 2024-04-25T22:36:22.793 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T22:36:22.788+0000 7f3801006640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 389311) UID: 0 2024-04-25T22:39:09.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:39:09.468 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:39:39.543 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:39:39.546 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:40:09.626 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:40:09.633 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:40:39.734 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:40:39.763 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:41:09.807 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:41:09.811 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:41:39.880 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:41:39.884 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:42:09.935 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:42:09.939 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:42:39.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:42:39.997 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:43:10.040 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:43:10.044 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:43:40.114 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:43:40.117 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:44:10.211 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:44:10.215 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:44:40.284 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:44:40.287 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:44:40.306 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T22:44:40.299+0000 7f2c3013d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36461) UID: 0 2024-04-25T22:44:40.306 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T22:44:40.303+0000 7f7fd7c37640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36461) UID: 0 2024-04-25T22:44:40.307 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T22:44:40.303+0000 7f0f14e8e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36461) UID: 0 2024-04-25T22:44:40.307 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T22:44:40.303+0000 7f55ebc07640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36461) UID: 0 2024-04-25T22:47:35.294 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:47:35.298 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:48:05.585 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:48:05.588 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:48:35.628 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:48:35.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:49:05.715 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:49:05.720 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:49:35.857 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:49:35.861 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:50:05.955 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:50:05.959 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:50:36.002 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:50:36.005 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:51:06.124 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:51:06.129 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:51:36.171 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:51:36.175 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:52:06.277 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:52:06.280 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:52:36.392 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:52:36.395 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:53:06.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:53:06.645 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:53:36.758 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:53:36.762 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:54:06.874 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:54:06.878 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:54:36.939 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:54:36.942 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:55:07.055 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:55:07.059 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:55:37.110 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:55:37.113 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:56:07.242 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:56:07.246 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:56:37.361 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:56:37.364 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:57:07.472 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:57:07.475 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:57:37.592 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:57:37.596 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:58:07.681 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:58:07.685 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:58:37.790 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:58:37.793 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:59:07.907 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:59:07.911 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:59:37.997 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T22:59:38.000 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:00:08.413 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:00:08.417 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:00:38.460 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:00:38.463 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:01:08.566 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:01:08.570 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:01:38.622 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:01:38.628 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:02:08.745 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:02:08.748 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:02:38.926 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:02:38.929 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:03:09.047 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:03:09.050 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:03:39.096 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:03:39.099 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:04:09.205 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:04:09.208 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:04:39.406 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:04:39.410 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:05:09.612 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:05:09.616 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:05:09.636 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T23:05:09.633+0000 7f55ebc07640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36582) UID: 0 2024-04-25T23:05:09.636 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T23:05:09.633+0000 7f7fd7c37640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36582) UID: 0 2024-04-25T23:05:09.637 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T23:05:09.633+0000 7f2c3013d640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36582) UID: 0 2024-04-25T23:05:09.637 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T23:05:09.633+0000 7f0f14e8e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 36582) UID: 0 2024-04-25T23:07:59.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:07:59.137 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:08:29.311 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:08:29.315 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:08:59.495 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:08:59.499 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:09:29.566 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:09:29.570 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:09:59.738 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:09:59.742 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:10:29.878 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:10:29.882 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:10:59.942 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:10:59.946 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:11:30.014 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:11:30.018 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:12:00.122 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:12:00.125 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:12:30.252 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:12:30.256 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:13:00.349 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:13:00.353 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:13:30.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:13:30.401 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:14:00.442 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:14:00.446 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:14:30.576 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:14:30.579 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:15:00.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:15:00.739 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:15:30.797 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:15:30.800 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:16:00.911 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:16:00.914 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:16:30.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:16:30.962 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:17:01.063 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:17:01.066 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:17:31.210 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:17:31.214 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:18:01.266 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:18:01.269 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:18:31.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:18:31.652 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:19:01.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:19:01.709 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:19:31.804 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:19:31.807 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:20:01.869 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:20:01.873 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:20:32.077 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:20:32.080 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:21:02.368 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:21:02.372 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:21:32.477 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:21:32.480 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:22:02.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:22:02.576 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:22:32.729 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:22:32.733 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:23:02.777 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:23:02.780 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:23:32.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:23:32.966 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:24:03.009 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:24:03.012 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:24:33.156 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:24:33.160 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:25:03.231 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:25:03.234 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:25:33.460 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:25:33.464 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:26:03.575 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:26:03.578 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:26:33.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:26:33.673 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:27:03.878 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:27:03.882 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:27:34.026 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:27:34.029 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:28:04.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:28:04.137 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:28:34.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:28:34.243 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:29:04.534 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:29:04.537 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:29:34.671 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:29:34.674 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:30:04.812 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:30:04.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:30:34.943 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:30:34.947 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:31:04.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:31:04.995 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:31:35.306 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:31:35.310 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:32:05.427 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:32:05.431 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:32:35.568 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:32:35.572 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:33:05.690 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:33:05.693 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:33:35.745 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:33:35.748 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:34:06.077 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:34:06.080 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:34:36.131 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:34:36.134 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:35:06.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:35:06.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:35:36.386 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:35:36.389 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:36:06.471 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:36:06.474 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:36:36.564 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:36:36.569 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:37:06.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:37:06.691 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:37:36.768 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:37:36.773 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:38:06.984 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:38:06.987 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:38:37.383 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:38:37.386 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:39:07.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:39:07.433 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:39:37.521 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:39:37.524 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:40:07.774 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:40:07.778 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:40:38.130 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:40:38.133 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:41:08.177 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:41:08.180 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:41:38.309 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:41:38.314 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:42:08.440 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:42:08.443 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:42:38.553 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:42:38.557 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:43:08.678 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:43:08.681 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:43:38.724 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:43:38.727 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:44:08.819 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:44:08.822 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:44:38.861 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:44:38.864 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:45:09.225 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:45:09.228 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:45:39.457 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:45:39.460 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:46:09.513 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:46:09.516 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:46:39.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:46:39.614 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:47:09.741 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:47:09.745 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:47:40.014 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:47:40.017 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:48:10.069 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:48:10.073 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:48:40.125 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:48:40.129 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:49:10.248 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:49:10.251 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:49:40.365 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:49:40.369 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:50:10.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:50:10.433 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:50:40.562 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:50:40.566 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:50:40.586 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-25T23:50:40.598+0000 7f837496c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 394586) UID: 0 2024-04-25T23:50:40.586 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-25T23:50:40.598+0000 7efca5c83640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 394586) UID: 0 2024-04-25T23:50:40.587 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-25T23:50:40.598+0000 7f9a177b0640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 394586) UID: 0 2024-04-25T23:50:40.587 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-25T23:50:40.598+0000 7f3801006640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 394586) UID: 0 2024-04-25T23:53:35.585 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:53:35.588 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:54:05.746 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:54:05.750 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:54:35.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:54:35.796 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:55:05.910 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:55:05.914 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:55:35.984 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:55:35.988 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:56:06.075 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:56:06.078 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:56:36.117 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:56:36.121 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:57:06.248 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:57:06.251 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:57:36.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:57:36.340 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:58:06.420 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:58:06.423 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:58:36.662 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:58:36.665 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:59:06.785 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:59:06.799 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:59:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:generic/538 5763s 2024-04-25T23:59:14.142 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/538 2024-04-25T23:59:14.142 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:14.146 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:14.158 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:14.159 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/539) 2024-04-25T23:59:15.170 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:15.175 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:15.175 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:18.611 INFO:teuthology.orchestra.run.smithi094.stdout:generic/539 3s 2024-04-25T23:59:18.618 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/539 2024-04-25T23:59:18.619 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:18.622 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:18.635 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:18.635 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/540) 2024-04-25T23:59:19.215 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:19.219 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:19.219 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:19.365 INFO:teuthology.orchestra.run.smithi094.stdout:generic/540 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:19.369 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/540 2024-04-25T23:59:19.369 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/540 2024-04-25T23:59:19.369 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:19.371 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:19.379 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:19.379 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/541) 2024-04-25T23:59:19.565 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:19.569 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:19.569 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:19.673 INFO:teuthology.orchestra.run.smithi094.stdout:generic/541 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:19.678 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/541 2024-04-25T23:59:19.678 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/541 2024-04-25T23:59:19.678 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:19.680 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:19.688 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:19.689 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/542) 2024-04-25T23:59:19.872 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:19.876 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:19.876 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:19.981 INFO:teuthology.orchestra.run.smithi094.stdout:generic/542 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:19.985 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/542 2024-04-25T23:59:19.985 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/542 2024-04-25T23:59:19.985 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:19.987 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:19.995 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:19.995 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/543) 2024-04-25T23:59:20.179 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:20.184 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:20.184 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:20.289 INFO:teuthology.orchestra.run.smithi094.stdout:generic/543 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:20.292 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/543 2024-04-25T23:59:20.292 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/543 2024-04-25T23:59:20.293 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:20.295 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:20.303 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:20.304 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/544) 2024-04-25T23:59:20.486 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:20.491 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:20.491 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:20.599 INFO:teuthology.orchestra.run.smithi094.stdout:generic/544 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:20.603 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/544 2024-04-25T23:59:20.603 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/544 2024-04-25T23:59:20.603 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:20.605 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:20.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:20.613 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/545) 2024-04-25T23:59:20.799 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:20.804 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:20.804 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:21.005 INFO:teuthology.orchestra.run.smithi094.stdout:generic/545 [not run] file system doesn't support chattr +i 2024-04-25T23:59:21.008 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/545 2024-04-25T23:59:21.008 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/545 2024-04-25T23:59:21.008 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:21.011 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:21.019 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:21.020 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/546) 2024-04-25T23:59:21.201 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:21.205 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:21.205 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:21.398 INFO:teuthology.orchestra.run.smithi094.stdout:generic/546 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:21.402 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/546 2024-04-25T23:59:21.402 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/546 2024-04-25T23:59:21.402 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:21.404 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:21.412 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:21.412 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/547) 2024-04-25T23:59:21.596 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:21.600 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:21.600 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:21.723 INFO:teuthology.orchestra.run.smithi094.stdout:generic/547 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:21.726 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/547 2024-04-25T23:59:21.727 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/547 2024-04-25T23:59:21.727 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:21.729 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:21.738 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:21.738 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/548) 2024-04-25T23:59:21.928 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:21.932 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:21.932 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:22.101 INFO:teuthology.orchestra.run.smithi094.stdout:generic/548 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:22.105 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/548 2024-04-25T23:59:22.105 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/548 2024-04-25T23:59:22.105 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:22.107 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:22.116 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:22.116 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/549) 2024-04-25T23:59:22.301 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:22.305 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:22.434 INFO:teuthology.orchestra.run.smithi094.stdout:generic/549 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:22.438 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/549 2024-04-25T23:59:22.439 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/549 2024-04-25T23:59:22.439 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:22.441 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:22.451 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:22.451 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/550) 2024-04-25T23:59:22.636 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:22.640 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:22.640 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:22.769 INFO:teuthology.orchestra.run.smithi094.stdout:generic/550 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:22.772 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/550 2024-04-25T23:59:22.772 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/550 2024-04-25T23:59:22.773 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:22.775 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:22.783 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:22.783 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/551) 2024-04-25T23:59:22.967 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:22.971 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:22.971 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:23.075 INFO:teuthology.orchestra.run.smithi094.stdout:generic/551 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:23.078 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/551 2024-04-25T23:59:23.078 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/551 2024-04-25T23:59:23.078 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:23.081 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:23.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:23.089 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/552) 2024-04-25T23:59:23.273 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:23.278 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:23.278 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:23.401 INFO:teuthology.orchestra.run.smithi094.stdout:generic/552 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:23.405 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/552 2024-04-25T23:59:23.405 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/552 2024-04-25T23:59:23.405 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:23.407 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:23.415 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:23.415 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/553) 2024-04-25T23:59:23.600 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:23.604 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:23.604 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:23.806 INFO:teuthology.orchestra.run.smithi094.stdout:generic/553 [not run] xfs_io chattr +i failed (old kernel/wrong fs/bad args?) 2024-04-25T23:59:23.809 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/553 2024-04-25T23:59:23.809 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/553 2024-04-25T23:59:23.809 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:23.812 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:23.819 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:23.819 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/554) 2024-04-25T23:59:24.287 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:24.291 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:24.291 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:24.399 INFO:teuthology.orchestra.run.smithi094.stdout:generic/554 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:24.402 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/554 2024-04-25T23:59:24.402 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/554 2024-04-25T23:59:24.403 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:24.405 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:24.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:24.413 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/555) 2024-04-25T23:59:24.595 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:24.600 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:24.600 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:24.756 INFO:teuthology.orchestra.run.smithi094.stdout:generic/555 [not run] xfs_io chattr +ia failed (old kernel/wrong fs/bad args?) 2024-04-25T23:59:24.759 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/555 2024-04-25T23:59:24.759 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/555 2024-04-25T23:59:24.759 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:24.762 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:24.769 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:24.770 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/556) 2024-04-25T23:59:25.305 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:25.309 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:25.309 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:25.414 INFO:teuthology.orchestra.run.smithi094.stdout:generic/556 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:25.417 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/556 2024-04-25T23:59:25.417 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/556 2024-04-25T23:59:25.417 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:25.419 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:25.427 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:25.427 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/557) 2024-04-25T23:59:25.611 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:25.615 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:25.615 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:25.741 INFO:teuthology.orchestra.run.smithi094.stdout:generic/557 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:25.745 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/557 2024-04-25T23:59:25.745 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/557 2024-04-25T23:59:25.745 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:25.747 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:25.756 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:25.756 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/558) 2024-04-25T23:59:25.942 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:25.946 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:26.070 INFO:teuthology.orchestra.run.smithi094.stdout:generic/558 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:26.074 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/558 2024-04-25T23:59:26.074 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/558 2024-04-25T23:59:26.074 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:26.085 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:26.085 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/559) 2024-04-25T23:59:26.269 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:26.273 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:26.376 INFO:teuthology.orchestra.run.smithi094.stdout:generic/559 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:26.380 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/559 2024-04-25T23:59:26.380 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/559 2024-04-25T23:59:26.380 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:26.390 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:26.390 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/560) 2024-04-25T23:59:26.572 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:26.576 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:26.681 INFO:teuthology.orchestra.run.smithi094.stdout:generic/560 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:26.685 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/560 2024-04-25T23:59:26.685 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/560 2024-04-25T23:59:26.685 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:26.695 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:26.696 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/561) 2024-04-25T23:59:26.882 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:26.886 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:26.886 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:26.992 INFO:teuthology.orchestra.run.smithi094.stdout:generic/561 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:26.996 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/561 2024-04-25T23:59:26.996 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/561 2024-04-25T23:59:26.996 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:27.007 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:27.007 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/562) 2024-04-25T23:59:27.190 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:27.297 INFO:teuthology.orchestra.run.smithi094.stdout:generic/562 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:27.302 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/562 2024-04-25T23:59:27.302 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/562 2024-04-25T23:59:27.302 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:27.304 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:27.312 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:27.312 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/563) 2024-04-25T23:59:27.499 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:27.503 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:27.503 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:27.635 INFO:teuthology.orchestra.run.smithi094.stdout:generic/563 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:27.639 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/563 2024-04-25T23:59:27.639 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/563 2024-04-25T23:59:27.639 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:27.641 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:27.649 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:27.649 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/564) 2024-04-25T23:59:27.834 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:27.838 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:27.838 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:28.439 INFO:teuthology.orchestra.run.smithi094.stdout:generic/564 1s 2024-04-25T23:59:28.445 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/564 2024-04-25T23:59:28.445 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:28.448 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:28.456 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:28.456 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/565) 2024-04-25T23:59:28.990 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:28.994 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:28.994 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:29.099 INFO:teuthology.orchestra.run.smithi094.stdout:generic/565 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:29.102 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/565 2024-04-25T23:59:29.103 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/565 2024-04-25T23:59:29.103 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:29.105 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:29.113 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:29.113 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/566) 2024-04-25T23:59:29.304 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:29.308 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:29.309 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:29.412 INFO:teuthology.orchestra.run.smithi094.stdout:generic/566 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:29.416 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/566 2024-04-25T23:59:29.416 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/566 2024-04-25T23:59:29.416 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:29.426 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:29.427 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/567) 2024-04-25T23:59:29.612 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:29.616 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:29.616 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:29.718 INFO:teuthology.orchestra.run.smithi094.stdout:generic/567 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:29.722 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/567 2024-04-25T23:59:29.722 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/567 2024-04-25T23:59:29.722 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:29.725 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:29.732 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:29.733 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/568) 2024-04-25T23:59:29.918 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:29.922 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:29.922 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:30.108 INFO:teuthology.orchestra.run.smithi094.stdout:generic/568 1s 2024-04-25T23:59:30.113 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/568 2024-04-25T23:59:30.113 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:30.115 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:30.198 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:30.198 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/569) 2024-04-25T23:59:30.652 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:30.656 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:30.656 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:30.809 INFO:teuthology.orchestra.run.smithi094.stdout:generic/569 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:30.813 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/569 2024-04-25T23:59:30.813 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/569 2024-04-25T23:59:30.814 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:30.816 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:30.824 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:30.824 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/570) 2024-04-25T23:59:31.012 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:31.121 INFO:teuthology.orchestra.run.smithi094.stdout:generic/570 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:31.125 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/570 2024-04-25T23:59:31.125 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/570 2024-04-25T23:59:31.125 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:31.127 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:31.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:31.136 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/571) 2024-04-25T23:59:31.322 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:31.326 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:31.326 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:36.884 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:59:36.887 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T23:59:37.572 INFO:teuthology.orchestra.run.smithi094.stdout:generic/571 6s 2024-04-25T23:59:37.580 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/571 2024-04-25T23:59:37.580 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:37.584 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:37.910 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:37.910 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/572) 2024-04-25T23:59:38.182 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:38.184 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:38.184 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:38.290 INFO:teuthology.orchestra.run.smithi094.stdout:generic/572 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:38.293 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/572 2024-04-25T23:59:38.293 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/572 2024-04-25T23:59:38.293 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:38.296 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:38.304 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:38.304 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/573) 2024-04-25T23:59:38.489 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:38.494 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:38.494 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:38.597 INFO:teuthology.orchestra.run.smithi094.stdout:generic/573 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:38.600 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/573 2024-04-25T23:59:38.600 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/573 2024-04-25T23:59:38.600 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:38.603 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:38.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:38.611 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/574) 2024-04-25T23:59:38.793 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:38.797 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:38.797 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:38.902 INFO:teuthology.orchestra.run.smithi094.stdout:generic/574 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:38.906 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/574 2024-04-25T23:59:38.906 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/574 2024-04-25T23:59:38.906 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:38.908 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:38.916 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:38.917 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/575) 2024-04-25T23:59:39.102 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:39.106 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:39.106 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:39.210 INFO:teuthology.orchestra.run.smithi094.stdout:generic/575 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:39.213 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/575 2024-04-25T23:59:39.214 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/575 2024-04-25T23:59:39.214 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:39.215 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:39.223 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:39.224 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/576) 2024-04-25T23:59:39.410 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:39.412 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:39.412 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:39.544 INFO:teuthology.orchestra.run.smithi094.stdout:generic/576 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:39.548 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/576 2024-04-25T23:59:39.548 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/576 2024-04-25T23:59:39.548 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:39.558 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:39.559 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/577) 2024-04-25T23:59:39.741 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:39.745 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:39.745 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:39.847 INFO:teuthology.orchestra.run.smithi094.stdout:generic/577 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:39.851 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/577 2024-04-25T23:59:39.851 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/577 2024-04-25T23:59:39.851 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:39.853 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:39.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:39.861 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/578) 2024-04-25T23:59:40.047 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:40.051 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:40.204 INFO:teuthology.orchestra.run.smithi094.stdout:generic/578 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-25T23:59:40.207 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/578 2024-04-25T23:59:40.207 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/578 2024-04-25T23:59:40.207 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:40.209 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:40.217 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:40.217 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/579) 2024-04-25T23:59:40.404 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:40.409 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:40.516 INFO:teuthology.orchestra.run.smithi094.stdout:generic/579 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:40.520 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/579 2024-04-25T23:59:40.520 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/579 2024-04-25T23:59:40.520 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:40.522 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:40.530 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:40.530 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/580) 2024-04-25T23:59:40.716 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:40.720 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:40.720 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:40.850 INFO:teuthology.orchestra.run.smithi094.stdout:generic/580 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:40.854 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/580 2024-04-25T23:59:40.854 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/580 2024-04-25T23:59:40.854 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:40.865 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:40.865 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/581) 2024-04-25T23:59:41.050 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:41.052 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:41.052 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:41.986 INFO:teuthology.orchestra.run.smithi094.stdout:generic/581 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:41.993 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/581 2024-04-25T23:59:41.993 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/581 2024-04-25T23:59:41.993 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:41.996 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:42.008 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:42.009 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/582) 2024-04-25T23:59:42.248 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:42.253 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:42.253 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:42.422 INFO:teuthology.orchestra.run.smithi094.stdout:generic/582 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:42.426 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/582 2024-04-25T23:59:42.426 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/582 2024-04-25T23:59:42.426 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:42.429 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:42.439 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:42.439 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/583) 2024-04-25T23:59:42.649 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:42.654 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:42.654 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:42.801 INFO:teuthology.orchestra.run.smithi094.stdout:generic/583 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:42.805 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/583 2024-04-25T23:59:42.805 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/583 2024-04-25T23:59:42.805 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:42.807 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:42.816 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:42.816 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/584) 2024-04-25T23:59:43.006 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:43.010 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:43.010 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:43.142 INFO:teuthology.orchestra.run.smithi094.stdout:generic/584 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:43.146 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/584 2024-04-25T23:59:43.146 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/584 2024-04-25T23:59:43.146 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:43.148 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:43.157 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:43.157 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/585) 2024-04-25T23:59:43.342 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:43.346 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:43.346 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:43.456 INFO:teuthology.orchestra.run.smithi094.stdout:generic/585 [not run] this test requires a valid $SCRATCH_DEV 2024-04-25T23:59:43.460 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/585 2024-04-25T23:59:43.460 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/585 2024-04-25T23:59:43.460 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-25T23:59:43.462 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T23:59:43.470 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-25T23:59:43.470 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/586) 2024-04-25T23:59:43.654 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-25T23:59:43.658 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T23:59:43.659 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:06.991 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T00:00:06.994 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T00:00:16.575 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-26T00:00:16.593+0000 7f837496c640 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror cephfs-mirror (PID: 415888) UID: 0 2024-04-26T00:00:16.575 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-26T00:00:16.593+0000 7f9a177b0640 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror cephfs-mirror (PID: 415888) UID: 0 2024-04-26T00:00:16.576 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-26T00:00:16.593+0000 7f3801006640 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror cephfs-mirror (PID: 415888) UID: 0 2024-04-26T00:00:16.576 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-26T00:00:16.593+0000 7efca5c83640 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror cephfs-mirror (PID: 415888) UID: 0 2024-04-26T00:00:16.576 INFO:tasks.ceph.mds.a.smithi094.stderr:2024-04-26T00:00:16.593+0000 7fb5271b5640 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror cephfs-mirror (PID: 415888) UID: 0 2024-04-26T00:00:16.577 INFO:tasks.ceph.mds.c.smithi094.stderr:2024-04-26T00:00:16.593+0000 7f3de9731640 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror cephfs-mirror (PID: 415888) UID: 0 2024-04-26T00:00:16.619 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-26T00:00:16.617+0000 7ff5e25c8640 -1 Fail to open '/proc/415888/cmdline' error = (2) No such file or directory 2024-04-26T00:00:16.620 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-26T00:00:16.617+0000 7ff5e25c8640 -1 received signal: Hangup from (PID: 415888) UID: 0 2024-04-26T00:00:16.629 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-04-26T00:00:16.621+0000 7fb03807a640 -1 Fail to open '/proc/415888/cmdline' error = (2) No such file or directory 2024-04-26T00:00:16.629 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-04-26T00:00:16.621+0000 7fb03807a640 -1 received signal: Hangup from (PID: 415888) UID: 0 2024-04-26T00:00:16.629 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-04-26T00:00:16.625+0000 7fb03807a640 -1 mon.a@0(leader) e1 *** Got Signal Hangup *** 2024-04-26T00:00:16.654 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-04-26T00:00:16.673+0000 7fb03807a640 -1 received signal: Hangup from (PID: 415898) UID: 0 2024-04-26T00:00:16.654 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-04-26T00:00:16.673+0000 7fb03807a640 -1 mon.a@0(leader) e1 *** Got Signal Hangup *** 2024-04-26T00:00:16.654 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-04-26T00:00:16.673+0000 7f9a177b0640 -1 received signal: Hangup from (PID: 415898) UID: 0 2024-04-26T00:00:16.654 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-04-26T00:00:16.673+0000 7f837496c640 -1 received signal: Hangup from (PID: 415898) UID: 0 2024-04-26T00:00:16.655 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-04-26T00:00:16.673+0000 7f3801006640 -1 received signal: Hangup from (PID: 415898) UID: 0 2024-04-26T00:00:16.655 INFO:tasks.ceph.mds.c.smithi094.stderr:2024-04-26T00:00:16.673+0000 7f3de9731640 -1 received signal: Hangup from (PID: 415898) UID: 0 2024-04-26T00:00:16.655 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-04-26T00:00:16.673+0000 7ff5e25c8640 -1 received signal: Hangup from (PID: 415898) UID: 0 2024-04-26T00:00:16.656 INFO:tasks.ceph.mds.a.smithi094.stderr:2024-04-26T00:00:16.673+0000 7fb5271b5640 -1 received signal: Hangup from (PID: 415898) UID: 0 2024-04-26T00:00:16.656 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-04-26T00:00:16.673+0000 7efca5c83640 -1 received signal: Hangup from (PID: 415898) UID: 0 2024-04-26T00:00:19.552 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-26T00:00:19.568+0000 7f2c3013d640 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror cephfs-mirror (PID: 36945) UID: 0 2024-04-26T00:00:19.553 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-26T00:00:19.568+0000 7f7fd7c37640 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror cephfs-mirror (PID: 36945) UID: 0 2024-04-26T00:00:19.553 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-26T00:00:19.568+0000 7f55ebc07640 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror cephfs-mirror (PID: 36945) UID: 0 2024-04-26T00:00:19.554 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-26T00:00:19.568+0000 7f0f14e8e640 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror cephfs-mirror (PID: 36945) UID: 0 2024-04-26T00:00:19.580 INFO:tasks.ceph.mon.b.smithi138.stderr:2024-04-26T00:00:19.596+0000 7faf395b8640 -1 Fail to open '/proc/36945/cmdline' error = (2) No such file or directory 2024-04-26T00:00:19.580 INFO:tasks.ceph.mon.b.smithi138.stderr:2024-04-26T00:00:19.596+0000 7faf395b8640 -1 received signal: Hangup from (PID: 36945) UID: 0 2024-04-26T00:00:19.593 INFO:tasks.ceph.mon.c.smithi138.stderr:2024-04-26T00:00:19.608+0000 7f56651ee640 -1 Fail to open '/proc/36945/cmdline' error = (2) No such file or directory 2024-04-26T00:00:19.593 INFO:tasks.ceph.mon.c.smithi138.stderr:2024-04-26T00:00:19.608+0000 7f56651ee640 -1 received signal: Hangup from (PID: 36945) UID: 0 2024-04-26T00:00:19.600 INFO:tasks.ceph.mon.b.smithi138.stderr:2024-04-26T00:00:19.616+0000 7faf395b8640 -1 mon.b@1(peon) e1 *** Got Signal Hangup *** 2024-04-26T00:00:19.601 INFO:tasks.ceph.mon.c.smithi138.stderr:2024-04-26T00:00:19.616+0000 7f56651ee640 -1 mon.c@2(peon) e1 *** Got Signal Hangup *** 2024-04-26T00:00:19.635 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-26T00:00:19.644+0000 7f4e374a1640 -1 Fail to open '/proc/36945/cmdline' error = (2) No such file or directory 2024-04-26T00:00:19.635 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-26T00:00:19.652+0000 7f4e374a1640 -1 received signal: Hangup from (PID: 36945) UID: 0 2024-04-26T00:00:19.645 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-04-26T00:00:19.660+0000 7f8a12394640 -1 Fail to open '/proc/36945/cmdline' error = (2) No such file or directory 2024-04-26T00:00:19.645 INFO:tasks.ceph.mds.d.smithi138.stderr:2024-04-26T00:00:19.660+0000 7f40ae102640 -1 Fail to open '/proc/36945/cmdline' error = (2) No such file or directory 2024-04-26T00:00:19.669 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-04-26T00:00:19.684+0000 7f8a12394640 -1 received signal: Hangup from (PID: 36945) UID: 0 2024-04-26T00:00:19.669 INFO:tasks.ceph.mds.d.smithi138.stderr:2024-04-26T00:00:19.684+0000 7f40ae102640 -1 received signal: Hangup from (PID: 36945) UID: 0 2024-04-26T00:00:19.698 INFO:tasks.ceph.mon.b.smithi138.stderr:2024-04-26T00:00:19.712+0000 7faf395b8640 -1 received signal: Hangup from pkill -1 -x ceph-mon|ceph-mgr|ceph-mds|ceph-osd|ceph-fuse|radosgw|rbd-mirror|cephfs-mirror (PID: 36959) UID: 0 2024-04-26T00:00:19.698 INFO:tasks.ceph.mon.b.smithi138.stderr:2024-04-26T00:00:19.712+0000 7faf395b8640 -1 mon.b@1(peon) e1 *** Got Signal Hangup *** 2024-04-26T00:00:19.698 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-26T00:00:19.712+0000 7f4e374a1640 -1 received signal: Hangup from pkill -1 -x ceph-mon|ceph-mgr|ceph-mds|ceph-osd|ceph-fuse|radosgw|rbd-mirror|cephfs-mirror (PID: 36959) UID: 0 2024-04-26T00:00:19.699 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-26T00:00:19.712+0000 7f7fd7c37640 -1 received signal: Hangup from (PID: 36959) UID: 0 2024-04-26T00:00:19.699 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-26T00:00:19.712+0000 7f0f14e8e640 -1 received signal: Hangup from pkill -1 -x ceph-mon|ceph-mgr|ceph-mds|ceph-osd|ceph-fuse|radosgw|rbd-mirror|cephfs-mirror (PID: 36959) UID: 0 2024-04-26T00:00:19.699 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-26T00:00:19.712+0000 7f2c3013d640 -1 received signal: Hangup from pkill -1 -x ceph-mon|ceph-mgr|ceph-mds|ceph-osd|ceph-fuse|radosgw|rbd-mirror|cephfs-mirror (PID: 36959) UID: 0 2024-04-26T00:00:19.700 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-26T00:00:19.712+0000 7f55ebc07640 -1 received signal: Hangup from pkill -1 -x ceph-mon|ceph-mgr|ceph-mds|ceph-osd|ceph-fuse|radosgw|rbd-mirror|cephfs-mirror (PID: 36959) UID: 0 2024-04-26T00:00:19.700 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-04-26T00:00:19.712+0000 7f8a12394640 -1 received signal: Hangup from (PID: 36959) UID: 0 2024-04-26T00:00:19.700 INFO:tasks.ceph.mds.d.smithi138.stderr:2024-04-26T00:00:19.712+0000 7f40ae102640 -1 received signal: Hangup from pkill -1 -x ceph-mon|ceph-mgr|ceph-mds|ceph-osd|ceph-fuse|radosgw|rbd-mirror|cephfs-mirror (PID: 36959) UID: 0 2024-04-26T00:00:19.700 INFO:tasks.ceph.mon.c.smithi138.stderr:2024-04-26T00:00:19.712+0000 7f56651ee640 -1 received signal: Hangup from (PID: 36959) UID: 0 2024-04-26T00:00:19.700 INFO:tasks.ceph.mon.c.smithi138.stderr:2024-04-26T00:00:19.712+0000 7f56651ee640 -1 mon.c@2(peon) e1 *** Got Signal Hangup *** 2024-04-26T00:00:30.009 INFO:teuthology.orchestra.run.smithi094.stdout:generic/586 47s 2024-04-26T00:00:30.015 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/586 2024-04-26T00:00:30.015 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:30.379 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:30.380 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/587) 2024-04-26T00:00:30.702 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:30.706 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:30.706 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:30.913 INFO:teuthology.orchestra.run.smithi094.stdout:generic/587 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-26T00:00:30.917 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/587 2024-04-26T00:00:30.917 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/587 2024-04-26T00:00:30.917 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:30.920 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:30.928 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:30.929 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/588) 2024-04-26T00:00:31.132 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:31.136 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:31.136 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:31.253 INFO:teuthology.orchestra.run.smithi094.stdout:generic/588 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:31.257 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/588 2024-04-26T00:00:31.257 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/588 2024-04-26T00:00:31.257 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:31.259 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:31.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:31.267 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/589) 2024-04-26T00:00:31.453 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:31.457 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:31.457 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:31.565 INFO:teuthology.orchestra.run.smithi094.stdout:generic/589 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:31.568 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/589 2024-04-26T00:00:31.568 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/589 2024-04-26T00:00:31.568 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:31.579 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:31.579 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/590) 2024-04-26T00:00:31.763 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:31.767 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:31.873 INFO:teuthology.orchestra.run.smithi094.stdout:generic/590 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:31.876 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/590 2024-04-26T00:00:31.876 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/590 2024-04-26T00:00:31.876 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:31.878 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:31.886 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:31.886 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/591) 2024-04-26T00:00:32.074 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:32.079 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:32.079 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:generic/591 0s 2024-04-26T00:00:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/591 2024-04-26T00:00:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:32.462 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:32.472 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:32.472 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/592) 2024-04-26T00:00:33.105 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:33.109 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:33.109 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:33.242 INFO:teuthology.orchestra.run.smithi094.stdout:generic/592 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:33.244 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/592 2024-04-26T00:00:33.244 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/592 2024-04-26T00:00:33.244 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:33.245 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:33.254 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:33.254 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/593) 2024-04-26T00:00:33.443 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:33.447 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:33.447 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:33.578 INFO:teuthology.orchestra.run.smithi094.stdout:generic/593 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:33.582 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/593 2024-04-26T00:00:33.582 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/593 2024-04-26T00:00:33.582 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:33.584 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:33.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:33.593 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/594) 2024-04-26T00:00:33.794 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:33.798 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:33.910 INFO:teuthology.orchestra.run.smithi094.stdout:generic/594 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:33.913 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/594 2024-04-26T00:00:33.913 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/594 2024-04-26T00:00:33.914 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:33.916 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:33.924 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:33.924 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/595) 2024-04-26T00:00:34.109 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:34.114 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:34.255 INFO:teuthology.orchestra.run.smithi094.stdout:generic/595 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:34.258 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/595 2024-04-26T00:00:34.258 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/595 2024-04-26T00:00:34.258 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:34.260 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:34.270 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:34.271 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/596) 2024-04-26T00:00:34.456 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:34.460 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:34.460 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:34.578 INFO:teuthology.orchestra.run.smithi094.stdout:generic/596 [not run] accton utility required, skipped this test 2024-04-26T00:00:34.582 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/596 2024-04-26T00:00:34.582 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/596 2024-04-26T00:00:34.582 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:34.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:34.592 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/597) 2024-04-26T00:00:34.780 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:34.784 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:34.784 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:35.391 INFO:teuthology.orchestra.run.smithi094.stdout:generic/597 - output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/597.out.bad) 2024-04-26T00:00:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/597.out 2024-04-25 16:52:50.107099540 +0000 2024-04-26T00:00:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/597.out.bad 2024-04-26 00:00:35.405041686 +0000 2024-04-26T00:00:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -4,11 +4,11 @@ 2024-04-26T00:00:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: fs.protected_symlinks = 0 2024-04-26T00:00:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: successfully followed symlink 2024-04-26T00:00:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: fs.protected_symlinks = 1 2024-04-26T00:00:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: -Permission denied 2024-04-26T00:00:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: +successfully followed symlink 2024-04-26T00:00:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: == Test hardlink create protection when 2024-04-26T00:00:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: == process != target owner and process cannot read target 2024-04-26T00:00:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: fs.protected_hardlinks = 0 2024-04-26T00:00:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: -successfully created hardlink 2024-04-26T00:00:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: +Permission denied 2024-04-26T00:00:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: fs.protected_hardlinks = 1 2024-04-26T00:00:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: -ln: failed to create hard link 'TEST_DIR/597/sticky_dir/hardlink' => 'TEST_DIR/597/target': Operation not permitted 2024-04-26T00:00:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: +Permission denied 2024-04-26T00:00:35.395 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/597 2024-04-26T00:00:35.395 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/597 2024-04-26T00:00:35.395 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-26T00:00:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:35.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:00:35.406 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-26T00:00:35.406 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-26T00:00:35.406 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/598) 2024-04-26T00:00:36.041 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:36.045 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:36.566 INFO:teuthology.orchestra.run.smithi094.stdout:generic/598 - output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/598.out.bad) 2024-04-26T00:00:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/598.out 2024-04-25 16:52:50.107099540 +0000 2024-04-26T00:00:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/598.out.bad 2024-04-26 00:00:36.577023200 +0000 2024-04-26T00:00:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -5,13 +5,10 @@ 2024-04-26T00:00:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: = only group writable dir 2024-04-26T00:00:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: fs.protected_regular = 1 2024-04-26T00:00:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: = group & world writable dir 2024-04-26T00:00:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: -Permission denied 2024-04-26T00:00:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: = only group writable dir 2024-04-26T00:00:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: fs.protected_regular = 2 2024-04-26T00:00:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: = group & world writable dir 2024-04-26T00:00:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: -Permission denied 2024-04-26T00:00:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: = only group writable dir 2024-04-26T00:00:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: -Permission denied 2024-04-26T00:00:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: == Test fifo open when owned by another and fifo owner != dir owner 2024-04-26T00:00:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: fs.protected_fifos = 0 2024-04-26T00:00:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -19,10 +16,7 @@ 2024-04-26T00:00:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: = only group writable dir 2024-04-26T00:00:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: fs.protected_fifos = 1 2024-04-26T00:00:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: = group & world writable dir 2024-04-26T00:00:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: -Permission denied 2024-04-26T00:00:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: = only group writable dir 2024-04-26T00:00:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: fs.protected_fifos = 2 2024-04-26T00:00:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: = group & world writable dir 2024-04-26T00:00:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: -Permission denied 2024-04-26T00:00:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: = only group writable dir 2024-04-26T00:00:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: -Permission denied 2024-04-26T00:00:36.571 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/598 2024-04-26T00:00:36.571 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/598 2024-04-26T00:00:36.571 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-26T00:00:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:36.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:00:36.582 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-26T00:00:36.583 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-26T00:00:36.583 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/599) 2024-04-26T00:00:37.122 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T00:00:37.124 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T00:00:37.133 INFO:teuthology.orchestra.run.smithi094.stderr:error: state file /var/lib/logrotate/status is already locked 2024-04-26T00:00:37.133 INFO:teuthology.orchestra.run.smithi094.stderr:logrotate does not support parallel execution on the same set of logfiles. 2024-04-26T00:00:37.134 DEBUG:teuthology.orchestra.run:got remote process result: 3 2024-04-26T00:00:37.140 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph.py", line 189, in invoke_logrotate run.wait(procs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 3: 'sudo logrotate /etc/logrotate.d/ceph-test.conf' 2024-04-26T00:00:37.161 INFO:teuthology.orchestra.run.smithi138.stderr:error: state file /var/lib/logrotate/status is already locked 2024-04-26T00:00:37.161 INFO:teuthology.orchestra.run.smithi138.stderr:logrotate does not support parallel execution on the same set of logfiles. 2024-04-26T00:00:37.264 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:37.268 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:37.268 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:37.374 INFO:teuthology.orchestra.run.smithi094.stdout:generic/599 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:37.377 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/599 2024-04-26T00:00:37.377 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/599 2024-04-26T00:00:37.377 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:37.380 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:37.388 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:37.388 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/600) 2024-04-26T00:00:37.577 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:37.581 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:37.581 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:37.686 INFO:teuthology.orchestra.run.smithi094.stdout:generic/600 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:37.689 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/600 2024-04-26T00:00:37.690 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/600 2024-04-26T00:00:37.690 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:37.692 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:37.700 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:37.700 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/601) 2024-04-26T00:00:37.893 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:37.897 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:37.897 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:38.001 INFO:teuthology.orchestra.run.smithi094.stdout:generic/601 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:38.005 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/601 2024-04-26T00:00:38.005 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/601 2024-04-26T00:00:38.005 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:38.007 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:38.017 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:38.017 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/602) 2024-04-26T00:00:38.204 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:38.208 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:38.208 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:38.338 INFO:teuthology.orchestra.run.smithi094.stdout:generic/602 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:38.342 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/602 2024-04-26T00:00:38.343 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/602 2024-04-26T00:00:38.343 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:38.344 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:38.352 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:38.352 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/603) 2024-04-26T00:00:38.535 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:38.539 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:38.539 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:38.646 INFO:teuthology.orchestra.run.smithi094.stdout:generic/603 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:38.650 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/603 2024-04-26T00:00:38.650 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/603 2024-04-26T00:00:38.650 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:38.652 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:38.661 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:38.661 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/604) 2024-04-26T00:00:38.844 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:38.848 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:38.849 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:38.952 INFO:teuthology.orchestra.run.smithi094.stdout:generic/604 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:38.956 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/604 2024-04-26T00:00:38.956 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/604 2024-04-26T00:00:38.956 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:38.958 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:38.967 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:38.967 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/605) 2024-04-26T00:00:39.152 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:39.156 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:39.156 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:39.264 INFO:teuthology.orchestra.run.smithi094.stdout:generic/605 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:39.268 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/605 2024-04-26T00:00:39.268 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/605 2024-04-26T00:00:39.268 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:39.271 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:39.279 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:39.279 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/606) 2024-04-26T00:00:39.462 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:39.466 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:39.569 INFO:teuthology.orchestra.run.smithi094.stdout:generic/606 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:39.572 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/606 2024-04-26T00:00:39.572 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/606 2024-04-26T00:00:39.572 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:39.575 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:39.587 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:39.587 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/607) 2024-04-26T00:00:39.771 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:39.775 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:39.775 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:39.882 INFO:teuthology.orchestra.run.smithi094.stdout:generic/607 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:39.886 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/607 2024-04-26T00:00:39.886 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/607 2024-04-26T00:00:39.886 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:39.897 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:39.897 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/608) 2024-04-26T00:00:40.084 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:40.089 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:40.089 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:40.193 INFO:teuthology.orchestra.run.smithi094.stdout:generic/608 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:40.199 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/608 2024-04-26T00:00:40.199 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/608 2024-04-26T00:00:40.199 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:40.199 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:40.208 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:40.208 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/609) 2024-04-26T00:00:40.395 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:40.399 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:40.985 INFO:teuthology.orchestra.run.smithi094.stdout:generic/609 0s 2024-04-26T00:00:40.992 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/609 2024-04-26T00:00:40.992 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:40.995 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:41.005 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:41.005 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/610) 2024-04-26T00:00:41.652 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:41.656 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:41.656 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:41.758 INFO:teuthology.orchestra.run.smithi094.stdout:generic/610 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:41.762 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/610 2024-04-26T00:00:41.762 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/610 2024-04-26T00:00:41.762 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:41.764 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:41.772 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:41.773 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/611) 2024-04-26T00:00:41.960 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:41.964 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:41.964 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:42.127 INFO:teuthology.orchestra.run.smithi094.stdout:generic/611 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:42.131 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/611 2024-04-26T00:00:42.131 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/611 2024-04-26T00:00:42.131 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:42.133 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:42.141 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:42.141 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/612) 2024-04-26T00:00:42.341 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:42.345 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:42.345 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:42.510 INFO:teuthology.orchestra.run.smithi094.stdout:generic/612 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-26T00:00:42.514 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/612 2024-04-26T00:00:42.514 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/612 2024-04-26T00:00:42.514 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:42.516 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:42.524 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:42.524 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/613) 2024-04-26T00:00:43.157 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:43.162 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:43.162 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:43.381 INFO:teuthology.orchestra.run.smithi094.stdout:generic/613 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:43.385 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/613 2024-04-26T00:00:43.385 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/613 2024-04-26T00:00:43.385 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:43.387 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:43.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:43.395 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/614) 2024-04-26T00:00:43.580 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:43.584 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:43.690 INFO:teuthology.orchestra.run.smithi094.stdout:generic/614 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:43.694 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/614 2024-04-26T00:00:43.694 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/614 2024-04-26T00:00:43.694 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:43.696 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:43.705 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:43.706 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/615) 2024-04-26T00:00:43.890 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:43.895 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:43.895 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:43.999 INFO:teuthology.orchestra.run.smithi094.stdout:generic/615 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:00:44.002 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/615 2024-04-26T00:00:44.002 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/615 2024-04-26T00:00:44.002 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:00:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:00:44.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:00:44.013 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/616) 2024-04-26T00:00:44.196 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:00:44.200 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:00:44.200 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:07:38.058 INFO:teuthology.orchestra.run.smithi094.stdout:generic/616 414s 2024-04-26T00:07:38.065 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/616 2024-04-26T00:07:38.065 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:07:38.068 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:07:38.080 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:07:38.081 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/617) 2024-04-26T00:07:38.312 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:07:38.316 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:07:38.317 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:17.539 INFO:teuthology.orchestra.run.smithi094.stdout:generic/617 159s 2024-04-26T00:10:17.546 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/617 2024-04-26T00:10:17.546 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:17.550 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:17.753 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:17.754 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/618) 2024-04-26T00:10:18.160 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:18.164 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:18.164 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:18.268 INFO:teuthology.orchestra.run.smithi094.stdout:generic/618 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:18.272 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/618 2024-04-26T00:10:18.272 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/618 2024-04-26T00:10:18.272 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:18.275 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:18.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:18.283 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/619) 2024-04-26T00:10:18.469 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:18.473 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:18.473 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:18.576 INFO:teuthology.orchestra.run.smithi094.stdout:generic/619 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:18.579 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/619 2024-04-26T00:10:18.579 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/619 2024-04-26T00:10:18.579 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:18.582 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:18.589 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:18.590 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/620) 2024-04-26T00:10:18.775 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:18.779 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:18.779 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:18.915 INFO:teuthology.orchestra.run.smithi094.stdout:generic/620 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:18.918 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/620 2024-04-26T00:10:18.918 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/620 2024-04-26T00:10:18.918 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:18.921 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:18.929 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:18.929 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/621) 2024-04-26T00:10:19.115 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:19.119 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:19.119 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:19.248 INFO:teuthology.orchestra.run.smithi094.stdout:generic/621 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:19.251 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/621 2024-04-26T00:10:19.251 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/621 2024-04-26T00:10:19.251 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:19.254 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:19.262 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:19.266 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/622) 2024-04-26T00:10:19.449 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:19.454 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:19.454 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:19.558 INFO:teuthology.orchestra.run.smithi094.stdout:generic/622 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:19.561 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/622 2024-04-26T00:10:19.562 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/622 2024-04-26T00:10:19.562 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:19.564 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:19.572 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:19.572 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/623) 2024-04-26T00:10:19.758 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:19.762 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:19.762 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:19.888 INFO:teuthology.orchestra.run.smithi094.stdout:generic/623 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:19.892 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/623 2024-04-26T00:10:19.892 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/623 2024-04-26T00:10:19.892 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:19.893 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:19.902 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:19.902 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/624) 2024-04-26T00:10:20.086 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:20.090 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:20.090 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:20.194 INFO:teuthology.orchestra.run.smithi094.stdout:generic/624 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:20.197 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/624 2024-04-26T00:10:20.197 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/624 2024-04-26T00:10:20.198 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:20.200 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:20.208 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:20.208 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/625) 2024-04-26T00:10:20.398 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:20.402 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:20.402 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:20.505 INFO:teuthology.orchestra.run.smithi094.stdout:generic/625 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:20.508 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/625 2024-04-26T00:10:20.509 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/625 2024-04-26T00:10:20.509 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:20.519 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:20.519 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/626) 2024-04-26T00:10:20.704 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:20.708 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:20.708 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:20.830 INFO:teuthology.orchestra.run.smithi094.stdout:generic/626 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:20.833 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/626 2024-04-26T00:10:20.833 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/626 2024-04-26T00:10:20.833 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:20.844 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:20.844 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/627) 2024-04-26T00:10:21.031 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:21.035 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:21.035 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:21.141 INFO:teuthology.orchestra.run.smithi094.stdout:generic/627 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:21.146 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/627 2024-04-26T00:10:21.146 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/627 2024-04-26T00:10:21.146 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:21.147 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:21.156 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:21.156 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/628) 2024-04-26T00:10:21.344 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:21.348 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:21.497 INFO:teuthology.orchestra.run.smithi094.stdout:generic/628 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:21.500 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/628 2024-04-26T00:10:21.501 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/628 2024-04-26T00:10:21.501 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:21.511 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:21.511 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/629) 2024-04-26T00:10:21.697 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:21.701 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:21.701 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:21.822 INFO:teuthology.orchestra.run.smithi094.stdout:generic/629 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:21.826 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/629 2024-04-26T00:10:21.826 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/629 2024-04-26T00:10:21.826 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:21.836 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:21.837 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/630) 2024-04-26T00:10:22.021 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:22.026 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:22.026 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:22.128 INFO:teuthology.orchestra.run.smithi094.stdout:generic/630 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:22.132 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/630 2024-04-26T00:10:22.132 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/630 2024-04-26T00:10:22.132 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:22.134 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:22.142 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:22.142 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/631) 2024-04-26T00:10:22.327 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:22.331 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:22.437 INFO:teuthology.orchestra.run.smithi094.stdout:generic/631 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:22.441 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/631 2024-04-26T00:10:22.441 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/631 2024-04-26T00:10:22.441 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:22.443 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:22.451 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:22.452 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/632) 2024-04-26T00:10:22.635 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:22.639 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:22.639 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:29.856 INFO:teuthology.orchestra.run.smithi094.stdout:generic/632 7s 2024-04-26T00:10:29.864 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/632 2024-04-26T00:10:29.864 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:29.867 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:29.909 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:29.909 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/633) 2024-04-26T00:10:30.139 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:30.144 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:30.144 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:30.643 INFO:teuthology.orchestra.run.smithi094.stdout:generic/633 [failed, exit status 1]- output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/633.out.bad) 2024-04-26T00:10:30.646 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/633.out 2024-04-25 16:52:50.111099484 +0000 2024-04-26T00:10:30.646 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/633.out.bad 2024-04-26 00:10:30.659654549 +0000 2024-04-26T00:10:30.646 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -1,2 +1,5 @@ 2024-04-26T00:10:30.646 INFO:teuthology.orchestra.run.smithi094.stdout: QA output created by 633 2024-04-26T00:10:30.646 INFO:teuthology.orchestra.run.smithi094.stdout: Silence is golden 2024-04-26T00:10:30.646 INFO:teuthology.orchestra.run.smithi094.stdout: +vfstest.c: 1164: protected_symlinks - Success - failure: openat 2024-04-26T00:10:30.647 INFO:teuthology.orchestra.run.smithi094.stdout: +vfstest.c: 1171: protected_symlinks - Success - failure: wait_for_pid 2024-04-26T00:10:30.647 INFO:teuthology.orchestra.run.smithi094.stdout: +vfstest.c: 2418: run_test - Success - failure: following protected symlinks on regular mounts 2024-04-26T00:10:30.648 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/633 2024-04-26T00:10:30.649 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/633 2024-04-26T00:10:30.649 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-26T00:10:30.652 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:30.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:10:30.672 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-26T00:10:30.672 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-26T00:10:30.673 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/634) 2024-04-26T00:10:31.219 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:31.223 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:31.223 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:31.327 INFO:teuthology.orchestra.run.smithi094.stdout:generic/634 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:31.330 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/634 2024-04-26T00:10:31.330 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/634 2024-04-26T00:10:31.330 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:31.333 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:31.341 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:31.341 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/635) 2024-04-26T00:10:31.524 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:31.529 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:31.529 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:31.635 INFO:teuthology.orchestra.run.smithi094.stdout:generic/635 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:31.636 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/635 2024-04-26T00:10:31.636 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/635 2024-04-26T00:10:31.636 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:31.638 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:31.647 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:31.647 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/636) 2024-04-26T00:10:31.833 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:31.841 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:31.943 INFO:teuthology.orchestra.run.smithi094.stdout:generic/636 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:31.947 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/636 2024-04-26T00:10:31.947 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/636 2024-04-26T00:10:31.947 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:31.949 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:31.957 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:31.957 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/637) 2024-04-26T00:10:32.165 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:32.167 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:32.167 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:34.284 INFO:teuthology.orchestra.run.smithi094.stdout:generic/637 2s 2024-04-26T00:10:34.292 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/637 2024-04-26T00:10:34.292 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:34.295 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:34.306 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:34.306 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/638) 2024-04-26T00:10:34.902 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:34.907 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:34.907 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:generic/638 1s 2024-04-26T00:10:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/638 2024-04-26T00:10:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:35.078 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:35.086 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:35.087 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/639) 2024-04-26T00:10:35.270 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:35.274 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:35.274 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:40.014 INFO:teuthology.orchestra.run.smithi094.stdout:generic/639 4s 2024-04-26T00:10:40.021 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/639 2024-04-26T00:10:40.022 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:40.036 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:40.036 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/640) 2024-04-26T00:10:40.269 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:40.274 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:40.274 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:40.412 INFO:teuthology.orchestra.run.smithi094.stdout:generic/640 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:40.416 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/640 2024-04-26T00:10:40.416 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/640 2024-04-26T00:10:40.416 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:40.429 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:40.430 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/641) 2024-04-26T00:10:40.650 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:40.655 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:40.655 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:40.776 INFO:teuthology.orchestra.run.smithi094.stdout:generic/641 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:40.780 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/641 2024-04-26T00:10:40.781 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/641 2024-04-26T00:10:40.781 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:40.783 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:40.792 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:40.793 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/642) 2024-04-26T00:10:40.995 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:41.000 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:41.000 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:41.106 INFO:teuthology.orchestra.run.smithi094.stdout:generic/642 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:41.110 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/642 2024-04-26T00:10:41.110 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/642 2024-04-26T00:10:41.110 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:41.112 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:41.122 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:41.122 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/643) 2024-04-26T00:10:41.309 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:41.313 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:41.415 INFO:teuthology.orchestra.run.smithi094.stdout:generic/643 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:41.419 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/643 2024-04-26T00:10:41.419 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/643 2024-04-26T00:10:41.419 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:41.421 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:41.429 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:41.429 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/644) 2024-04-26T00:10:41.618 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:41.622 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:41.622 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:41.727 INFO:teuthology.orchestra.run.smithi094.stdout:generic/644 [not run] vfstest not support by ceph-fuse 2024-04-26T00:10:41.731 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/644 2024-04-26T00:10:41.731 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/644 2024-04-26T00:10:41.731 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:41.733 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:41.741 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:41.741 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/645) 2024-04-26T00:10:41.924 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:41.928 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:41.928 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:42.038 INFO:teuthology.orchestra.run.smithi094.stdout:generic/645 [not run] vfstest not support by ceph-fuse 2024-04-26T00:10:42.042 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/645 2024-04-26T00:10:42.042 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/645 2024-04-26T00:10:42.042 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:42.044 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:42.052 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:42.052 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/646) 2024-04-26T00:10:42.236 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:42.240 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:42.240 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:42.340 INFO:teuthology.orchestra.run.smithi094.stdout:generic/646 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:42.344 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/646 2024-04-26T00:10:42.344 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/646 2024-04-26T00:10:42.344 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:42.346 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:42.355 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:42.356 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/647) 2024-04-26T00:10:42.538 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:42.542 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:42.542 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:43.028 INFO:teuthology.orchestra.run.smithi094.stdout:generic/647 1s 2024-04-26T00:10:43.036 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/647 2024-04-26T00:10:43.036 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:43.040 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:43.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:43.051 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/648) 2024-04-26T00:10:43.598 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:43.603 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:43.726 INFO:teuthology.orchestra.run.smithi094.stdout:generic/648 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T00:10:43.730 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/648 2024-04-26T00:10:43.730 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/648 2024-04-26T00:10:43.730 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:43.732 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:43.740 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:43.741 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/649) 2024-04-26T00:10:43.926 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:43.930 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:43.930 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:44.101 INFO:teuthology.orchestra.run.smithi094.stdout:generic/649 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-26T00:10:44.104 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/649 2024-04-26T00:10:44.104 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/649 2024-04-26T00:10:44.104 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T00:10:44.106 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T00:10:44.114 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T00:10:44.115 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/650) 2024-04-26T00:10:44.642 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T00:10:44.646 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T00:10:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:38.428 INFO:teuthology.orchestra.run.smithi094.stdout:generic/650 3714s 2024-04-26T01:12:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/650 2024-04-26T01:12:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:38.500 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:38.510 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:38.510 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/651) 2024-04-26T01:12:39.654 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:39.659 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:39.659 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:39.847 INFO:teuthology.orchestra.run.smithi094.stdout:generic/651 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:39.851 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/651 2024-04-26T01:12:39.851 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/651 2024-04-26T01:12:39.851 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:39.854 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:39.865 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:39.865 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/652) 2024-04-26T01:12:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:40.192 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:40.192 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:40.349 INFO:teuthology.orchestra.run.smithi094.stdout:generic/652 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:40.353 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/652 2024-04-26T01:12:40.353 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/652 2024-04-26T01:12:40.353 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:40.357 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:40.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:40.373 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/653) 2024-04-26T01:12:40.645 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:40.649 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:40.650 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:40.806 INFO:teuthology.orchestra.run.smithi094.stdout:generic/653 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:40.815 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/653 2024-04-26T01:12:40.816 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/653 2024-04-26T01:12:40.816 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:40.827 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:40.827 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/654) 2024-04-26T01:12:41.148 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:41.153 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:41.153 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:41.297 INFO:teuthology.orchestra.run.smithi094.stdout:generic/654 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:41.305 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/654 2024-04-26T01:12:41.306 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/654 2024-04-26T01:12:41.306 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:41.368 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:41.368 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/655) 2024-04-26T01:12:41.648 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:41.653 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:41.653 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:41.863 INFO:teuthology.orchestra.run.smithi094.stdout:generic/655 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:41.868 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/655 2024-04-26T01:12:41.868 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/655 2024-04-26T01:12:41.868 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:41.872 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:41.882 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:41.882 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/656) 2024-04-26T01:12:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:42.180 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:42.180 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:42.549 INFO:teuthology.orchestra.run.smithi094.stdout:generic/656 [not run] vfstest not support by ceph-fuse 2024-04-26T01:12:42.554 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/656 2024-04-26T01:12:42.554 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/656 2024-04-26T01:12:42.554 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:42.557 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:42.568 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:42.568 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/657) 2024-04-26T01:12:42.822 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:42.827 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:42.827 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:43.082 INFO:teuthology.orchestra.run.smithi094.stdout:generic/657 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:43.092 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/657 2024-04-26T01:12:43.093 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/657 2024-04-26T01:12:43.093 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:43.096 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:43.104 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:43.104 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/658) 2024-04-26T01:12:43.400 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:43.404 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:43.404 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:43.936 INFO:teuthology.orchestra.run.smithi094.stdout:generic/658 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:43.940 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/658 2024-04-26T01:12:43.941 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/658 2024-04-26T01:12:43.941 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:43.943 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:43.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:43.954 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/659) 2024-04-26T01:12:44.239 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:44.244 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:44.244 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:44.637 INFO:teuthology.orchestra.run.smithi094.stdout:generic/659 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:44.641 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/659 2024-04-26T01:12:44.641 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/659 2024-04-26T01:12:44.641 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:44.658 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:44.658 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/660) 2024-04-26T01:12:44.968 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:44.972 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:44.973 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:45.271 INFO:teuthology.orchestra.run.smithi094.stdout:generic/660 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:45.276 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/660 2024-04-26T01:12:45.276 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/660 2024-04-26T01:12:45.276 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:45.290 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:45.290 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/661) 2024-04-26T01:12:45.609 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:45.613 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:45.613 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:46.048 INFO:teuthology.orchestra.run.smithi094.stdout:generic/661 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:46.052 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/661 2024-04-26T01:12:46.052 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/661 2024-04-26T01:12:46.052 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:46.054 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:46.063 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:46.063 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/662) 2024-04-26T01:12:46.358 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:46.362 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:46.362 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:47.085 INFO:teuthology.orchestra.run.smithi094.stdout:generic/662 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:47.089 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/662 2024-04-26T01:12:47.090 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/662 2024-04-26T01:12:47.090 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:47.093 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:47.110 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:47.110 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/663) 2024-04-26T01:12:47.424 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:47.433 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:47.433 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:48.108 INFO:teuthology.orchestra.run.smithi094.stdout:generic/663 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:48.117 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/663 2024-04-26T01:12:48.118 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/663 2024-04-26T01:12:48.118 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:48.120 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:48.131 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:48.132 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/664) 2024-04-26T01:12:48.397 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:48.404 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:48.404 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:49.243 INFO:teuthology.orchestra.run.smithi094.stdout:generic/664 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:49.248 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/664 2024-04-26T01:12:49.248 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/664 2024-04-26T01:12:49.248 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:49.251 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:49.261 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:49.261 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/665) 2024-04-26T01:12:49.554 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:49.565 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:49.565 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:49.813 INFO:teuthology.orchestra.run.smithi094.stdout:generic/665 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:49.817 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/665 2024-04-26T01:12:49.817 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/665 2024-04-26T01:12:49.817 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:49.837 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:49.837 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/666) 2024-04-26T01:12:50.102 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:50.107 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:50.107 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:50.524 INFO:teuthology.orchestra.run.smithi094.stdout:generic/666 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:50.528 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/666 2024-04-26T01:12:50.528 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/666 2024-04-26T01:12:50.529 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:50.540 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:50.540 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/667) 2024-04-26T01:12:50.814 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:50.822 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:50.822 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:51.207 INFO:teuthology.orchestra.run.smithi094.stdout:generic/667 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:51.215 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/667 2024-04-26T01:12:51.215 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/667 2024-04-26T01:12:51.215 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:51.217 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:51.230 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:51.230 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/668) 2024-04-26T01:12:51.553 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:51.557 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:51.558 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:51.709 INFO:teuthology.orchestra.run.smithi094.stdout:generic/668 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:51.712 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/668 2024-04-26T01:12:51.713 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/668 2024-04-26T01:12:51.713 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:51.715 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:51.727 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:51.727 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/669) 2024-04-26T01:12:52.023 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:52.028 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:52.211 INFO:teuthology.orchestra.run.smithi094.stdout:generic/669 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:52.215 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/669 2024-04-26T01:12:52.215 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/669 2024-04-26T01:12:52.215 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:52.217 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:52.227 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:52.227 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/670) 2024-04-26T01:12:52.530 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:52.536 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:52.785 INFO:teuthology.orchestra.run.smithi094.stdout:generic/670 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:52.789 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/670 2024-04-26T01:12:52.789 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/670 2024-04-26T01:12:52.789 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:52.802 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:52.803 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/671) 2024-04-26T01:12:53.066 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:53.072 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:53.217 INFO:teuthology.orchestra.run.smithi094.stdout:generic/671 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:53.221 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/671 2024-04-26T01:12:53.221 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/671 2024-04-26T01:12:53.221 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:53.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:53.233 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/672) 2024-04-26T01:12:53.508 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:53.512 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:53.670 INFO:teuthology.orchestra.run.smithi094.stdout:generic/672 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:53.675 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/672 2024-04-26T01:12:53.675 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/672 2024-04-26T01:12:53.675 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:53.678 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:53.690 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:53.690 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/673) 2024-04-26T01:12:53.983 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:53.987 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:53.988 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:54.577 INFO:teuthology.orchestra.run.smithi094.stdout:generic/673 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:54.582 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/673 2024-04-26T01:12:54.582 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/673 2024-04-26T01:12:54.582 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:54.584 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:54.596 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:54.596 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/674) 2024-04-26T01:12:54.897 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:54.903 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:54.903 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:55.172 INFO:teuthology.orchestra.run.smithi094.stdout:generic/674 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:55.176 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/674 2024-04-26T01:12:55.176 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/674 2024-04-26T01:12:55.177 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:55.207 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:55.207 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/675) 2024-04-26T01:12:55.481 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:55.486 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:55.486 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:55.662 INFO:teuthology.orchestra.run.smithi094.stdout:generic/675 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:12:55.671 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/675 2024-04-26T01:12:55.671 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/675 2024-04-26T01:12:55.671 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:12:55.674 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:12:55.691 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:12:55.691 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/676) 2024-04-26T01:12:56.013 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:12:56.018 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:12:56.018 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:09.173 INFO:teuthology.orchestra.run.smithi094.stdout:generic/676 1693s 2024-04-26T01:41:09.272 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/676 2024-04-26T01:41:09.272 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:09.274 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:09.288 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:09.289 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/677) 2024-04-26T01:41:10.572 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:10.577 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:10.577 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:10.798 INFO:teuthology.orchestra.run.smithi094.stdout:generic/677 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/677 2024-04-26T01:41:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/677 2024-04-26T01:41:10.803 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:10.805 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:10.819 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:10.819 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/678) 2024-04-26T01:41:11.105 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:11.110 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:11.576 INFO:teuthology.orchestra.run.smithi094.stdout:generic/678 0s 2024-04-26T01:41:11.589 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/678 2024-04-26T01:41:11.589 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:11.603 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:11.603 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/679) 2024-04-26T01:41:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:12.311 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:12.473 INFO:teuthology.orchestra.run.smithi094.stdout:generic/679 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:12.477 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/679 2024-04-26T01:41:12.477 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/679 2024-04-26T01:41:12.477 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:12.479 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:12.497 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:12.498 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/680) 2024-04-26T01:41:12.814 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:12.823 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:12.824 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:13.540 INFO:teuthology.orchestra.run.smithi094.stdout:generic/680 1s 2024-04-26T01:41:13.548 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/680 2024-04-26T01:41:13.548 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:13.552 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:13.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:13.575 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/681) 2024-04-26T01:41:14.289 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:14.297 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:14.297 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:14.470 INFO:teuthology.orchestra.run.smithi094.stdout:generic/681 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-26T01:41:14.479 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/681 2024-04-26T01:41:14.480 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/681 2024-04-26T01:41:14.480 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:14.482 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:14.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:14.493 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/682) 2024-04-26T01:41:14.790 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:14.795 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:14.795 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:14.946 INFO:teuthology.orchestra.run.smithi094.stdout:generic/682 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-26T01:41:14.951 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/682 2024-04-26T01:41:14.951 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/682 2024-04-26T01:41:14.951 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:14.953 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:14.968 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:14.969 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/683) 2024-04-26T01:41:15.272 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:15.276 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:15.276 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:15.524 INFO:teuthology.orchestra.run.smithi094.stdout:generic/683 [not run] /tmp/tmp.PS50Hvgg8stest: file alloc unit 4194304 larger than op length 65536 2024-04-26T01:41:15.528 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/683 2024-04-26T01:41:15.528 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/683 2024-04-26T01:41:15.528 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:15.531 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:15.551 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:15.551 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/684) 2024-04-26T01:41:15.913 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:15.918 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:17.619 INFO:teuthology.orchestra.run.smithi094.stdout:generic/684 [not run] /tmp/tmp.PS50Hvgg8stest: file alloc unit 4194304 larger than op length 65536 2024-04-26T01:41:17.623 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/684 2024-04-26T01:41:17.623 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/684 2024-04-26T01:41:17.623 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:17.626 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:17.635 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:17.636 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/685) 2024-04-26T01:41:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:17.909 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:20.813 INFO:teuthology.orchestra.run.smithi094.stdout:generic/685 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-26T01:41:20.817 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/685 2024-04-26T01:41:20.818 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/685 2024-04-26T01:41:20.818 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:20.821 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:20.831 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:20.832 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/686) 2024-04-26T01:41:21.131 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:21.138 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:23.894 INFO:teuthology.orchestra.run.smithi094.stdout:generic/686 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-26T01:41:23.898 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/686 2024-04-26T01:41:23.898 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/686 2024-04-26T01:41:23.898 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:23.900 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:23.921 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:23.921 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/687) 2024-04-26T01:41:24.204 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:24.209 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:26.473 INFO:teuthology.orchestra.run.smithi094.stdout:generic/687 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-04-26T01:41:26.477 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/687 2024-04-26T01:41:26.477 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/687 2024-04-26T01:41:26.477 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:26.480 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:26.490 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:26.490 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/688) 2024-04-26T01:41:26.813 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:26.818 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:26.818 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:27.124 INFO:teuthology.orchestra.run.smithi094.stdout:generic/688 [not run] /tmp/tmp.PS50Hvgg8stest: file alloc unit 4194304 larger than op length 65536 2024-04-26T01:41:27.129 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/688 2024-04-26T01:41:27.129 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/688 2024-04-26T01:41:27.129 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:27.136 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:27.149 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:27.150 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/689) 2024-04-26T01:41:27.440 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:27.445 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:27.445 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:27.652 INFO:teuthology.orchestra.run.smithi094.stdout:generic/689 [not run] vfstest not support by ceph-fuse 2024-04-26T01:41:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/689 2024-04-26T01:41:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/689 2024-04-26T01:41:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:27.659 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:27.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:27.671 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/690) 2024-04-26T01:41:27.993 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:27.997 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:28.202 INFO:teuthology.orchestra.run.smithi094.stdout:generic/690 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:28.207 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/690 2024-04-26T01:41:28.207 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/690 2024-04-26T01:41:28.207 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:28.210 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:28.223 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:28.223 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/691) 2024-04-26T01:41:28.513 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:28.518 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:28.710 INFO:teuthology.orchestra.run.smithi094.stdout:generic/691 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:28.712 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/691 2024-04-26T01:41:28.712 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/691 2024-04-26T01:41:28.713 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:28.715 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:28.727 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:28.727 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/692) 2024-04-26T01:41:29.012 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:29.025 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:29.025 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:29.195 INFO:teuthology.orchestra.run.smithi094.stdout:generic/692 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:29.199 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/692 2024-04-26T01:41:29.199 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/692 2024-04-26T01:41:29.199 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:29.202 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:29.214 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:29.214 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/693) 2024-04-26T01:41:29.519 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:29.524 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:29.723 INFO:teuthology.orchestra.run.smithi094.stdout:generic/693 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:29.731 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/693 2024-04-26T01:41:29.731 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/693 2024-04-26T01:41:29.731 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:29.734 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:29.751 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:29.751 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/694) 2024-04-26T01:41:30.043 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:30.047 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:30.852 INFO:teuthology.orchestra.run.smithi094.stdout:generic/694 0s 2024-04-26T01:41:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/694 2024-04-26T01:41:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:30.951 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:30.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:30.960 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/695) 2024-04-26T01:41:31.260 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:31.265 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:31.265 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:31.495 INFO:teuthology.orchestra.run.smithi094.stdout:generic/695 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:31.499 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/695 2024-04-26T01:41:31.499 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/695 2024-04-26T01:41:31.499 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:31.502 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:31.510 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:31.511 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/696) 2024-04-26T01:41:31.855 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:31.860 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:31.860 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:32.048 INFO:teuthology.orchestra.run.smithi094.stdout:generic/696 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:32.052 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/696 2024-04-26T01:41:32.052 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/696 2024-04-26T01:41:32.052 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:32.055 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:32.071 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:32.071 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/697) 2024-04-26T01:41:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:32.372 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:generic/697 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-26T01:41:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/697 2024-04-26T01:41:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/697 2024-04-26T01:41:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:32.617 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:32.626 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:32.626 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/698) 2024-04-26T01:41:32.952 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:32.956 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:32.956 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:33.141 INFO:teuthology.orchestra.run.smithi094.stdout:generic/698 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:33.144 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/698 2024-04-26T01:41:33.144 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/698 2024-04-26T01:41:33.144 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:33.152 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:33.163 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:33.164 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/699) 2024-04-26T01:41:33.478 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:33.482 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:33.482 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:33.782 INFO:teuthology.orchestra.run.smithi094.stdout:generic/699 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:33.787 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/699 2024-04-26T01:41:33.787 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/699 2024-04-26T01:41:33.787 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:33.790 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:33.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:33.805 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/700) 2024-04-26T01:41:34.074 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:34.078 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:34.078 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:34.271 INFO:teuthology.orchestra.run.smithi094.stdout:generic/700 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:34.275 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/700 2024-04-26T01:41:34.275 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/700 2024-04-26T01:41:34.275 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:34.277 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:34.289 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:34.289 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/701) 2024-04-26T01:41:34.553 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:34.558 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:35.060 INFO:teuthology.orchestra.run.smithi094.stdout:generic/701 1s 2024-04-26T01:41:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/701 2024-04-26T01:41:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:35.076 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:35.087 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:35.088 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/702) 2024-04-26T01:41:35.758 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:35.765 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:generic/702 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:35.915 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/702 2024-04-26T01:41:35.915 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/702 2024-04-26T01:41:35.915 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:35.933 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:35.933 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/703) 2024-04-26T01:41:36.208 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:36.213 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:36.422 INFO:teuthology.orchestra.run.smithi094.stdout:generic/703 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:36.425 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/703 2024-04-26T01:41:36.425 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/703 2024-04-26T01:41:36.425 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:36.444 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:36.445 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/704) 2024-04-26T01:41:36.718 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:36.724 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:36.724 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:36.991 INFO:teuthology.orchestra.run.smithi094.stdout:generic/704 [not run] require ceph-fuse to be valid block disk 2024-04-26T01:41:36.995 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/704 2024-04-26T01:41:36.995 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/704 2024-04-26T01:41:36.995 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:36.998 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:37.007 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:37.007 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/705) 2024-04-26T01:41:37.303 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:37.307 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:37.307 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:37.452 INFO:teuthology.orchestra.run.smithi094.stdout:generic/705 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:37.456 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/705 2024-04-26T01:41:37.456 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/705 2024-04-26T01:41:37.456 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:37.460 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:37.470 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:37.471 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/706) 2024-04-26T01:41:37.781 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:37.786 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:37.786 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:39.956 INFO:teuthology.orchestra.run.smithi094.stdout:generic/706 2s 2024-04-26T01:41:39.962 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/706 2024-04-26T01:41:39.963 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:39.965 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:40.215 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:40.216 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/707) 2024-04-26T01:41:40.670 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:40.675 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:40.675 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:40.846 INFO:teuthology.orchestra.run.smithi094.stdout:generic/707 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:41:40.850 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/707 2024-04-26T01:41:40.850 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/707 2024-04-26T01:41:40.850 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:40.889 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:40.889 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/708) 2024-04-26T01:41:41.192 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:41.197 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:41.197 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:41.917 INFO:teuthology.orchestra.run.smithi094.stdout:generic/708 0s 2024-04-26T01:41:41.922 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/708 2024-04-26T01:41:41.922 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:41.925 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:41.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:41.973 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/709) 2024-04-26T01:41:42.604 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:42.617 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:42.617 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:42.827 INFO:teuthology.orchestra.run.smithi094.stdout:generic/709 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-26T01:41:42.834 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/709 2024-04-26T01:41:42.834 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/709 2024-04-26T01:41:42.834 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:42.835 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:42.850 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:42.850 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/710) 2024-04-26T01:41:43.499 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:43.504 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:43.505 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:43.709 INFO:teuthology.orchestra.run.smithi094.stdout:generic/710 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-26T01:41:43.713 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/710 2024-04-26T01:41:43.714 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/710 2024-04-26T01:41:43.714 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:43.716 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:43.734 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:43.734 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/711) 2024-04-26T01:41:44.462 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:44.467 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:44.691 INFO:teuthology.orchestra.run.smithi094.stdout:generic/711 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-26T01:41:44.696 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/711 2024-04-26T01:41:44.696 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/711 2024-04-26T01:41:44.696 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:44.709 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:44.709 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/712) 2024-04-26T01:41:45.388 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:45.393 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:45.393 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:45.629 INFO:teuthology.orchestra.run.smithi094.stdout:generic/712 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-26T01:41:45.633 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/712 2024-04-26T01:41:45.633 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/712 2024-04-26T01:41:45.633 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:45.636 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:45.650 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:45.651 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/713) 2024-04-26T01:41:46.290 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:46.295 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:46.295 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:generic/713 [not run] xfs_io swapext -v exchrange -s 64k -l 64k not supported on ceph-fuse 2024-04-26T01:41:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/713 2024-04-26T01:41:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/713 2024-04-26T01:41:46.531 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:46.534 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:46.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:46.544 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/714) 2024-04-26T01:41:47.230 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:47.234 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:47.234 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:47.500 INFO:teuthology.orchestra.run.smithi094.stdout:generic/714 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-26T01:41:47.503 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/714 2024-04-26T01:41:47.504 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/714 2024-04-26T01:41:47.504 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:47.512 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:47.522 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:47.522 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/715) 2024-04-26T01:41:48.230 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:48.235 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:48.235 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:48.441 INFO:teuthology.orchestra.run.smithi094.stdout:generic/715 [not run] xfs_io swapext -v exchrange -s 64k -l 64k not supported on ceph-fuse 2024-04-26T01:41:48.445 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/715 2024-04-26T01:41:48.445 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/715 2024-04-26T01:41:48.445 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:48.448 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:48.456 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:48.457 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/716) 2024-04-26T01:41:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:49.180 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:49.180 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:49.583 INFO:teuthology.orchestra.run.smithi094.stdout:generic/716 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-26T01:41:49.588 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/716 2024-04-26T01:41:49.588 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/716 2024-04-26T01:41:49.588 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:49.590 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:49.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:49.599 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/717) 2024-04-26T01:41:50.291 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:50.295 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:50.295 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:50.570 INFO:teuthology.orchestra.run.smithi094.stdout:generic/717 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-26T01:41:50.574 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/717 2024-04-26T01:41:50.574 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/717 2024-04-26T01:41:50.574 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:50.588 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:50.588 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/718) 2024-04-26T01:41:51.298 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:51.304 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:51.530 INFO:teuthology.orchestra.run.smithi094.stdout:generic/718 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-26T01:41:51.533 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/718 2024-04-26T01:41:51.533 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/718 2024-04-26T01:41:51.534 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:51.536 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:51.556 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:51.556 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/719) 2024-04-26T01:41:52.269 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:52.280 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:53.390 INFO:teuthology.orchestra.run.smithi094.stdout:generic/719 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-26T01:41:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/719 2024-04-26T01:41:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/719 2024-04-26T01:41:53.396 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:53.408 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:53.408 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/720) 2024-04-26T01:41:54.086 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:54.091 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:54.370 INFO:teuthology.orchestra.run.smithi094.stdout:generic/720 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-26T01:41:54.379 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/720 2024-04-26T01:41:54.380 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/720 2024-04-26T01:41:54.380 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:54.385 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:54.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:54.396 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/721) 2024-04-26T01:41:55.099 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:55.104 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:55.104 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:55.345 INFO:teuthology.orchestra.run.smithi094.stdout:generic/721 [not run] xfs_io startupdate support is missing 2024-04-26T01:41:55.350 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/721 2024-04-26T01:41:55.350 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/721 2024-04-26T01:41:55.350 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:55.354 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:55.371 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:55.371 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/722) 2024-04-26T01:41:55.689 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:55.694 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:55.694 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:55.936 INFO:teuthology.orchestra.run.smithi094.stdout:generic/722 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-26T01:41:55.941 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/722 2024-04-26T01:41:55.941 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/722 2024-04-26T01:41:55.941 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:55.943 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:55.952 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:55.952 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/723) 2024-04-26T01:41:56.642 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:56.646 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:56.646 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:56.887 INFO:teuthology.orchestra.run.smithi094.stdout:generic/723 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-26T01:41:56.892 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/723 2024-04-26T01:41:56.892 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/723 2024-04-26T01:41:56.892 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:56.895 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:56.912 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:56.912 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/724) 2024-04-26T01:41:57.591 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:57.595 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:57.826 INFO:teuthology.orchestra.run.smithi094.stdout:generic/724 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-26T01:41:57.835 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/724 2024-04-26T01:41:57.836 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/724 2024-04-26T01:41:57.836 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:57.845 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:57.867 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:57.868 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/725) 2024-04-26T01:41:58.575 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:58.580 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:58.580 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:58.821 INFO:teuthology.orchestra.run.smithi094.stdout:generic/725 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-26T01:41:58.825 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/725 2024-04-26T01:41:58.825 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/725 2024-04-26T01:41:58.825 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:41:58.827 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:41:58.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:41:58.841 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/726) 2024-04-26T01:41:59.577 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:41:59.581 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:41:59.582 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:00.224 INFO:teuthology.orchestra.run.smithi094.stdout:generic/726 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-26T01:42:00.229 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/726 2024-04-26T01:42:00.229 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/726 2024-04-26T01:42:00.229 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:42:00.234 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:00.245 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:42:00.245 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/727) 2024-04-26T01:42:00.926 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:42:00.931 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:42:00.931 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:01.298 INFO:teuthology.orchestra.run.smithi094.stdout:generic/727 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-26T01:42:01.309 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/727 2024-04-26T01:42:01.309 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/727 2024-04-26T01:42:01.309 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:42:01.312 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:01.327 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:42:01.327 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/728) 2024-04-26T01:42:02.028 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:42:02.032 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:42:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:generic/728 4s 2024-04-26T01:42:06.323 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/728 2024-04-26T01:42:06.323 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:42:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:06.344 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:42:06.344 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/729) 2024-04-26T01:42:06.662 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:42:06.667 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:42:06.667 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:07.296 INFO:teuthology.orchestra.run.smithi094.stdout:generic/729 1s 2024-04-26T01:42:07.308 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/729 2024-04-26T01:42:07.308 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:42:07.312 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:07.324 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:42:07.324 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/730) 2024-04-26T01:42:08.005 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:42:08.010 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:42:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:generic/730 [not run] require ceph-fuse to be valid block disk 2024-04-26T01:42:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/730 2024-04-26T01:42:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/730 2024-04-26T01:42:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:42:08.202 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:08.214 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:42:08.214 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/731) 2024-04-26T01:42:08.498 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:42:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:42:08.502 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:08.671 INFO:teuthology.orchestra.run.smithi094.stdout:generic/731 [not run] require ceph-fuse to be valid block disk 2024-04-26T01:42:08.675 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/731 2024-04-26T01:42:08.675 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/731 2024-04-26T01:42:08.675 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:42:08.678 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:08.700 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:42:08.700 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/732) 2024-04-26T01:42:08.960 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:42:08.965 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:42:08.966 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:09.135 INFO:teuthology.orchestra.run.smithi094.stdout:generic/732 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:42:09.141 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/732 2024-04-26T01:42:09.141 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/732 2024-04-26T01:42:09.141 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:42:09.144 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:09.157 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:42:09.157 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/733) 2024-04-26T01:42:09.430 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:42:09.434 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:42:09.434 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:09.608 INFO:teuthology.orchestra.run.smithi094.stdout:generic/733 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:42:09.612 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/733 2024-04-26T01:42:09.612 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/733 2024-04-26T01:42:09.612 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:42:09.615 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:09.625 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:42:09.625 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/734) 2024-04-26T01:42:09.886 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:42:09.890 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:42:09.891 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:10.156 INFO:teuthology.orchestra.run.smithi094.stdout:generic/734 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-26T01:42:10.160 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/734 2024-04-26T01:42:10.160 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/734 2024-04-26T01:42:10.160 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:42:10.162 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:10.172 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:42:10.172 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/735) 2024-04-26T01:42:10.840 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:42:10.845 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:42:10.845 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:12.509 INFO:teuthology.orchestra.run.smithi094.stdout:generic/735 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-26T01:42:12.520 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/735 2024-04-26T01:42:12.520 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/735 2024-04-26T01:42:12.520 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:42:12.523 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:42:12.535 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:42:12.535 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/736) 2024-04-26T01:42:13.232 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:42:13.237 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:42:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:46:00.685 INFO:teuthology.orchestra.run.smithi094.stdout:generic/736 227s 2024-04-26T01:46:00.702 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/736 2024-04-26T01:46:00.702 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:46:00.705 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:46:00.716 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:46:00.716 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/737) 2024-04-26T01:46:01.400 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:46:01.404 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:46:01.405 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:46:01.648 INFO:teuthology.orchestra.run.smithi094.stdout:generic/737 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:46:01.652 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/737 2024-04-26T01:46:01.653 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/737 2024-04-26T01:46:01.653 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:46:01.655 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:46:01.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:46:01.668 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/738) 2024-04-26T01:46:01.949 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:46:01.954 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:46:01.954 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:46:03.050 INFO:teuthology.orchestra.run.smithi094.stdout:generic/738 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:46:03.058 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/738 2024-04-26T01:46:03.058 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/738 2024-04-26T01:46:03.058 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:46:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:46:03.075 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:46:03.076 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/739) 2024-04-26T01:46:03.383 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:46:03.388 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:46:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:46:03.681 INFO:teuthology.orchestra.run.smithi094.stdout:generic/739 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:46:03.685 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/739 2024-04-26T01:46:03.685 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/739 2024-04-26T01:46:03.685 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:46:03.688 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:46:03.703 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:46:03.703 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/741) 2024-04-26T01:46:04.002 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:46:04.007 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:46:04.007 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:46:04.251 INFO:teuthology.orchestra.run.smithi094.stdout:generic/741 [not run] this test requires a valid $SCRATCH_DEV 2024-04-26T01:46:04.255 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/741 2024-04-26T01:46:04.255 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/741 2024-04-26T01:46:04.255 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T01:46:04.257 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T01:46:04.269 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T01:46:04.269 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/742) 2024-04-26T01:46:04.590 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T01:46:04.594 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T01:46:04.594 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:04.793 INFO:teuthology.orchestra.run.smithi094.stdout:generic/742 [failed, exit status 1]- output mismatch (see /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/742.out.bad) 2024-04-26T02:28:05.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:28:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: --- tests/generic/742.out 2024-04-25 16:52:50.119099372 +0000 2024-04-26T02:28:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: +++ /tmp/tmp.FooT1Le6w4xfstests-dev/results//generic/742.out.bad 2024-04-26 02:28:04.746584772 +0000 2024-04-26T02:28:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: @@ -1,2 +1,3 @@ 2024-04-26T02:28:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: QA output created by 742 2024-04-26T02:28:05.556 INFO:teuthology.orchestra.run.smithi094.stdout: Silence is golden 2024-04-26T02:28:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: +fiemap-fault: fiemap failed 95: Operation not supported 2024-04-26T02:28:05.557 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/742 2024-04-26T02:28:05.557 INFO:teuthology.orchestra.run.smithi094.stdout:Failures: generic/742 2024-04-26T02:28:05.557 INFO:teuthology.orchestra.run.smithi094.stdout:Failed 1 of 1 tests 2024-04-26T02:28:05.557 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:05.557 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-26T02:28:05.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-26T02:28:05.558 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/tmp.FooT1Le6w4xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/743) 2024-04-26T02:28:06.356 INFO:teuthology.orchestra.run.smithi094.stdout:FSTYP -- ceph-fuse 2024-04-26T02:28:06.362 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM -- Linux/x86_64 smithi094 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-26T02:28:06.362 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:06.656 INFO:teuthology.orchestra.run.smithi094.stdout:generic/743 [not run] xfs_io madvise doesn't support -R 2024-04-26T02:28:06.662 INFO:teuthology.orchestra.run.smithi094.stdout:Ran: generic/743 2024-04-26T02:28:06.662 INFO:teuthology.orchestra.run.smithi094.stdout:Not run: generic/743 2024-04-26T02:28:06.662 INFO:teuthology.orchestra.run.smithi094.stdout:Passed all 1 tests 2024-04-26T02:28:06.665 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:06.675 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-26T02:28:06.675 INFO:tasks.cephfs.xfstests_dev:=========================================================Total number of tests failed = 17========================================================= 2024-04-26T02:28:06.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo userdel --force --remove fsgqa 2024-04-26T02:28:06.779 INFO:teuthology.orchestra.run.smithi094.stderr:userdel: fsgqa mail spool (/var/mail/fsgqa) not found 2024-04-26T02:28:07.132 DEBUG:teuthology.orchestra.run.smithi094:> sudo userdel --force --remove 123456-fsgqa 2024-04-26T02:28:07.159 INFO:teuthology.orchestra.run.smithi094.stderr:userdel: 123456-fsgqa mail spool (/var/mail/123456-fsgqa) not found 2024-04-26T02:28:07.159 INFO:teuthology.orchestra.run.smithi094.stderr:userdel: 123456-fsgqa home directory (/home/123456-fsgqa) not found 2024-04-26T02:28:07.462 DEBUG:teuthology.orchestra.run.smithi094:> sudo groupdel fsgqa 2024-04-26T02:28:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:groupdel: group 'fsgqa' does not exist 2024-04-26T02:28:07.606 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-26T02:28:07.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:28:07.606 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo stat /tmp/tmp.FooT1Le6w4xfstests-dev/results) 2024-04-26T02:28:07.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:28:07.635 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: cd: /home/ubuntu/cephtest/mnt.0: Permission denied 2024-04-26T02:28:07.635 INFO:tasks.cephfs.xfstests_dev:xfstests-dev repo contains not directory named "results". repo location: /tmp/tmp.FooT1Le6w4xfstests-dev 2024-04-26T02:28:07.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf /tmp/tmp.FooT1Le6w4xfstests-dev 2024-04-26T02:28:07.931 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T02:28:07.931 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T02:28:08.060 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T02:28:08.391 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T02:28:08.391 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T02:28:08.391 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T02:28:08.391 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:28:08.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16115 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T02:28:08.429 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T02:28:08.429 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-26T02:28:08.429 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:28:08.441 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16120 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T02:28:08.441 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T02:28:08.441 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T02:28:08.441 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:28:08.478 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16115 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T02:28:08.478 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T02:28:08.478 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-26T02:28:08.478 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:28:08.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16120 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T02:28:08.493 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T02:28:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.258+0000 7fd6ed79a640 1 Processor -- start 2024-04-26T02:28:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.258+0000 7fd6ed79a640 1 -- start start 2024-04-26T02:28:09.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.262+0000 7fd6ed79a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd6e80d1c90 0x7fd6e80d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:09.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.262+0000 7fd6ed79a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd6e80d15b0 0x7fd6e80b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:09.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.262+0000 7fd6ed79a640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e81637a0 con 0x7fd6e80b5980 2024-04-26T02:28:09.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.262+0000 7fd6ed79a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd6e80d0be0 con 0x7fd6e80d15b0 2024-04-26T02:28:09.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.262+0000 7fd6ed79a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd6e806f120 con 0x7fd6e80d1c90 2024-04-26T02:28:09.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.262+0000 7fd6e77fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd6e80b5980 0x7fd6e8160430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:52416/0 (socket says 172.21.15.94:52416) 2024-04-26T02:28:09.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.262+0000 7fd6e77fe640 1 -- 172.21.15.94:0/2410672525 learned_addr learned my addr 172.21.15.94:0/2410672525 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:09.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.262+0000 7fd6e67fc640 1 --2- 172.21.15.94:0/2410672525 >> v2:172.21.15.94:3300/0 conn(0x7fd6e80d15b0 0x7fd6e80b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:09.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.262+0000 7fd6e67fc640 1 -- 172.21.15.94:0/2410672525 >> v2:172.21.15.138:3300/0 conn(0x7fd6e80d1c90 msgr2=0x7fd6e80d1070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:28:09.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.262+0000 7fd6e67fc640 1 --2- 172.21.15.94:0/2410672525 >> v2:172.21.15.138:3300/0 conn(0x7fd6e80d1c90 0x7fd6e80d1070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:09.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.262+0000 7fd6e67fc640 1 -- 172.21.15.94:0/2410672525 >> v1:172.21.15.94:6789/0 conn(0x7fd6e80b5980 legacy=0x7fd6e8160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:09.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.262+0000 7fd6e67fc640 1 -- 172.21.15.94:0/2410672525 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e806f400 con 0x7fd6e80d15b0 2024-04-26T02:28:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.266+0000 7fd6e67fc640 1 --2- 172.21.15.94:0/2410672525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d15b0 0x7fd6e80b5440 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80d0d60 tx=0x7fd6dc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4342b7cc98e296 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.266+0000 7fd6e5ffb640 1 -- 172.21.15.94:0/2410672525 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0b0070 con 0x7fd6e80d15b0 2024-04-26T02:28:09.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.266+0000 7fd6e5ffb640 1 -- 172.21.15.94:0/2410672525 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6dc0a4e50 con 0x7fd6e80d15b0 2024-04-26T02:28:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.266+0000 7fd6e5ffb640 1 -- 172.21.15.94:0/2410672525 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0ab820 con 0x7fd6e80d15b0 2024-04-26T02:28:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.266+0000 7fd6ed79a640 1 -- 172.21.15.94:0/2410672525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d15b0 msgr2=0x7fd6e80b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.266+0000 7fd6ed79a640 1 --2- 172.21.15.94:0/2410672525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d15b0 0x7fd6e80b5440 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80d0d60 tx=0x7fd6dc01c3f0 comp rx=0 tx=0).stop 2024-04-26T02:28:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.266+0000 7fd6ed79a640 1 -- 172.21.15.94:0/2410672525 shutdown_connections 2024-04-26T02:28:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.266+0000 7fd6ed79a640 1 --2- 172.21.15.94:0/2410672525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d15b0 0x7fd6e80b5440 secure :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80d0d60 tx=0x7fd6dc01c3f0 comp rx=0 tx=0).stop 2024-04-26T02:28:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.266+0000 7fd6ed79a640 1 --2- 172.21.15.94:0/2410672525 >> v2:172.21.15.138:3300/0 conn(0x7fd6e80d1c90 0x7fd6e80d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:09.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.266+0000 7fd6ed79a640 1 -- 172.21.15.94:0/2410672525 >> 172.21.15.94:0/2410672525 conn(0x7fd6e8071150 msgr2=0x7fd6e8071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:09.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.274+0000 7fd6ed79a640 1 -- 172.21.15.94:0/2410672525 shutdown_connections 2024-04-26T02:28:09.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.274+0000 7fd6ed79a640 1 -- 172.21.15.94:0/2410672525 wait complete. 2024-04-26T02:28:09.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.274+0000 7fd6ed79a640 1 Processor -- start 2024-04-26T02:28:09.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.274+0000 7fd6ed79a640 1 -- start start 2024-04-26T02:28:09.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6ed79a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6e80b5980 0x7fd6e8143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:09.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6ed79a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d1c90 0x7fd6e8143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:09.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6ed79a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6e808cb50 0x7fd6e814b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:09.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6ed79a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e80d04b0 con 0x7fd6e80d1c90 2024-04-26T02:28:09.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6ed79a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e80d20a0 con 0x7fd6e808cb50 2024-04-26T02:28:09.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6ed79a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e80d0630 con 0x7fd6e80b5980 2024-04-26T02:28:09.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6e6ffd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6e80b5980 0x7fd6e8143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:09.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6e77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6e808cb50 0x7fd6e814b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:09.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6e6ffd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6e80b5980 0x7fd6e8143720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:40488/0 (socket says 172.21.15.94:40488) 2024-04-26T02:28:09.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6e6ffd640 1 -- 172.21.15.94:0/1487182520 learned_addr learned my addr 172.21.15.94:0/1487182520 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:09.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6e6ffd640 1 -- 172.21.15.94:0/1487182520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6e808cb50 msgr2=0x7fd6e814b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:09.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6e6ffd640 1 --2- 172.21.15.94:0/1487182520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6e808cb50 0x7fd6e814b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:09.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6e6ffd640 1 -- 172.21.15.94:0/1487182520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d1c90 msgr2=0x7fd6e8143c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T02:28:09.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6e6ffd640 1 --2- 172.21.15.94:0/1487182520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d1c90 0x7fd6e8143c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:09.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6e6ffd640 1 -- 172.21.15.94:0/1487182520 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e808bab0 con 0x7fd6e80b5980 2024-04-26T02:28:09.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6e6ffd640 1 --2- 172.21.15.94:0/1487182520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6e80b5980 0x7fd6e8143720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800b410 tx=0x7fd6d800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:09.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6cf7fe640 1 -- 172.21.15.94:0/1487182520 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8015020 con 0x7fd6e80b5980 2024-04-26T02:28:09.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6cf7fe640 1 -- 172.21.15.94:0/1487182520 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d8011850 con 0x7fd6e80b5980 2024-04-26T02:28:09.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6cf7fe640 1 -- 172.21.15.94:0/1487182520 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8011c40 con 0x7fd6e80b5980 2024-04-26T02:28:09.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6ed79a640 1 -- 172.21.15.94:0/1487182520 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e808bd90 con 0x7fd6e80b5980 2024-04-26T02:28:09.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6ed79a640 1 -- 172.21.15.94:0/1487182520 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e808c350 con 0x7fd6e80b5980 2024-04-26T02:28:09.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.278+0000 7fd6ed79a640 1 -- 172.21.15.94:0/1487182520 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e806a730 con 0x7fd6e80b5980 2024-04-26T02:28:09.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.310+0000 7fd6cf7fe640 1 -- 172.21.15.94:0/1487182520 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd6d8018020 con 0x7fd6e80b5980 2024-04-26T02:28:09.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.310+0000 7fd6cf7fe640 1 --2- 172.21.15.94:0/1487182520 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd6d406f430 0x7fd6d4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:09.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.310+0000 7fd6cf7fe640 1 -- 172.21.15.94:0/1487182520 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fd6d808f4e0 con 0x7fd6e80b5980 2024-04-26T02:28:09.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.314+0000 7fd6e67fc640 1 --2- 172.21.15.94:0/1487182520 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd6d406f430 0x7fd6d4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:09.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.314+0000 7fd6e67fc640 1 --2- 172.21.15.94:0/1487182520 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd6d406f430 0x7fd6d4071940 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0ad000 tx=0x7fd6dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:09.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.406+0000 7fd6cf7fe640 1 -- 172.21.15.94:0/1487182520 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd6d801f070 con 0x7fd6e80b5980 2024-04-26T02:28:09.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.698+0000 7fd6ed79a640 1 -- 172.21.15.94:0/1487182520 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6e80b7580 con 0x7fd6e80b5980 2024-04-26T02:28:09.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.750+0000 7fd6cf7fe640 1 -- 172.21.15.94:0/1487182520 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5184 (secure 0 0 0) 0x7fd6d805c290 con 0x7fd6e80b5980 2024-04-26T02:28:09.760 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:09.760 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/478421533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":478421533},{"type":"v1","addr":"172.21.15.138:6833","nonce":478421533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2461866660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2461866660},{"type":"v1","addr":"172.21.15.138:6835","nonce":2461866660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5439,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2566493915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2566493915},{"type":"v1","addr":"172.21.15.94:6835","nonce":2566493915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:52:26.175197+0000","modified":"2024-04-25T16:52:29.233625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5442":{"gid":5442,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/2235039329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2235039329},{"type":"v1","addr":"172.21.15.94:6837","nonce":2235039329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5442,"qdb_cluster":[5442]},"id":2}]} 2024-04-26T02:28:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-04-26T02:28:09.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.758+0000 7fd6ed79a640 1 -- 172.21.15.94:0/1487182520 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd6d406f430 msgr2=0x7fd6d4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:09.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.758+0000 7fd6ed79a640 1 --2- 172.21.15.94:0/1487182520 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd6d406f430 0x7fd6d4071940 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0ad000 tx=0x7fd6dc008040 comp rx=0 tx=0).stop 2024-04-26T02:28:09.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.758+0000 7fd6ed79a640 1 -- 172.21.15.94:0/1487182520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6e80b5980 msgr2=0x7fd6e8143720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:09.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.758+0000 7fd6ed79a640 1 --2- 172.21.15.94:0/1487182520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6e80b5980 0x7fd6e8143720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800b410 tx=0x7fd6d800b8d0 comp rx=0 tx=0).stop 2024-04-26T02:28:09.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.758+0000 7fd6ed79a640 1 -- 172.21.15.94:0/1487182520 shutdown_connections 2024-04-26T02:28:09.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.758+0000 7fd6ed79a640 1 --2- 172.21.15.94:0/1487182520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6e808cb50 0x7fd6e814b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:09.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.758+0000 7fd6ed79a640 1 --2- 172.21.15.94:0/1487182520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6e80d1c90 0x7fd6e8143c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:09.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.758+0000 7fd6ed79a640 1 --2- 172.21.15.94:0/1487182520 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd6d406f430 0x7fd6d4071940 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:09.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.758+0000 7fd6ed79a640 1 --2- 172.21.15.94:0/1487182520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6e80b5980 0x7fd6e8143720 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:09.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.758+0000 7fd6ed79a640 1 -- 172.21.15.94:0/1487182520 >> 172.21.15.94:0/1487182520 conn(0x7fd6e8071150 msgr2=0x7fd6e80b6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:09.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.758+0000 7fd6ed79a640 1 -- 172.21.15.94:0/1487182520 shutdown_connections 2024-04-26T02:28:09.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:09.758+0000 7fd6ed79a640 1 -- 172.21.15.94:0/1487182520 wait complete. 2024-04-26T02:28:10.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T02:28:10.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.250+0000 7f271801e640 1 Processor -- start 2024-04-26T02:28:10.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.250+0000 7f271801e640 1 -- start start 2024-04-26T02:28:10.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.254+0000 7f271801e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f27100b5530 0x7f27100b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:10.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.254+0000 7f271801e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27100701f0 0x7f2710163eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:10.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.254+0000 7f271801e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27100b6600 con 0x7f27100b61f0 2024-04-26T02:28:10.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.254+0000 7f271801e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f271015dbe0 con 0x7f27100b5530 2024-04-26T02:28:10.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.254+0000 7f271801e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f27101605e0 con 0x7f27100701f0 2024-04-26T02:28:10.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.254+0000 7f27165bb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27100701f0 0x7f2710163eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:10.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.254+0000 7f27165bb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27100701f0 0x7f2710163eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:40498/0 (socket says 172.21.15.94:40498) 2024-04-26T02:28:10.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.254+0000 7f27165bb640 1 -- 172.21.15.94:0/4091339784 learned_addr learned my addr 172.21.15.94:0/4091339784 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:10.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.254+0000 7f2715dba640 1 --2- 172.21.15.94:0/4091339784 >> v2:172.21.15.138:3300/0 conn(0x7f27100b5530 0x7f27100b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:10.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.254+0000 7f27165bb640 1 -- 172.21.15.94:0/4091339784 >> v2:172.21.15.138:3300/0 conn(0x7f27100b5530 msgr2=0x7f27100b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:10.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.254+0000 7f27165bb640 1 --2- 172.21.15.94:0/4091339784 >> v2:172.21.15.138:3300/0 conn(0x7f27100b5530 0x7f27100b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:10.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.254+0000 7f27165bb640 1 -- 172.21.15.94:0/4091339784 >> v1:172.21.15.138:6789/0 conn(0x7f27100b61f0 legacy=0x7f27101637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:10.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.254+0000 7f27165bb640 1 -- 172.21.15.94:0/4091339784 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27101608c0 con 0x7f27100701f0 2024-04-26T02:28:10.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f27165bb640 1 --2- 172.21.15.94:0/4091339784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27100701f0 0x7f2710163eb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2708009e50 tx=0x7f270801c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a95293e0b38d5b44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:10.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f2714db8640 1 -- 172.21.15.94:0/4091339784 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27080a5750 con 0x7f27100701f0 2024-04-26T02:28:10.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f2714db8640 1 -- 172.21.15.94:0/4091339784 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27080a58f0 con 0x7f27100701f0 2024-04-26T02:28:10.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f2714db8640 1 -- 172.21.15.94:0/4091339784 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27080a5cc0 con 0x7f27100701f0 2024-04-26T02:28:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f271801e640 1 -- 172.21.15.94:0/4091339784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27100701f0 msgr2=0x7f2710163eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f271801e640 1 --2- 172.21.15.94:0/4091339784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27100701f0 0x7f2710163eb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2708009e50 tx=0x7f270801c9a0 comp rx=0 tx=0).stop 2024-04-26T02:28:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f271801e640 1 -- 172.21.15.94:0/4091339784 shutdown_connections 2024-04-26T02:28:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f271801e640 1 --2- 172.21.15.94:0/4091339784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27100701f0 0x7f2710163eb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f271801e640 1 --2- 172.21.15.94:0/4091339784 >> v2:172.21.15.138:3300/0 conn(0x7f27100b5530 0x7f27100b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f271801e640 1 -- 172.21.15.94:0/4091339784 >> 172.21.15.94:0/4091339784 conn(0x7f27100d1d30 msgr2=0x7f27100d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f271801e640 1 -- 172.21.15.94:0/4091339784 shutdown_connections 2024-04-26T02:28:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f271801e640 1 -- 172.21.15.94:0/4091339784 wait complete. 2024-04-26T02:28:10.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f271801e640 1 Processor -- start 2024-04-26T02:28:10.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.258+0000 7f271801e640 1 -- start start 2024-04-26T02:28:10.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f271801e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27100701f0 0x7f27100af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:10.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f271801e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27100b5530 0x7f2710161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:10.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f271801e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27100b61f0 0x7f2710159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:10.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f271801e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27101601b0 con 0x7f27100701f0 2024-04-26T02:28:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f271801e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27100b6a10 con 0x7f27100b5530 2024-04-26T02:28:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f271801e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2710160330 con 0x7f27100b61f0 2024-04-26T02:28:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f2715dba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27100701f0 0x7f27100af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f2715dba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27100701f0 0x7f27100af870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54290/0 (socket says 172.21.15.94:54290) 2024-04-26T02:28:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f2715dba640 1 -- 172.21.15.94:0/1355950776 learned_addr learned my addr 172.21.15.94:0/1355950776 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f27155b9640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27100b5530 0x7f2710161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f27165bb640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27100b61f0 0x7f2710159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f2715dba640 1 -- 172.21.15.94:0/1355950776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27100b61f0 msgr2=0x7f2710159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f2715dba640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27100b61f0 0x7f2710159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f2715dba640 1 -- 172.21.15.94:0/1355950776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27100b5530 msgr2=0x7f2710161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f2715dba640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27100b5530 0x7f2710161520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f2715dba640 1 -- 172.21.15.94:0/1355950776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2710158dc0 con 0x7f27100701f0 2024-04-26T02:28:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f2715dba640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27100701f0 0x7f27100af870 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f270c00e870 tx=0x7f270c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:10.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f27067fc640 1 -- 172.21.15.94:0/1355950776 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c017070 con 0x7f27100701f0 2024-04-26T02:28:10.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f27067fc640 1 -- 172.21.15.94:0/1355950776 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f270c00c940 con 0x7f27100701f0 2024-04-26T02:28:10.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f27067fc640 1 -- 172.21.15.94:0/1355950776 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c00ccf0 con 0x7f27100701f0 2024-04-26T02:28:10.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f271801e640 1 -- 172.21.15.94:0/1355950776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27100b0250 con 0x7f27100701f0 2024-04-26T02:28:10.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.262+0000 7f271801e640 1 -- 172.21.15.94:0/1355950776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27100b0810 con 0x7f27100701f0 2024-04-26T02:28:10.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.266+0000 7f271801e640 1 -- 172.21.15.94:0/1355950776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271006a730 con 0x7f27100701f0 2024-04-26T02:28:10.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.570+0000 7f27067fc640 1 -- 172.21.15.94:0/1355950776 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f270c01a070 con 0x7f27100701f0 2024-04-26T02:28:10.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.570+0000 7f27067fc640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f26f406f400 0x7f26f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:10.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.570+0000 7f27067fc640 1 -- 172.21.15.94:0/1355950776 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f270c08fa60 con 0x7f27100701f0 2024-04-26T02:28:10.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.570+0000 7f27155b9640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f26f406f400 0x7f26f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:10.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.570+0000 7f27155b9640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f26f406f400 0x7f26f4071910 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2710157cc0 tx=0x7f2700009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:10.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.666+0000 7f27067fc640 1 -- 172.21.15.94:0/1355950776 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f270c05c1c0 con 0x7f27100701f0 2024-04-26T02:28:10.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.894+0000 7f271801e640 1 -- 172.21.15.94:0/1355950776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f271015eec0 con 0x7f27100701f0 2024-04-26T02:28:10.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.898+0000 7f27067fc640 1 -- 172.21.15.94:0/1355950776 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17003 (secure 0 0 0) 0x7f270c0604c0 con 0x7f27100701f0 2024-04-26T02:28:11.034 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:11.035 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":40,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:52:35.869046+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:52:23.840281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T16:52:25.130347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T02:28:11.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.898+0000 7f271801e640 1 -- 172.21.15.94:0/1355950776 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f26f406f400 msgr2=0x7f26f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.898+0000 7f271801e640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f26f406f400 0x7f26f4071910 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2710157cc0 tx=0x7f2700009040 comp rx=0 tx=0).stop 2024-04-26T02:28:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.898+0000 7f271801e640 1 -- 172.21.15.94:0/1355950776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27100701f0 msgr2=0x7f27100af870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.898+0000 7f271801e640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27100701f0 0x7f27100af870 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f270c00e870 tx=0x7f270c00ec70 comp rx=0 tx=0).stop 2024-04-26T02:28:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.906+0000 7f271801e640 1 -- 172.21.15.94:0/1355950776 shutdown_connections 2024-04-26T02:28:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.906+0000 7f271801e640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f26f406f400 0x7f26f4071910 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.906+0000 7f271801e640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27100b61f0 0x7f2710159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.906+0000 7f271801e640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27100b5530 0x7f2710161520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.906+0000 7f271801e640 1 --2- 172.21.15.94:0/1355950776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27100701f0 0x7f27100af870 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.906+0000 7f271801e640 1 -- 172.21.15.94:0/1355950776 >> 172.21.15.94:0/1355950776 conn(0x7f27100d1d30 msgr2=0x7f2710068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.906+0000 7f271801e640 1 -- 172.21.15.94:0/1355950776 shutdown_connections 2024-04-26T02:28:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:10.906+0000 7f271801e640 1 -- 172.21.15.94:0/1355950776 wait complete. 2024-04-26T02:28:11.037 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:50:41.716256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:52:23.840281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:52:25.130347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T02:28:11.037 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T02:28:11.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.222+0000 7f4758a5d640 1 Processor -- start 2024-04-26T02:28:11.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.222+0000 7f4758a5d640 1 -- start start 2024-04-26T02:28:11.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.222+0000 7f4758a5d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f475415de30 0x7f4754157d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:11.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.222+0000 7f4758a5d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4754162250 con 0x7f475415f5e0 2024-04-26T02:28:11.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.222+0000 7f4758a5d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f475415ee20 con 0x7f47540701b0 2024-04-26T02:28:11.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.222+0000 7f4758a5d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f47541582b0 con 0x7f475415de30 2024-04-26T02:28:11.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.222+0000 7f4752d9d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f475415f5e0 0x7f475415f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:52430/0 (socket says 172.21.15.94:52430) 2024-04-26T02:28:11.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.222+0000 7f4752d9d640 1 -- 172.21.15.94:0/1003172679 learned_addr learned my addr 172.21.15.94:0/1003172679 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:11.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.222+0000 7f475259c640 1 --2- 172.21.15.94:0/1003172679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f475415de30 0x7f4754157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:11.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.222+0000 7f475259c640 1 -- 172.21.15.94:0/1003172679 >> v1:172.21.15.138:6789/0 conn(0x7f47540701b0 legacy=0x7f47540705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.222+0000 7f475259c640 1 -- 172.21.15.94:0/1003172679 >> v1:172.21.15.94:6789/0 conn(0x7f475415f5e0 legacy=0x7f475415f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.222+0000 7f475259c640 1 -- 172.21.15.94:0/1003172679 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475415f0e0 con 0x7f475415de30 2024-04-26T02:28:11.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.226+0000 7f475259c640 1 --2- 172.21.15.94:0/1003172679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f475415de30 0x7f4754157d70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47440097f0 tx=0x7f474401c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=266c2573f26dda3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:11.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.230+0000 7f475159a640 1 -- 172.21.15.94:0/1003172679 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47440b1070 con 0x7f475415de30 2024-04-26T02:28:11.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.230+0000 7f475159a640 1 -- 172.21.15.94:0/1003172679 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47440ac640 con 0x7f475415de30 2024-04-26T02:28:11.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.230+0000 7f475159a640 1 -- 172.21.15.94:0/1003172679 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47440aca10 con 0x7f475415de30 2024-04-26T02:28:11.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.234+0000 7f4758a5d640 1 -- 172.21.15.94:0/1003172679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f475415de30 msgr2=0x7f4754157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.234+0000 7f4758a5d640 1 --2- 172.21.15.94:0/1003172679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f475415de30 0x7f4754157d70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47440097f0 tx=0x7f474401c370 comp rx=0 tx=0).stop 2024-04-26T02:28:11.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.234+0000 7f4758a5d640 1 -- 172.21.15.94:0/1003172679 shutdown_connections 2024-04-26T02:28:11.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.234+0000 7f4758a5d640 1 --2- 172.21.15.94:0/1003172679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f475415de30 0x7f4754157d70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.234+0000 7f4758a5d640 1 -- 172.21.15.94:0/1003172679 >> 172.21.15.94:0/1003172679 conn(0x7f4754071f00 msgr2=0x7f4754073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.238+0000 7f4758a5d640 1 -- 172.21.15.94:0/1003172679 shutdown_connections 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.238+0000 7f4758a5d640 1 -- 172.21.15.94:0/1003172679 wait complete. 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.242+0000 7f4758a5d640 1 Processor -- start 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.242+0000 7f4758a5d640 1 -- start start 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.242+0000 7f4758a5d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47540701b0 0x7f47541524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.242+0000 7f4758a5d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f475415de30 0x7f475406fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.242+0000 7f4758a5d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475415f5e0 0x7f4754155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.242+0000 7f4758a5d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f475415eb40 con 0x7f475415f5e0 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.242+0000 7f4758a5d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f475415e860 con 0x7f47540701b0 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.242+0000 7f4758a5d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f475415e240 con 0x7f475415de30 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.242+0000 7f4752d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475415f5e0 0x7f4754155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.242+0000 7f4752d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475415f5e0 0x7f4754155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54302/0 (socket says 172.21.15.94:54302) 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.242+0000 7f4752d9d640 1 -- 172.21.15.94:0/3912369472 learned_addr learned my addr 172.21.15.94:0/3912369472 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.242+0000 7f4751d9b640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f475415de30 0x7f475406fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:11.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.246+0000 7f475259c640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47540701b0 0x7f47541524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.246+0000 7f4752d9d640 1 -- 172.21.15.94:0/3912369472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f475415de30 msgr2=0x7f475406fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.246+0000 7f4752d9d640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f475415de30 0x7f475406fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.246+0000 7f4752d9d640 1 -- 172.21.15.94:0/3912369472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47540701b0 msgr2=0x7f47541524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.246+0000 7f4752d9d640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47540701b0 0x7f47541524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.246+0000 7f4752d9d640 1 -- 172.21.15.94:0/3912369472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47541e7370 con 0x7f475415f5e0 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.250+0000 7f4752d9d640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475415f5e0 0x7f4754155230 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f474c00e580 tx=0x7f474c00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.250+0000 7f4742ffd640 1 -- 172.21.15.94:0/3912369472 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c016020 con 0x7f475415f5e0 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.250+0000 7f4742ffd640 1 -- 172.21.15.94:0/3912369472 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474c009500 con 0x7f475415f5e0 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.250+0000 7f4758a5d640 1 -- 172.21.15.94:0/3912369472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47541e7610 con 0x7f475415f5e0 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.250+0000 7f4742ffd640 1 -- 172.21.15.94:0/3912369472 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0098d0 con 0x7f475415f5e0 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.250+0000 7f4758a5d640 1 -- 172.21.15.94:0/3912369472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47540cc650 con 0x7f475415f5e0 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.254+0000 7f4758a5d640 1 -- 172.21.15.94:0/3912369472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f475406a730 con 0x7f475415f5e0 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.254+0000 7f4742ffd640 1 -- 172.21.15.94:0/3912369472 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f474c019020 con 0x7f475415f5e0 2024-04-26T02:28:11.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.254+0000 7f4742ffd640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f473406f400 0x7f4734071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.254+0000 7f4742ffd640 1 -- 172.21.15.94:0/3912369472 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f474c091fe0 con 0x7f475415f5e0 2024-04-26T02:28:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.258+0000 7f475259c640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f473406f400 0x7f4734071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.258+0000 7f475259c640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f473406f400 0x7f4734071910 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f47440ae000 tx=0x7f4744008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.262+0000 7f4742ffd640 1 -- 172.21.15.94:0/3912369472 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f474c020030 con 0x7f475415f5e0 2024-04-26T02:28:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.486+0000 7f4758a5d640 1 -- 172.21.15.94:0/3912369472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f47540b6610 con 0x7f475415f5e0 2024-04-26T02:28:11.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.718+0000 7f4742ffd640 1 -- 172.21.15.94:0/3912369472 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7f474c05e760 con 0x7f475415f5e0 2024-04-26T02:28:11.722 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-26T02:28:11.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.718+0000 7f4758a5d640 1 -- 172.21.15.94:0/3912369472 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f473406f400 msgr2=0x7f4734071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.718+0000 7f4758a5d640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f473406f400 0x7f4734071910 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f47440ae000 tx=0x7f4744008040 comp rx=0 tx=0).stop 2024-04-26T02:28:11.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.718+0000 7f4758a5d640 1 -- 172.21.15.94:0/3912369472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475415f5e0 msgr2=0x7f4754155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.718+0000 7f4758a5d640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475415f5e0 0x7f4754155230 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f474c00e580 tx=0x7f474c00ea40 comp rx=0 tx=0).stop 2024-04-26T02:28:11.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.718+0000 7f4758a5d640 1 -- 172.21.15.94:0/3912369472 shutdown_connections 2024-04-26T02:28:11.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.718+0000 7f4758a5d640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475415f5e0 0x7f4754155230 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.718+0000 7f4758a5d640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f473406f400 0x7f4734071910 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.718+0000 7f4758a5d640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f475415de30 0x7f475406fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.718+0000 7f4758a5d640 1 --2- 172.21.15.94:0/3912369472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47540701b0 0x7f47541524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.718+0000 7f4758a5d640 1 -- 172.21.15.94:0/3912369472 >> 172.21.15.94:0/3912369472 conn(0x7f4754071f00 msgr2=0x7f47540b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:11.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.718+0000 7f4758a5d640 1 -- 172.21.15.94:0/3912369472 shutdown_connections 2024-04-26T02:28:11.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.718+0000 7f4758a5d640 1 -- 172.21.15.94:0/3912369472 wait complete. 2024-04-26T02:28:11.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T02:28:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29c063d640 1 Processor -- start 2024-04-26T02:28:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29c063d640 1 -- start start 2024-04-26T02:28:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29c063d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29b8071350 0x7f29b8160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29c063d640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b80719c0 con 0x7f29b8070f40 2024-04-26T02:28:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29c063d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b815ceb0 con 0x7f29b806e5b0 2024-04-26T02:28:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29c063d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f29b8071760 con 0x7f29b8071350 2024-04-26T02:28:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29be3d9640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f29b8070f40 0x7f29b806dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:52444/0 (socket says 172.21.15.94:52444) 2024-04-26T02:28:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29be3d9640 1 -- 172.21.15.94:0/200610923 learned_addr learned my addr 172.21.15.94:0/200610923 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29bebda640 1 --2- 172.21.15.94:0/200610923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29b8071350 0x7f29b8160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29bd3d7640 1 -- 172.21.15.94:0/200610923 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1654250767 0 0) 0x7f29b815ceb0 con 0x7f29b806e5b0 2024-04-26T02:28:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29bd3d7640 1 -- 172.21.15.94:0/200610923 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29a4003530 con 0x7f29b806e5b0 2024-04-26T02:28:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29bd3d7640 1 -- 172.21.15.94:0/200610923 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3817942741 0 0) 0x7f29b80719c0 con 0x7f29b8070f40 2024-04-26T02:28:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29bd3d7640 1 -- 172.21.15.94:0/200610923 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29b815ceb0 con 0x7f29b8070f40 2024-04-26T02:28:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29bebda640 1 -- 172.21.15.94:0/200610923 >> v1:172.21.15.138:6789/0 conn(0x7f29b806e5b0 legacy=0x7f29b8160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29bebda640 1 -- 172.21.15.94:0/200610923 >> v1:172.21.15.94:6789/0 conn(0x7f29b8070f40 legacy=0x7f29b806dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.946+0000 7f29bebda640 1 -- 172.21.15.94:0/200610923 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b815d150 con 0x7f29b8071350 2024-04-26T02:28:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.950+0000 7f29bebda640 1 --2- 172.21.15.94:0/200610923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29b8071350 0x7f29b8160a10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009dc0 tx=0x7f29b001c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e230e8ae6941837 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.950+0000 7f29bd3d7640 1 -- 172.21.15.94:0/200610923 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00ac070 con 0x7f29b8071350 2024-04-26T02:28:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.950+0000 7f29bd3d7640 1 -- 172.21.15.94:0/200610923 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b00a6690 con 0x7f29b8071350 2024-04-26T02:28:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.950+0000 7f29bd3d7640 1 -- 172.21.15.94:0/200610923 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00a6a60 con 0x7f29b8071350 2024-04-26T02:28:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.950+0000 7f29c063d640 1 -- 172.21.15.94:0/200610923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29b8071350 msgr2=0x7f29b8160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.950+0000 7f29c063d640 1 --2- 172.21.15.94:0/200610923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29b8071350 0x7f29b8160a10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009dc0 tx=0x7f29b001c8c0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.950+0000 7f29c063d640 1 -- 172.21.15.94:0/200610923 shutdown_connections 2024-04-26T02:28:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.950+0000 7f29c063d640 1 --2- 172.21.15.94:0/200610923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29b8071350 0x7f29b8160a10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.950+0000 7f29c063d640 1 -- 172.21.15.94:0/200610923 >> 172.21.15.94:0/200610923 conn(0x7f29b8072fd0 msgr2=0x7f29b80733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.950+0000 7f29c063d640 1 -- 172.21.15.94:0/200610923 shutdown_connections 2024-04-26T02:28:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.950+0000 7f29c063d640 1 -- 172.21.15.94:0/200610923 wait complete. 2024-04-26T02:28:11.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.954+0000 7f29c063d640 1 Processor -- start 2024-04-26T02:28:11.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.954+0000 7f29c063d640 1 -- start start 2024-04-26T02:28:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.954+0000 7f29c063d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29b8070f40 0x7f29b8147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.954+0000 7f29c063d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29b8071350 0x7f29b8147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.954+0000 7f29c063d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29b81fcce0 0x7f29b81fd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.954+0000 7f29c063d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29b80703b0 con 0x7f29b8071350 2024-04-26T02:28:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.954+0000 7f29c063d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29b8070230 con 0x7f29b81fcce0 2024-04-26T02:28:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.954+0000 7f29bdbd8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29b8071350 0x7f29b8147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.954+0000 7f29c063d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f29b80b5360 con 0x7f29b8070f40 2024-04-26T02:28:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.954+0000 7f29bdbd8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29b8071350 0x7f29b8147780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54310/0 (socket says 172.21.15.94:54310) 2024-04-26T02:28:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.954+0000 7f29bdbd8640 1 -- 172.21.15.94:0/2906947000 learned_addr learned my addr 172.21.15.94:0/2906947000 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.954+0000 7f29bebda640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29b81fcce0 0x7f29b81fd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29be3d9640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29b8070f40 0x7f29b8147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29bebda640 1 -- 172.21.15.94:0/2906947000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29b8070f40 msgr2=0x7f29b8147240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29bebda640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29b8070f40 0x7f29b8147240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29bebda640 1 -- 172.21.15.94:0/2906947000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29b8071350 msgr2=0x7f29b8147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29bebda640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29b8071350 0x7f29b8147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29bebda640 1 -- 172.21.15.94:0/2906947000 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b816b160 con 0x7f29b81fcce0 2024-04-26T02:28:11.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29bebda640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29b81fcce0 0x7f29b81fd0f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f29b815d080 tx=0x7f29b0019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:11.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29aeffd640 1 -- 172.21.15.94:0/2906947000 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00bf070 con 0x7f29b81fcce0 2024-04-26T02:28:11.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29aeffd640 1 -- 172.21.15.94:0/2906947000 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b00a6490 con 0x7f29b81fcce0 2024-04-26T02:28:11.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29aeffd640 1 -- 172.21.15.94:0/2906947000 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00ac040 con 0x7f29b81fcce0 2024-04-26T02:28:11.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29c063d640 1 -- 172.21.15.94:0/2906947000 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b8148130 con 0x7f29b81fcce0 2024-04-26T02:28:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29c063d640 1 -- 172.21.15.94:0/2906947000 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b816b720 con 0x7f29b81fcce0 2024-04-26T02:28:11.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29c063d640 1 -- 172.21.15.94:0/2906947000 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b806a730 con 0x7f29b81fcce0 2024-04-26T02:28:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29aeffd640 1 -- 172.21.15.94:0/2906947000 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f29b00c2070 con 0x7f29b81fcce0 2024-04-26T02:28:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29aeffd640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f299c06f400 0x7f299c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29aeffd640 1 -- 172.21.15.94:0/2906947000 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f29b0129500 con 0x7f29b81fcce0 2024-04-26T02:28:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.958+0000 7f29be3d9640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f299c06f400 0x7f299c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:11.962+0000 7f29be3d9640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f299c06f400 0x7f299c071910 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f29b4000f70 tx=0x7f29b4007dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.122+0000 7f29aeffd640 1 -- 172.21.15.94:0/2906947000 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f29b0016540 con 0x7f29b81fcce0 2024-04-26T02:28:12.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.334+0000 7f29c063d640 1 -- 172.21.15.94:0/2906947000 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f29b814c9a0 con 0x7f29b81fcce0 2024-04-26T02:28:12.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.374+0000 7f29aeffd640 1 -- 172.21.15.94:0/2906947000 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7f29b00f5df0 con 0x7f29b81fcce0 2024-04-26T02:28:12.379 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-26T02:28:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.378+0000 7f29c063d640 1 -- 172.21.15.94:0/2906947000 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f299c06f400 msgr2=0x7f299c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.378+0000 7f29c063d640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f299c06f400 0x7f299c071910 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f29b4000f70 tx=0x7f29b4007dd0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.378+0000 7f29c063d640 1 -- 172.21.15.94:0/2906947000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29b81fcce0 msgr2=0x7f29b81fd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.378+0000 7f29c063d640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29b81fcce0 0x7f29b81fd0f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f29b815d080 tx=0x7f29b0019660 comp rx=0 tx=0).stop 2024-04-26T02:28:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.378+0000 7f29c063d640 1 -- 172.21.15.94:0/2906947000 shutdown_connections 2024-04-26T02:28:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.378+0000 7f29c063d640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29b81fcce0 0x7f29b81fd0f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.378+0000 7f29c063d640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29b8071350 0x7f29b8147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.378+0000 7f29c063d640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f299c06f400 0x7f299c071910 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.378+0000 7f29c063d640 1 --2- 172.21.15.94:0/2906947000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29b8070f40 0x7f29b8147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.378+0000 7f29c063d640 1 -- 172.21.15.94:0/2906947000 >> 172.21.15.94:0/2906947000 conn(0x7f29b8072fd0 msgr2=0x7f29b8160290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.378+0000 7f29c063d640 1 -- 172.21.15.94:0/2906947000 shutdown_connections 2024-04-26T02:28:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.378+0000 7f29c063d640 1 -- 172.21.15.94:0/2906947000 wait complete. 2024-04-26T02:28:12.414 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T02:28:12.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a083b2640 1 Processor -- start 2024-04-26T02:28:12.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a083b2640 1 -- start start 2024-04-26T02:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a083b2640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0a0006e5b0 0x7f0a0006f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a083b2640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0a0006f570 0x7f0a0006f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a083b2640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a0006fec0 con 0x7f0a000b4a50 2024-04-26T02:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a083b2640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f0a001610c0 con 0x7f0a0006e5b0 2024-04-26T02:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a083b2640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0a000b5030 con 0x7f0a0006f570 2024-04-26T02:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a0614e640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0a000b4a50 0x7f0a0006dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:33006/0 (socket says 172.21.15.94:33006) 2024-04-26T02:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a0614e640 1 -- 172.21.15.94:0/4208863939 learned_addr learned my addr 172.21.15.94:0/4208863939 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:12.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a0694f640 1 --2- 172.21.15.94:0/4208863939 >> v2:172.21.15.138:3300/0 conn(0x7f0a0006f570 0x7f0a0006f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a0694f640 1 -- 172.21.15.94:0/4208863939 >> v2:172.21.15.94:3300/0 conn(0x7f0a0006e5b0 msgr2=0x7f0a0006f030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a0694f640 1 --2- 172.21.15.94:0/4208863939 >> v2:172.21.15.94:3300/0 conn(0x7f0a0006e5b0 0x7f0a0006f030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a0694f640 1 -- 172.21.15.94:0/4208863939 >> v1:172.21.15.138:6789/0 conn(0x7f0a000b4a50 legacy=0x7f0a0006dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.590+0000 7f0a0694f640 1 -- 172.21.15.94:0/4208863939 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a000b5310 con 0x7f0a0006f570 2024-04-26T02:28:12.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.594+0000 7f0a0694f640 1 --2- 172.21.15.94:0/4208863939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a0006f570 0x7f0a0006f980 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0a0006ce20 tx=0x7f09f801c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=409897aac1b95fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.594+0000 7f0a0514c640 1 -- 172.21.15.94:0/4208863939 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f80b0600 con 0x7f0a0006f570 2024-04-26T02:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.594+0000 7f0a0514c640 1 -- 172.21.15.94:0/4208863939 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f80b07a0 con 0x7f0a0006f570 2024-04-26T02:28:12.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.594+0000 7f0a0514c640 1 -- 172.21.15.94:0/4208863939 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f80b0b70 con 0x7f0a0006f570 2024-04-26T02:28:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.602+0000 7f0a083b2640 1 -- 172.21.15.94:0/4208863939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a0006f570 msgr2=0x7f0a0006f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:12.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.602+0000 7f0a083b2640 1 --2- 172.21.15.94:0/4208863939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a0006f570 0x7f0a0006f980 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0a0006ce20 tx=0x7f09f801c750 comp rx=0 tx=0).stop 2024-04-26T02:28:12.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.602+0000 7f0a083b2640 1 -- 172.21.15.94:0/4208863939 shutdown_connections 2024-04-26T02:28:12.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.602+0000 7f0a083b2640 1 --2- 172.21.15.94:0/4208863939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a0006f570 0x7f0a0006f980 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.602+0000 7f0a083b2640 1 --2- 172.21.15.94:0/4208863939 >> v2:172.21.15.94:3300/0 conn(0x7f0a0006e5b0 0x7f0a0006f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.602+0000 7f0a083b2640 1 -- 172.21.15.94:0/4208863939 >> 172.21.15.94:0/4208863939 conn(0x7f0a00072fd0 msgr2=0x7f0a000733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:12.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.602+0000 7f0a083b2640 1 -- 172.21.15.94:0/4208863939 shutdown_connections 2024-04-26T02:28:12.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.602+0000 7f0a083b2640 1 -- 172.21.15.94:0/4208863939 wait complete. 2024-04-26T02:28:12.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.606+0000 7f0a083b2640 1 Processor -- start 2024-04-26T02:28:12.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.606+0000 7f0a083b2640 1 -- start start 2024-04-26T02:28:12.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a083b2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a0006f570 0x7f0a000af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:12.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a083b2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a000b4a50 0x7f0a000b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:12.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a0614e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a0006f570 0x7f0a000af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:12.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a0614e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a0006f570 0x7f0a000af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:40560/0 (socket says 172.21.15.94:40560) 2024-04-26T02:28:12.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a0594d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a000b4a50 0x7f0a000b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:12.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a0594d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a000b4a50 0x7f0a000b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54312/0 (socket says 172.21.15.94:54312) 2024-04-26T02:28:12.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a0614e640 1 -- 172.21.15.94:0/792575746 learned_addr learned my addr 172.21.15.94:0/792575746 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a083b2640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a00140630 0x7f0a000d0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a083b2640 1 -- 172.21.15.94:0/792575746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a00160a40 con 0x7f0a000b4a50 2024-04-26T02:28:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a083b2640 1 -- 172.21.15.94:0/792575746 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001608c0 con 0x7f0a00140630 2024-04-26T02:28:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a083b2640 1 -- 172.21.15.94:0/792575746 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0a00160bc0 con 0x7f0a0006f570 2024-04-26T02:28:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a0694f640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a00140630 0x7f0a000d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a0614e640 1 -- 172.21.15.94:0/792575746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a00140630 msgr2=0x7f0a000d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a0614e640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a00140630 0x7f0a000d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a0614e640 1 -- 172.21.15.94:0/792575746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a000b4a50 msgr2=0x7f0a000b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a0614e640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a000b4a50 0x7f0a000b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a0614e640 1 -- 172.21.15.94:0/792575746 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0013f870 con 0x7f0a0006f570 2024-04-26T02:28:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a0614e640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a0006f570 0x7f0a000af680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00c5d0 tx=0x7f09fc00ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f09f6ffd640 1 -- 172.21.15.94:0/792575746 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc018070 con 0x7f0a0006f570 2024-04-26T02:28:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f09f6ffd640 1 -- 172.21.15.94:0/792575746 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09fc0074e0 con 0x7f0a0006f570 2024-04-26T02:28:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f09f6ffd640 1 -- 172.21.15.94:0/792575746 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc004a30 con 0x7f0a0006f570 2024-04-26T02:28:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a083b2640 1 -- 172.21.15.94:0/792575746 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0013fb50 con 0x7f0a0006f570 2024-04-26T02:28:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.610+0000 7f0a083b2640 1 -- 172.21.15.94:0/792575746 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a00140110 con 0x7f0a0006f570 2024-04-26T02:28:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.614+0000 7f09f6ffd640 1 -- 172.21.15.94:0/792575746 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f09fc004bd0 con 0x7f0a0006f570 2024-04-26T02:28:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.614+0000 7f09f6ffd640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f09e406f400 0x7f09e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:12.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.614+0000 7f09f6ffd640 1 -- 172.21.15.94:0/792575746 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f09fc08e870 con 0x7f0a0006f570 2024-04-26T02:28:12.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.614+0000 7f0a0594d640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f09e406f400 0x7f09e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:12.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.614+0000 7f0a083b2640 1 -- 172.21.15.94:0/792575746 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a0006a730 con 0x7f0a0006f570 2024-04-26T02:28:12.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.614+0000 7f0a0594d640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f09e406f400 0x7f09e4071910 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0a000b2f20 tx=0x7f09f0009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:12.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.630+0000 7f09f6ffd640 1 -- 172.21.15.94:0/792575746 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f09fc05af50 con 0x7f0a0006f570 2024-04-26T02:28:12.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.858+0000 7f0a083b2640 1 -- 172.21.15.94:0/792575746 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0a000d1820 con 0x7f0a0006f570 2024-04-26T02:28:12.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.858+0000 7f09f6ffd640 1 -- 172.21.15.94:0/792575746 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7f09fc05f250 con 0x7f0a0006f570 2024-04-26T02:28:12.864 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-26T02:28:12.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.862+0000 7f0a083b2640 1 -- 172.21.15.94:0/792575746 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f09e406f400 msgr2=0x7f09e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:12.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.862+0000 7f0a083b2640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f09e406f400 0x7f09e4071910 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0a000b2f20 tx=0x7f09f0009060 comp rx=0 tx=0).stop 2024-04-26T02:28:12.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.862+0000 7f0a083b2640 1 -- 172.21.15.94:0/792575746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a0006f570 msgr2=0x7f0a000af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:12.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.862+0000 7f0a083b2640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a0006f570 0x7f0a000af680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00c5d0 tx=0x7f09fc00ca90 comp rx=0 tx=0).stop 2024-04-26T02:28:12.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.866+0000 7f0a083b2640 1 -- 172.21.15.94:0/792575746 shutdown_connections 2024-04-26T02:28:12.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.866+0000 7f0a083b2640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a00140630 0x7f0a000d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.866+0000 7f0a083b2640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a000b4a50 0x7f0a000b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.866+0000 7f0a083b2640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f09e406f400 0x7f09e4071910 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.866+0000 7f0a083b2640 1 --2- 172.21.15.94:0/792575746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a0006f570 0x7f0a000af680 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:12.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.866+0000 7f0a083b2640 1 -- 172.21.15.94:0/792575746 >> 172.21.15.94:0/792575746 conn(0x7f0a00072fd0 msgr2=0x7f0a000ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:12.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.866+0000 7f0a083b2640 1 -- 172.21.15.94:0/792575746 shutdown_connections 2024-04-26T02:28:12.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:12.866+0000 7f0a083b2640 1 -- 172.21.15.94:0/792575746 wait complete. 2024-04-26T02:28:12.908 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T02:28:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.098+0000 7f45706b5640 1 Processor -- start 2024-04-26T02:28:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.098+0000 7f45706b5640 1 -- start start 2024-04-26T02:28:13.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.102+0000 7f45706b5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f45680d1c90 0x7f45680d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.102+0000 7f45706b5640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4568160430 0x7f4568160840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.102+0000 7f45706b5640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4568160d80 con 0x7f45680d15b0 2024-04-26T02:28:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.102+0000 7f45706b5640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f45680d0be0 con 0x7f4568160430 2024-04-26T02:28:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.102+0000 7f45706b5640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f456806f120 con 0x7f45680d1c90 2024-04-26T02:28:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.102+0000 7f456e451640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f45680d1c90 0x7f45680d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.102+0000 7f456dc50640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f45680d15b0 0x7f45680b5440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:33022/0 (socket says 172.21.15.94:33022) 2024-04-26T02:28:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.102+0000 7f456dc50640 1 -- 172.21.15.94:0/1588823782 learned_addr learned my addr 172.21.15.94:0/1588823782 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.102+0000 7f456ec52640 1 --2- 172.21.15.94:0/1588823782 >> v2:172.21.15.94:3300/0 conn(0x7f4568160430 0x7f4568160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.102+0000 7f456ec52640 1 -- 172.21.15.94:0/1588823782 >> v2:172.21.15.138:3300/0 conn(0x7f45680d1c90 msgr2=0x7f45680d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.102+0000 7f456ec52640 1 --2- 172.21.15.94:0/1588823782 >> v2:172.21.15.138:3300/0 conn(0x7f45680d1c90 0x7f45680d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.102+0000 7f456ec52640 1 -- 172.21.15.94:0/1588823782 >> v1:172.21.15.138:6789/0 conn(0x7f45680d15b0 legacy=0x7f45680b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.102+0000 7f456ec52640 1 -- 172.21.15.94:0/1588823782 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456806f400 con 0x7f4568160430 2024-04-26T02:28:13.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.106+0000 7f456ec52640 1 --2- 172.21.15.94:0/1588823782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4568160430 0x7f4568160840 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f45680733a0 tx=0x7f456001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda2246409816662 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:13.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.110+0000 7f456d44f640 1 -- 172.21.15.94:0/1588823782 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45600b2070 con 0x7f4568160430 2024-04-26T02:28:13.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.110+0000 7f456d44f640 1 -- 172.21.15.94:0/1588823782 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45600a5810 con 0x7f4568160430 2024-04-26T02:28:13.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.110+0000 7f456d44f640 1 -- 172.21.15.94:0/1588823782 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45600a5be0 con 0x7f4568160430 2024-04-26T02:28:13.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.110+0000 7f45706b5640 1 -- 172.21.15.94:0/1588823782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4568160430 msgr2=0x7f4568160840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:13.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.110+0000 7f45706b5640 1 --2- 172.21.15.94:0/1588823782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4568160430 0x7f4568160840 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f45680733a0 tx=0x7f456001c870 comp rx=0 tx=0).stop 2024-04-26T02:28:13.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.118+0000 7f45706b5640 1 -- 172.21.15.94:0/1588823782 shutdown_connections 2024-04-26T02:28:13.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.118+0000 7f45706b5640 1 --2- 172.21.15.94:0/1588823782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4568160430 0x7f4568160840 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.118+0000 7f45706b5640 1 --2- 172.21.15.94:0/1588823782 >> v2:172.21.15.138:3300/0 conn(0x7f45680d1c90 0x7f45680d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.118+0000 7f45706b5640 1 -- 172.21.15.94:0/1588823782 >> 172.21.15.94:0/1588823782 conn(0x7f4568071150 msgr2=0x7f4568071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:13.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.126+0000 7f45706b5640 1 -- 172.21.15.94:0/1588823782 shutdown_connections 2024-04-26T02:28:13.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.126+0000 7f45706b5640 1 -- 172.21.15.94:0/1588823782 wait complete. 2024-04-26T02:28:13.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.126+0000 7f45706b5640 1 Processor -- start 2024-04-26T02:28:13.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.126+0000 7f45706b5640 1 -- start start 2024-04-26T02:28:13.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f45706b5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45680d15b0 0x7f4568148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f45706b5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45680d1c90 0x7f45680b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f45706b5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4568160430 0x7f456815f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f45706b5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f45680d04b0 con 0x7f4568160430 2024-04-26T02:28:13.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f45706b5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45680d20a0 con 0x7f45680d1c90 2024-04-26T02:28:13.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f45706b5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f45680d0630 con 0x7f45680d15b0 2024-04-26T02:28:13.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f456ec52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4568160430 0x7f456815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f456ec52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4568160430 0x7f456815f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54334/0 (socket says 172.21.15.94:54334) 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f456ec52640 1 -- 172.21.15.94:0/2953136623 learned_addr learned my addr 172.21.15.94:0/2953136623 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f456dc50640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45680d1c90 0x7f45680b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f456e451640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45680d15b0 0x7f4568148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f456ec52640 1 -- 172.21.15.94:0/2953136623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45680d15b0 msgr2=0x7f4568148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f456ec52640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45680d15b0 0x7f4568148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f456ec52640 1 -- 172.21.15.94:0/2953136623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45680d1c90 msgr2=0x7f45680b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f456ec52640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45680d1c90 0x7f45680b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f456ec52640 1 -- 172.21.15.94:0/2953136623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45680b5bd0 con 0x7f4568160430 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f456ec52640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4568160430 0x7f456815f360 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f4560019480 tx=0x7f4560019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f455effd640 1 -- 172.21.15.94:0/2953136623 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45600b2050 con 0x7f4568160430 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f455effd640 1 -- 172.21.15.94:0/2953136623 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45600be070 con 0x7f4568160430 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f45706b5640 1 -- 172.21.15.94:0/2953136623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45680b60e0 con 0x7f4568160430 2024-04-26T02:28:13.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f455effd640 1 -- 172.21.15.94:0/2953136623 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45600ad890 con 0x7f4568160430 2024-04-26T02:28:13.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f45706b5640 1 -- 172.21.15.94:0/2953136623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456814ec90 con 0x7f4568160430 2024-04-26T02:28:13.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.130+0000 7f455cff9640 1 -- 172.21.15.94:0/2953136623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45440052a0 con 0x7f4568160430 2024-04-26T02:28:13.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.134+0000 7f455effd640 1 -- 172.21.15.94:0/2953136623 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f45600ada30 con 0x7f4568160430 2024-04-26T02:28:13.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.134+0000 7f455effd640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f454c06f430 0x7f454c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.134+0000 7f455effd640 1 -- 172.21.15.94:0/2953136623 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f456012a4a0 con 0x7f4568160430 2024-04-26T02:28:13.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.134+0000 7f456e451640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f454c06f430 0x7f454c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:13.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.134+0000 7f456e451640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f454c06f430 0x7f454c071940 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f456806c780 tx=0x7f455800b3a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:13.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.142+0000 7f455effd640 1 -- 172.21.15.94:0/2953136623 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4560016520 con 0x7f4568160430 2024-04-26T02:28:13.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.414+0000 7f455cff9640 1 -- 172.21.15.94:0/2953136623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4544005590 con 0x7f4568160430 2024-04-26T02:28:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.446+0000 7f455effd640 1 -- 172.21.15.94:0/2953136623 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5184 (secure 0 0 0) 0x7f45600f6af0 con 0x7f4568160430 2024-04-26T02:28:13.452 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:13.452 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/478421533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":478421533},{"type":"v1","addr":"172.21.15.138:6833","nonce":478421533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2461866660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2461866660},{"type":"v1","addr":"172.21.15.138:6835","nonce":2461866660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5439,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2566493915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2566493915},{"type":"v1","addr":"172.21.15.94:6835","nonce":2566493915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:52:26.175197+0000","modified":"2024-04-25T16:52:29.233625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5442":{"gid":5442,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/2235039329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2235039329},{"type":"v1","addr":"172.21.15.94:6837","nonce":2235039329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5442,"qdb_cluster":[5442]},"id":2}]} 2024-04-26T02:28:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-04-26T02:28:13.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.450+0000 7f455cff9640 1 -- 172.21.15.94:0/2953136623 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f454c06f430 msgr2=0x7f454c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:13.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.450+0000 7f455cff9640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f454c06f430 0x7f454c071940 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f456806c780 tx=0x7f455800b3a0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.450+0000 7f455cff9640 1 -- 172.21.15.94:0/2953136623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4568160430 msgr2=0x7f456815f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:13.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.450+0000 7f455cff9640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4568160430 0x7f456815f360 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f4560019480 tx=0x7f4560019610 comp rx=0 tx=0).stop 2024-04-26T02:28:13.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.450+0000 7f455cff9640 1 -- 172.21.15.94:0/2953136623 shutdown_connections 2024-04-26T02:28:13.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.450+0000 7f455cff9640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4568160430 0x7f456815f360 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.450+0000 7f455cff9640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45680d1c90 0x7f45680b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.450+0000 7f455cff9640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f454c06f430 0x7f454c071940 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.450+0000 7f455cff9640 1 --2- 172.21.15.94:0/2953136623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45680d15b0 0x7f4568148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.450+0000 7f455cff9640 1 -- 172.21.15.94:0/2953136623 >> 172.21.15.94:0/2953136623 conn(0x7f4568071150 msgr2=0x7f456806f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:13.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.450+0000 7f455cff9640 1 -- 172.21.15.94:0/2953136623 shutdown_connections 2024-04-26T02:28:13.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.450+0000 7f455cff9640 1 -- 172.21.15.94:0/2953136623 wait complete. 2024-04-26T02:28:13.487 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T02:28:13.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.658+0000 7f567cb0e640 1 Processor -- start 2024-04-26T02:28:13.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.658+0000 7f567cb0e640 1 -- start start 2024-04-26T02:28:13.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567cb0e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f567815e8f0 0x7f56780d2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567cb0e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56781620a0 0x7f567815f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567cb0e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56780d2980 con 0x7f56780d2570 2024-04-26T02:28:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567cb0e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f567815dea0 con 0x7f567815e8f0 2024-04-26T02:28:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567cb0e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f567815e460 con 0x7f56781620a0 2024-04-26T02:28:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f5675d9b640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f56780d2570 0x7f5678161990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:33038/0 (socket says 172.21.15.94:33038) 2024-04-26T02:28:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f5675d9b640 1 -- 172.21.15.94:0/2163742031 learned_addr learned my addr 172.21.15.94:0/2163742031 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3459446774 0 0) 0x7f566c001120 con 0x7f56780d2570 2024-04-26T02:28:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5660003530 con 0x7f56780d2570 2024-04-26T02:28:13.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3934561664 0 0) 0x7f5660003530 con 0x7f56780d2570 2024-04-26T02:28:13.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56781620a0 msgr2=0x7f567815f060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:28:13.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 --2- 172.21.15.94:0/2163742031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56781620a0 0x7f567815f060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 >> v2:172.21.15.138:3300/0 conn(0x7f567815e8f0 msgr2=0x7f56780d2030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:28:13.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 --2- 172.21.15.94:0/2163742031 >> v2:172.21.15.138:3300/0 conn(0x7f567815e8f0 0x7f56780d2030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56780b7c60 con 0x7f56780d2570 2024-04-26T02:28:13.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f566c001850 con 0x7f56780d2570 2024-04-26T02:28:13.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 >> v1:172.21.15.138:6789/0 conn(0x7f56780d2570 legacy=0x7f5678161990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:13.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 --2- 172.21.15.94:0/2163742031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56600039c0 0x7f5660003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 --2- 172.21.15.94:0/2163742031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5660004810 0x7f5660004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 --2- 172.21.15.94:0/2163742031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56600052b0 0x7f566000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f567815dea0 con 0x7f56600039c0 2024-04-26T02:28:13.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f567815e460 con 0x7f56600052b0 2024-04-26T02:28:13.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f566000c280 con 0x7f5660004810 2024-04-26T02:28:13.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f5675d9b640 1 --2- 172.21.15.94:0/2163742031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5660004810 0x7f5660004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:13.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f5675d9b640 1 -- 172.21.15.94:0/2163742031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56600052b0 msgr2=0x7f566000b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:28:13.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f5675d9b640 1 --2- 172.21.15.94:0/2163742031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56600052b0 0x7f566000b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f5675d9b640 1 -- 172.21.15.94:0/2163742031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56600039c0 msgr2=0x7f5660003e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:28:13.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f5675d9b640 1 --2- 172.21.15.94:0/2163742031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56600039c0 0x7f5660003e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.662+0000 7f5675d9b640 1 -- 172.21.15.94:0/2163742031 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5660003530 con 0x7f5660004810 2024-04-26T02:28:13.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.666+0000 7f5675d9b640 1 --2- 172.21.15.94:0/2163742031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5660004810 0x7f5660004c20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c410 tx=0x7f566c01f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e2f22521831ed07 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:13.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.666+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c00c0e0 con 0x7f5660004810 2024-04-26T02:28:13.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.666+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f566c005b00 con 0x7f5660004810 2024-04-26T02:28:13.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.666+0000 7f567559a640 1 -- 172.21.15.94:0/2163742031 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c007070 con 0x7f5660004810 2024-04-26T02:28:13.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.666+0000 7f567cb0e640 1 -- 172.21.15.94:0/2163742031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5660004810 msgr2=0x7f5660004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:13.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.666+0000 7f567cb0e640 1 --2- 172.21.15.94:0/2163742031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5660004810 0x7f5660004c20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c410 tx=0x7f566c01f460 comp rx=0 tx=0).stop 2024-04-26T02:28:13.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.670+0000 7f5676d9d640 1 -- 172.21.15.94:0/2163742031 reap_dead start 2024-04-26T02:28:13.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.670+0000 7f567cb0e640 1 -- 172.21.15.94:0/2163742031 shutdown_connections 2024-04-26T02:28:13.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.670+0000 7f567cb0e640 1 -- 172.21.15.94:0/2163742031 >> 172.21.15.94:0/2163742031 conn(0x7f56780cfed0 msgr2=0x7f56780d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:13.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.670+0000 7f567cb0e640 1 -- 172.21.15.94:0/2163742031 shutdown_connections 2024-04-26T02:28:13.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.670+0000 7f567cb0e640 1 -- 172.21.15.94:0/2163742031 wait complete. 2024-04-26T02:28:13.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.674+0000 7f567cb0e640 1 Processor -- start 2024-04-26T02:28:13.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.674+0000 7f567cb0e640 1 -- start start 2024-04-26T02:28:13.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567cb0e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5678152450 0x7f56781504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567cb0e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56781499d0 0x7f56780c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567cb0e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56780c9530 0x7f56780c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567cb0e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56780b7a00 con 0x7f5678152450 2024-04-26T02:28:13.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567cb0e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56780b8220 con 0x7f56780c9530 2024-04-26T02:28:13.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567cb0e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f567815e0f0 con 0x7f56781499d0 2024-04-26T02:28:13.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5678152450 0x7f56781504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:13.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5678152450 0x7f56781504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54344/0 (socket says 172.21.15.94:54344) 2024-04-26T02:28:13.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567659c640 1 -- 172.21.15.94:0/1937494246 learned_addr learned my addr 172.21.15.94:0/1937494246 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:13.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f5675d9b640 1 --2- 172.21.15.94:0/1937494246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56781499d0 0x7f56780c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:13.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567659c640 1 -- 172.21.15.94:0/1937494246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56781499d0 msgr2=0x7f56780c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:13.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567659c640 1 --2- 172.21.15.94:0/1937494246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56781499d0 0x7f56780c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567659c640 1 -- 172.21.15.94:0/1937494246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56780c9530 msgr2=0x7f56780c9940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T02:28:13.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567659c640 1 --2- 172.21.15.94:0/1937494246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56780c9530 0x7f56780c9940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567659c640 1 -- 172.21.15.94:0/1937494246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567814e7c0 con 0x7f5678152450 2024-04-26T02:28:13.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567659c640 1 --2- 172.21.15.94:0/1937494246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5678152450 0x7f56781504b0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f566800b410 tx=0x7f566800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:13.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f5666ffd640 1 -- 172.21.15.94:0/1937494246 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668015020 con 0x7f5678152450 2024-04-26T02:28:13.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f5666ffd640 1 -- 172.21.15.94:0/1937494246 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5668011830 con 0x7f5678152450 2024-04-26T02:28:13.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f5666ffd640 1 -- 172.21.15.94:0/1937494246 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668011ca0 con 0x7f5678152450 2024-04-26T02:28:13.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567cb0e640 1 -- 172.21.15.94:0/1937494246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56780c6de0 con 0x7f5678152450 2024-04-26T02:28:13.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567cb0e640 1 -- 172.21.15.94:0/1937494246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56780c8ad0 con 0x7f5678152450 2024-04-26T02:28:13.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f567cb0e640 1 -- 172.21.15.94:0/1937494246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f567806a730 con 0x7f5678152450 2024-04-26T02:28:13.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.678+0000 7f5666ffd640 1 -- 172.21.15.94:0/1937494246 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5668018020 con 0x7f5678152450 2024-04-26T02:28:13.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.682+0000 7f5666ffd640 1 --2- 172.21.15.94:0/1937494246 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f565806f400 0x7f5658071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:13.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.682+0000 7f5666ffd640 1 -- 172.21.15.94:0/1937494246 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f566808f350 con 0x7f5678152450 2024-04-26T02:28:13.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.682+0000 7f5675d9b640 1 --2- 172.21.15.94:0/1937494246 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f565806f400 0x7f5658071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:13.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.682+0000 7f5675d9b640 1 --2- 172.21.15.94:0/1937494246 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f565806f400 0x7f5658071910 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f566c0ae000 tx=0x7f566c007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:13.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.686+0000 7f5666ffd640 1 -- 172.21.15.94:0/1937494246 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f566801f020 con 0x7f5678152450 2024-04-26T02:28:13.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f567cb0e640 1 -- 172.21.15.94:0/1937494246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56780b2af0 con 0x7f5678152450 2024-04-26T02:28:13.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f5666ffd640 1 -- 172.21.15.94:0/1937494246 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17003 (secure 0 0 0) 0x7f566805c100 con 0x7f5678152450 2024-04-26T02:28:13.935 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:13.935 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":40,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:52:35.869046+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:52:23.840281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T16:52:25.130347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T02:28:13.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f567cb0e640 1 -- 172.21.15.94:0/1937494246 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f565806f400 msgr2=0x7f5658071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:13.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f567cb0e640 1 --2- 172.21.15.94:0/1937494246 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f565806f400 0x7f5658071910 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f566c0ae000 tx=0x7f566c007040 comp rx=0 tx=0).stop 2024-04-26T02:28:13.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f567cb0e640 1 -- 172.21.15.94:0/1937494246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5678152450 msgr2=0x7f56781504b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:13.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f567cb0e640 1 --2- 172.21.15.94:0/1937494246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5678152450 0x7f56781504b0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f566800b410 tx=0x7f566800b8d0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f567cb0e640 1 -- 172.21.15.94:0/1937494246 shutdown_connections 2024-04-26T02:28:13.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f567cb0e640 1 --2- 172.21.15.94:0/1937494246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56780c9530 0x7f56780c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f567cb0e640 1 --2- 172.21.15.94:0/1937494246 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f565806f400 0x7f5658071910 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f567cb0e640 1 --2- 172.21.15.94:0/1937494246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56781499d0 0x7f56780c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f567cb0e640 1 --2- 172.21.15.94:0/1937494246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5678152450 0x7f56781504b0 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:13.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f567cb0e640 1 -- 172.21.15.94:0/1937494246 >> 172.21.15.94:0/1937494246 conn(0x7f56780cfed0 msgr2=0x7f56780b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:13.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f567cb0e640 1 -- 172.21.15.94:0/1937494246 shutdown_connections 2024-04-26T02:28:13.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:13.930+0000 7f567cb0e640 1 -- 172.21.15.94:0/1937494246 wait complete. 2024-04-26T02:28:13.965 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T02:28:13.965 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T02:28:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.138+0000 7fad2f742640 1 Processor -- start 2024-04-26T02:28:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.138+0000 7fad2f742640 1 -- start start 2024-04-26T02:28:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.138+0000 7fad2f742640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fad280cdcb0 0x7fad28072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.138+0000 7fad2f742640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad280b7df0 0x7fad28161c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.138+0000 7fad2f742640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad28073090 con 0x7fad28072c80 2024-04-26T02:28:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.138+0000 7fad2f742640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fad280d2120 con 0x7fad280cdcb0 2024-04-26T02:28:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.138+0000 7fad2f742640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fad280d2400 con 0x7fad280b7df0 2024-04-26T02:28:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.138+0000 7fad2ccdd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fad28072c80 0x7fad28161580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:52454/0 (socket says 172.21.15.94:52454) 2024-04-26T02:28:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.138+0000 7fad2ccdd640 1 -- 172.21.15.94:0/941694324 learned_addr learned my addr 172.21.15.94:0/941694324 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.138+0000 7fad2dcdf640 1 --2- 172.21.15.94:0/941694324 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad280b7df0 0x7fad28161c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.142+0000 7fad1ffff640 1 -- 172.21.15.94:0/941694324 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3286706618 0 0) 0x7fad18001120 con 0x7fad28072c80 2024-04-26T02:28:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.142+0000 7fad1ffff640 1 -- 172.21.15.94:0/941694324 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad14003530 con 0x7fad28072c80 2024-04-26T02:28:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.142+0000 7fad2dcdf640 1 -- 172.21.15.94:0/941694324 >> v2:172.21.15.138:3300/0 conn(0x7fad280cdcb0 msgr2=0x7fad28072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.142+0000 7fad2dcdf640 1 --2- 172.21.15.94:0/941694324 >> v2:172.21.15.138:3300/0 conn(0x7fad280cdcb0 0x7fad28072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.142+0000 7fad2dcdf640 1 -- 172.21.15.94:0/941694324 >> v1:172.21.15.94:6789/0 conn(0x7fad28072c80 legacy=0x7fad28161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.142+0000 7fad2dcdf640 1 -- 172.21.15.94:0/941694324 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad280d26e0 con 0x7fad280b7df0 2024-04-26T02:28:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.142+0000 7fad2dcdf640 1 --2- 172.21.15.94:0/941694324 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad280b7df0 0x7fad28161c90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fad240097c0 tx=0x7fad2401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3534392f1a5c4e75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:14.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.142+0000 7fad1ffff640 1 -- 172.21.15.94:0/941694324 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad240b1070 con 0x7fad280b7df0 2024-04-26T02:28:14.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.142+0000 7fad1ffff640 1 -- 172.21.15.94:0/941694324 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad240a8050 con 0x7fad280b7df0 2024-04-26T02:28:14.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.142+0000 7fad1ffff640 1 -- 172.21.15.94:0/941694324 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad240ac8a0 con 0x7fad280b7df0 2024-04-26T02:28:14.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.146+0000 7fad2f742640 1 -- 172.21.15.94:0/941694324 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad280b7df0 msgr2=0x7fad28161c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:14.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.146+0000 7fad2f742640 1 --2- 172.21.15.94:0/941694324 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad280b7df0 0x7fad28161c90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fad240097c0 tx=0x7fad2401c400 comp rx=0 tx=0).stop 2024-04-26T02:28:14.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.150+0000 7fad2f742640 1 -- 172.21.15.94:0/941694324 shutdown_connections 2024-04-26T02:28:14.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.150+0000 7fad2f742640 1 --2- 172.21.15.94:0/941694324 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad280b7df0 0x7fad28161c90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.150+0000 7fad2f742640 1 --2- 172.21.15.94:0/941694324 >> v2:172.21.15.138:3300/0 conn(0x7fad280cdcb0 0x7fad28072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.150+0000 7fad2f742640 1 -- 172.21.15.94:0/941694324 >> 172.21.15.94:0/941694324 conn(0x7fad280d3850 msgr2=0x7fad280d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.154+0000 7fad2f742640 1 -- 172.21.15.94:0/941694324 shutdown_connections 2024-04-26T02:28:14.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.154+0000 7fad2f742640 1 -- 172.21.15.94:0/941694324 wait complete. 2024-04-26T02:28:14.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2f742640 1 Processor -- start 2024-04-26T02:28:14.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2f742640 1 -- start start 2024-04-26T02:28:14.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2f742640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad28072c80 0x7fad28154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:14.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2f742640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad280b7df0 0x7fad281581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:14.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2f742640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad280cdcb0 0x7fad2806ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:14.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2f742640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad280b6680 con 0x7fad28072c80 2024-04-26T02:28:14.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2f742640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad280b6500 con 0x7fad280b7df0 2024-04-26T02:28:14.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2f742640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fad280ce0c0 con 0x7fad280cdcb0 2024-04-26T02:28:14.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2d4de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad28072c80 0x7fad28154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:14.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2d4de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad28072c80 0x7fad28154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54360/0 (socket says 172.21.15.94:54360) 2024-04-26T02:28:14.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2d4de640 1 -- 172.21.15.94:0/1206074231 learned_addr learned my addr 172.21.15.94:0/1206074231 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:14.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2ccdd640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad280b7df0 0x7fad281581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:14.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2dcdf640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad280cdcb0 0x7fad2806ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:14.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2d4de640 1 -- 172.21.15.94:0/1206074231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad280cdcb0 msgr2=0x7fad2806ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:14.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2d4de640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad280cdcb0 0x7fad2806ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2d4de640 1 -- 172.21.15.94:0/1206074231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad280b7df0 msgr2=0x7fad281581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:14.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2d4de640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad280b7df0 0x7fad281581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2d4de640 1 -- 172.21.15.94:0/1206074231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2806fe20 con 0x7fad28072c80 2024-04-26T02:28:14.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad2d4de640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad28072c80 0x7fad28154870 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fad2000ba80 tx=0x7fad2000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:14.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.158+0000 7fad1dffb640 1 -- 172.21.15.94:0/1206074231 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20019070 con 0x7fad28072c80 2024-04-26T02:28:14.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.162+0000 7fad2f742640 1 -- 172.21.15.94:0/1206074231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2806e1b0 con 0x7fad28072c80 2024-04-26T02:28:14.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.162+0000 7fad1dffb640 1 -- 172.21.15.94:0/1206074231 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2000cc20 con 0x7fad28072c80 2024-04-26T02:28:14.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.162+0000 7fad1dffb640 1 -- 172.21.15.94:0/1206074231 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20014440 con 0x7fad28072c80 2024-04-26T02:28:14.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.162+0000 7fad2f742640 1 -- 172.21.15.94:0/1206074231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad28154f20 con 0x7fad28072c80 2024-04-26T02:28:14.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.162+0000 7fad137fe640 1 -- 172.21.15.94:0/1206074231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad000052a0 con 0x7fad28072c80 2024-04-26T02:28:14.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.162+0000 7fad1dffb640 1 -- 172.21.15.94:0/1206074231 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fad2001c070 con 0x7fad28072c80 2024-04-26T02:28:14.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.162+0000 7fad1dffb640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fad0c06f400 0x7fad0c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:14.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.162+0000 7fad1dffb640 1 -- 172.21.15.94:0/1206074231 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fad2008f1c0 con 0x7fad28072c80 2024-04-26T02:28:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.166+0000 7fad2ccdd640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fad0c06f400 0x7fad0c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.166+0000 7fad1dffb640 1 -- 172.21.15.94:0/1206074231 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fad2005b830 con 0x7fad28072c80 2024-04-26T02:28:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.170+0000 7fad2ccdd640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fad0c06f400 0x7fad0c071910 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fad280c99c0 tx=0x7fad18007b80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:14.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.446+0000 7fad137fe640 1 -- 172.21.15.94:0/1206074231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad00005860 con 0x7fad28072c80 2024-04-26T02:28:14.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.446+0000 7fad1dffb640 1 -- 172.21.15.94:0/1206074231 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5184 (secure 0 0 0) 0x7fad2005fb30 con 0x7fad28072c80 2024-04-26T02:28:14.450 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:14.450 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/478421533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":478421533},{"type":"v1","addr":"172.21.15.138:6833","nonce":478421533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2461866660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2461866660},{"type":"v1","addr":"172.21.15.138:6835","nonce":2461866660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5439,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2566493915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2566493915},{"type":"v1","addr":"172.21.15.94:6835","nonce":2566493915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:52:26.175197+0000","modified":"2024-04-25T16:52:29.233625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5442":{"gid":5442,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/2235039329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2235039329},{"type":"v1","addr":"172.21.15.94:6837","nonce":2235039329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5442,"qdb_cluster":[5442]},"id":2}]} 2024-04-26T02:28:14.451 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-04-26T02:28:14.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.446+0000 7fad137fe640 1 -- 172.21.15.94:0/1206074231 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fad0c06f400 msgr2=0x7fad0c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:14.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.446+0000 7fad137fe640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fad0c06f400 0x7fad0c071910 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fad280c99c0 tx=0x7fad18007b80 comp rx=0 tx=0).stop 2024-04-26T02:28:14.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.446+0000 7fad137fe640 1 -- 172.21.15.94:0/1206074231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad28072c80 msgr2=0x7fad28154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:14.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.446+0000 7fad137fe640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad28072c80 0x7fad28154870 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fad2000ba80 tx=0x7fad2000bf40 comp rx=0 tx=0).stop 2024-04-26T02:28:14.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.450+0000 7fad137fe640 1 -- 172.21.15.94:0/1206074231 shutdown_connections 2024-04-26T02:28:14.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.450+0000 7fad137fe640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fad0c06f400 0x7fad0c071910 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.450+0000 7fad137fe640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad280cdcb0 0x7fad2806ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.450+0000 7fad137fe640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad280b7df0 0x7fad281581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.450+0000 7fad137fe640 1 --2- 172.21.15.94:0/1206074231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad28072c80 0x7fad28154870 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.450+0000 7fad137fe640 1 -- 172.21.15.94:0/1206074231 >> 172.21.15.94:0/1206074231 conn(0x7fad280d3850 msgr2=0x7fad280d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.450+0000 7fad137fe640 1 -- 172.21.15.94:0/1206074231 shutdown_connections 2024-04-26T02:28:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.450+0000 7fad137fe640 1 -- 172.21.15.94:0/1206074231 wait complete. 2024-04-26T02:28:14.483 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T02:28:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.658+0000 7ff2ea2ed640 1 Processor -- start 2024-04-26T02:28:14.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.666+0000 7ff2ea2ed640 1 -- start start 2024-04-26T02:28:14.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2ea2ed640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff2e40cf0e0 0x7ff2e415f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:14.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2ea2ed640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e40d1700 0x7ff2e40b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:14.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2ea2ed640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2e40b5360 con 0x7ff2e40b66e0 2024-04-26T02:28:14.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2ea2ed640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ff2e40b6250 con 0x7ff2e40cf0e0 2024-04-26T02:28:14.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2ea2ed640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e40b5640 con 0x7ff2e40d1700 2024-04-26T02:28:14.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e3fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff2e40cf0e0 0x7ff2e415f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:14.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e3fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff2e40cf0e0 0x7ff2e415f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54364/0 (socket says 172.21.15.94:54364) 2024-04-26T02:28:14.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e3fff640 1 -- 172.21.15.94:0/2058117199 learned_addr learned my addr 172.21.15.94:0/2058117199 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:14.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e888a640 1 --2- 172.21.15.94:0/2058117199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e40d1700 0x7ff2e40b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:14.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/2058117199 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3477727070 0 0) 0x7ff2e40b5360 con 0x7ff2e40b66e0 2024-04-26T02:28:14.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/2058117199 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2cc003530 con 0x7ff2e40b66e0 2024-04-26T02:28:14.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e888a640 1 -- 172.21.15.94:0/2058117199 >> v2:172.21.15.94:3300/0 conn(0x7ff2e40cf0e0 msgr2=0x7ff2e415f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:14.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e888a640 1 --2- 172.21.15.94:0/2058117199 >> v2:172.21.15.94:3300/0 conn(0x7ff2e40cf0e0 0x7ff2e415f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e888a640 1 -- 172.21.15.94:0/2058117199 >> v1:172.21.15.138:6789/0 conn(0x7ff2e40b66e0 legacy=0x7ff2e40d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:14.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e888a640 1 -- 172.21.15.94:0/2058117199 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e40b5920 con 0x7ff2e40d1700 2024-04-26T02:28:14.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e888a640 1 --2- 172.21.15.94:0/2058117199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e40d1700 0x7ff2e40b6af0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0097c0 tx=0x7ff2dc01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=350297dfa677fec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:14.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/2058117199 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc0b1070 con 0x7ff2e40d1700 2024-04-26T02:28:14.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/2058117199 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2dc0ac410 con 0x7ff2e40d1700 2024-04-26T02:28:14.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.670+0000 7ff2e2ffd640 1 -- 172.21.15.94:0/2058117199 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc0ac7e0 con 0x7ff2e40d1700 2024-04-26T02:28:14.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.674+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2058117199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e40d1700 msgr2=0x7ff2e40b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:14.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.674+0000 7ff2ea2ed640 1 --2- 172.21.15.94:0/2058117199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e40d1700 0x7ff2e40b6af0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0097c0 tx=0x7ff2dc01c4c0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.674+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2058117199 shutdown_connections 2024-04-26T02:28:14.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.674+0000 7ff2ea2ed640 1 --2- 172.21.15.94:0/2058117199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e40d1700 0x7ff2e40b6af0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.674+0000 7ff2ea2ed640 1 --2- 172.21.15.94:0/2058117199 >> v2:172.21.15.94:3300/0 conn(0x7ff2e40cf0e0 0x7ff2e415f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.674+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2058117199 >> 172.21.15.94:0/2058117199 conn(0x7ff2e4070f40 msgr2=0x7ff2e4071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:14.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.678+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2058117199 shutdown_connections 2024-04-26T02:28:14.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.678+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2058117199 wait complete. 2024-04-26T02:28:14.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2ea2ed640 1 Processor -- start 2024-04-26T02:28:14.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2ea2ed640 1 -- start start 2024-04-26T02:28:14.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2ea2ed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e40b66e0 0x7ff2e4151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:14.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2ea2ed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e40cf0e0 0x7ff2e40b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:14.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2ea2ed640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e4073860 0x7ff2e4073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:14.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2e3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e40b66e0 0x7ff2e4151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:14.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2e3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e40b66e0 0x7ff2e4151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54374/0 (socket says 172.21.15.94:54374) 2024-04-26T02:28:14.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2e3fff640 1 -- 172.21.15.94:0/2048432638 learned_addr learned my addr 172.21.15.94:0/2048432638 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:14.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2e37fe640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e40cf0e0 0x7ff2e40b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:14.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2048432638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e416cb30 con 0x7ff2e40b66e0 2024-04-26T02:28:14.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2048432638 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e416c9b0 con 0x7ff2e40cf0e0 2024-04-26T02:28:14.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2048432638 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e416ccb0 con 0x7ff2e4073860 2024-04-26T02:28:14.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2e888a640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e4073860 0x7ff2e4073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:14.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2e3fff640 1 -- 172.21.15.94:0/2048432638 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e4073860 msgr2=0x7ff2e4073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:14.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2e3fff640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e4073860 0x7ff2e4073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2e3fff640 1 -- 172.21.15.94:0/2048432638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e40cf0e0 msgr2=0x7ff2e40b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:14.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2e3fff640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e40cf0e0 0x7ff2e40b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.682+0000 7ff2e3fff640 1 -- 172.21.15.94:0/2048432638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e40741b0 con 0x7ff2e40b66e0 2024-04-26T02:28:14.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.686+0000 7ff2e3fff640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e40b66e0 0x7ff2e4151ce0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009cd0 tx=0x7ff2d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:14.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.686+0000 7ff2e0ff9640 1 -- 172.21.15.94:0/2048432638 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400cda0 con 0x7ff2e40b66e0 2024-04-26T02:28:14.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.686+0000 7ff2e0ff9640 1 -- 172.21.15.94:0/2048432638 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d4019070 con 0x7ff2e40b66e0 2024-04-26T02:28:14.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.686+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2048432638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e40b16b0 con 0x7ff2e40b66e0 2024-04-26T02:28:14.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.686+0000 7ff2e0ff9640 1 -- 172.21.15.94:0/2048432638 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40146c0 con 0x7ff2e40b66e0 2024-04-26T02:28:14.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.686+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2048432638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e40b1c70 con 0x7ff2e40b66e0 2024-04-26T02:28:14.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.686+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2048432638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e406a730 con 0x7ff2e40b66e0 2024-04-26T02:28:14.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.686+0000 7ff2e0ff9640 1 -- 172.21.15.94:0/2048432638 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff2d4012050 con 0x7ff2e40b66e0 2024-04-26T02:28:14.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.686+0000 7ff2e0ff9640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff2d006f430 0x7ff2d0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:14.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.686+0000 7ff2e0ff9640 1 -- 172.21.15.94:0/2048432638 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7ff2d408f970 con 0x7ff2e40b66e0 2024-04-26T02:28:14.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.686+0000 7ff2e37fe640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff2d006f430 0x7ff2d0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:14.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.690+0000 7ff2e37fe640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff2d006f430 0x7ff2d0071940 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40b5550 tx=0x7ff2d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:14.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.698+0000 7ff2e0ff9640 1 -- 172.21.15.94:0/2048432638 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff2d4023020 con 0x7ff2e40b66e0 2024-04-26T02:28:14.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.970+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2048432638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2e40d04e0 con 0x7ff2e40b66e0 2024-04-26T02:28:14.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.970+0000 7ff2e0ff9640 1 -- 172.21.15.94:0/2048432638 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5184 (secure 0 0 0) 0x7ff2e40741b0 con 0x7ff2e40b66e0 2024-04-26T02:28:14.975 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:14.975 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/478421533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":478421533},{"type":"v1","addr":"172.21.15.138:6833","nonce":478421533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2461866660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2461866660},{"type":"v1","addr":"172.21.15.138:6835","nonce":2461866660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5439,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2566493915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2566493915},{"type":"v1","addr":"172.21.15.94:6835","nonce":2566493915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T16:52:26.175197+0000","modified":"2024-04-25T16:52:29.233625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5442":{"gid":5442,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6837/2235039329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2235039329},{"type":"v1","addr":"172.21.15.94:6837","nonce":2235039329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5442,"qdb_cluster":[5442]},"id":2}]} 2024-04-26T02:28:14.975 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-04-26T02:28:14.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.970+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2048432638 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff2d006f430 msgr2=0x7ff2d0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:14.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.970+0000 7ff2ea2ed640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff2d006f430 0x7ff2d0071940 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40b5550 tx=0x7ff2d8009040 comp rx=0 tx=0).stop 2024-04-26T02:28:14.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.970+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2048432638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e40b66e0 msgr2=0x7ff2e4151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:14.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.970+0000 7ff2ea2ed640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e40b66e0 0x7ff2e4151ce0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009cd0 tx=0x7ff2d400c490 comp rx=0 tx=0).stop 2024-04-26T02:28:14.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.974+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2048432638 shutdown_connections 2024-04-26T02:28:14.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.974+0000 7ff2ea2ed640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff2d006f430 0x7ff2d0071940 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.974+0000 7ff2ea2ed640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e4073860 0x7ff2e4073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.974+0000 7ff2ea2ed640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e40cf0e0 0x7ff2e40b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.974+0000 7ff2ea2ed640 1 --2- 172.21.15.94:0/2048432638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e40b66e0 0x7ff2e4151ce0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:14.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.974+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2048432638 >> 172.21.15.94:0/2048432638 conn(0x7ff2e4070f40 msgr2=0x7ff2e41635a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:14.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.974+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2048432638 shutdown_connections 2024-04-26T02:28:14.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:14.974+0000 7ff2ea2ed640 1 -- 172.21.15.94:0/2048432638 wait complete. 2024-04-26T02:28:15.006 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T02:28:15.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b40347640 1 Processor -- start 2024-04-26T02:28:15.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b40347640 1 -- start start 2024-04-26T02:28:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b40347640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b38070570 0x7f3b3806ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b40347640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3b380cee20 0x7f3b380cf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b40347640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b3806e830 con 0x7f3b380d1340 2024-04-26T02:28:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b40347640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f3b3806d790 con 0x7f3b380cee20 2024-04-26T02:28:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b40347640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3806da70 con 0x7f3b38070570 2024-04-26T02:28:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b3d8e2640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3b380d1340 0x7f3b380d1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.94:33052/0 (socket says 172.21.15.94:33052) 2024-04-26T02:28:15.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b3e0e3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b38070570 0x7f3b3806ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b3d8e2640 1 -- 172.21.15.94:0/1165502148 learned_addr learned my addr 172.21.15.94:0/1165502148 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b3e8e4640 1 --2- 172.21.15.94:0/1165502148 >> v2:172.21.15.94:3300/0 conn(0x7f3b380cee20 0x7f3b380cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b3d0e1640 1 -- 172.21.15.94:0/1165502148 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3769688171 0 0) 0x7f3b3806e830 con 0x7f3b380d1340 2024-04-26T02:28:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b3d0e1640 1 -- 172.21.15.94:0/1165502148 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b24003530 con 0x7f3b380d1340 2024-04-26T02:28:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b3e8e4640 1 -- 172.21.15.94:0/1165502148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b38070570 msgr2=0x7f3b3806ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b3e8e4640 1 --2- 172.21.15.94:0/1165502148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b38070570 0x7f3b3806ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b3e8e4640 1 -- 172.21.15.94:0/1165502148 >> v1:172.21.15.138:6789/0 conn(0x7f3b380d1340 legacy=0x7f3b380d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:15.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.170+0000 7f3b3e8e4640 1 -- 172.21.15.94:0/1165502148 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3806dd50 con 0x7f3b380cee20 2024-04-26T02:28:15.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.174+0000 7f3b3e8e4640 1 --2- 172.21.15.94:0/1165502148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b380cee20 0x7f3b380cf230 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009d90 tx=0x7f3b3001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f9dc76340bd008 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:15.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.174+0000 7f3b3d0e1640 1 -- 172.21.15.94:0/1165502148 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b300b2070 con 0x7f3b380cee20 2024-04-26T02:28:15.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.174+0000 7f3b3d0e1640 1 -- 172.21.15.94:0/1165502148 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b300a5920 con 0x7f3b380cee20 2024-04-26T02:28:15.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.174+0000 7f3b3d0e1640 1 -- 172.21.15.94:0/1165502148 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b300a5cf0 con 0x7f3b380cee20 2024-04-26T02:28:15.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.174+0000 7f3b40347640 1 -- 172.21.15.94:0/1165502148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b380cee20 msgr2=0x7f3b380cf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:15.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.174+0000 7f3b40347640 1 --2- 172.21.15.94:0/1165502148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b380cee20 0x7f3b380cf230 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009d90 tx=0x7f3b3001c970 comp rx=0 tx=0).stop 2024-04-26T02:28:15.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.174+0000 7f3b40347640 1 -- 172.21.15.94:0/1165502148 shutdown_connections 2024-04-26T02:28:15.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.174+0000 7f3b40347640 1 --2- 172.21.15.94:0/1165502148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b380cee20 0x7f3b380cf230 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.174+0000 7f3b40347640 1 --2- 172.21.15.94:0/1165502148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b38070570 0x7f3b3806ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.174+0000 7f3b40347640 1 -- 172.21.15.94:0/1165502148 >> 172.21.15.94:0/1165502148 conn(0x7f3b38071150 msgr2=0x7f3b38071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:15.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.174+0000 7f3b40347640 1 -- 172.21.15.94:0/1165502148 shutdown_connections 2024-04-26T02:28:15.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.174+0000 7f3b40347640 1 -- 172.21.15.94:0/1165502148 wait complete. 2024-04-26T02:28:15.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.178+0000 7f3b40347640 1 Processor -- start 2024-04-26T02:28:15.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.178+0000 7f3b40347640 1 -- start start 2024-04-26T02:28:15.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.178+0000 7f3b40347640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b38070570 0x7f3b381447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:15.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b40347640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b380cee20 0x7f3b3815c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:15.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b40347640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b380d1340 0x7f3b380b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:15.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b40347640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3816aac0 con 0x7f3b38070570 2024-04-26T02:28:15.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b40347640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3816a940 con 0x7f3b380cee20 2024-04-26T02:28:15.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b40347640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3816ac40 con 0x7f3b380d1340 2024-04-26T02:28:15.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b3e0e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b38070570 0x7f3b381447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b3e0e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b38070570 0x7f3b381447d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54394/0 (socket says 172.21.15.94:54394) 2024-04-26T02:28:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b3e0e3640 1 -- 172.21.15.94:0/2798038019 learned_addr learned my addr 172.21.15.94:0/2798038019 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b3e8e4640 1 --2- 172.21.15.94:0/2798038019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b380d1340 0x7f3b380b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b3e8e4640 1 -- 172.21.15.94:0/2798038019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b380cee20 msgr2=0x7f3b3815c9c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T02:28:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b3e8e4640 1 --2- 172.21.15.94:0/2798038019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b380cee20 0x7f3b3815c9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b3e8e4640 1 -- 172.21.15.94:0/2798038019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b38070570 msgr2=0x7f3b381447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b3e8e4640 1 --2- 172.21.15.94:0/2798038019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b38070570 0x7f3b381447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b3e8e4640 1 -- 172.21.15.94:0/2798038019 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b38140af0 con 0x7f3b380d1340 2024-04-26T02:28:15.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b3e8e4640 1 --2- 172.21.15.94:0/2798038019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b380d1340 0x7f3b380b2880 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3b300ac000 tx=0x7f3b30019710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:15.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b2effd640 1 -- 172.21.15.94:0/2798038019 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b300b2050 con 0x7f3b380d1340 2024-04-26T02:28:15.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b2effd640 1 -- 172.21.15.94:0/2798038019 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b300b3040 con 0x7f3b380d1340 2024-04-26T02:28:15.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b2effd640 1 -- 172.21.15.94:0/2798038019 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b300ad850 con 0x7f3b380d1340 2024-04-26T02:28:15.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b40347640 1 -- 172.21.15.94:0/2798038019 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b38140d90 con 0x7f3b380d1340 2024-04-26T02:28:15.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b40347640 1 -- 172.21.15.94:0/2798038019 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b38141350 con 0x7f3b380d1340 2024-04-26T02:28:15.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b2effd640 1 -- 172.21.15.94:0/2798038019 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3b300c0020 con 0x7f3b380d1340 2024-04-26T02:28:15.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b2effd640 1 --2- 172.21.15.94:0/2798038019 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3b1c06f430 0x7f3b1c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:15.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b2effd640 1 -- 172.21.15.94:0/2798038019 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f3b30129c80 con 0x7f3b380d1340 2024-04-26T02:28:15.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.182+0000 7f3b3e0e3640 1 --2- 172.21.15.94:0/2798038019 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3b1c06f430 0x7f3b1c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:15.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.186+0000 7f3b3e0e3640 1 --2- 172.21.15.94:0/2798038019 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3b1c06f430 0x7f3b1c071940 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009b80 tx=0x7f3b34009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.186+0000 7f3b2cff9640 1 -- 172.21.15.94:0/2798038019 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3806a730 con 0x7f3b380d1340 2024-04-26T02:28:15.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.194+0000 7f3b2effd640 1 -- 172.21.15.94:0/2798038019 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3b38140af0 con 0x7f3b380d1340 2024-04-26T02:28:15.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.430+0000 7f3b2cff9640 1 -- 172.21.15.94:0/2798038019 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b380d0c10 con 0x7f3b380d1340 2024-04-26T02:28:15.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.570+0000 7f3b2effd640 1 -- 172.21.15.94:0/2798038019 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17003 (secure 0 0 0) 0x7f3b30016560 con 0x7f3b380d1340 2024-04-26T02:28:15.580 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:15.580 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":40,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-25T16:52:35.869046+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:52:23.840281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T16:52:25.130347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T02:28:15.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.574+0000 7f3b2cff9640 1 -- 172.21.15.94:0/2798038019 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3b1c06f430 msgr2=0x7f3b1c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:15.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.574+0000 7f3b2cff9640 1 --2- 172.21.15.94:0/2798038019 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3b1c06f430 0x7f3b1c071940 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009b80 tx=0x7f3b34009190 comp rx=0 tx=0).stop 2024-04-26T02:28:15.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.574+0000 7f3b2cff9640 1 -- 172.21.15.94:0/2798038019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b380d1340 msgr2=0x7f3b380b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:15.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.574+0000 7f3b2cff9640 1 --2- 172.21.15.94:0/2798038019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b380d1340 0x7f3b380b2880 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3b300ac000 tx=0x7f3b30019710 comp rx=0 tx=0).stop 2024-04-26T02:28:15.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.574+0000 7f3b2cff9640 1 -- 172.21.15.94:0/2798038019 shutdown_connections 2024-04-26T02:28:15.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.574+0000 7f3b2cff9640 1 --2- 172.21.15.94:0/2798038019 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3b1c06f430 0x7f3b1c071940 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.574+0000 7f3b2cff9640 1 --2- 172.21.15.94:0/2798038019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b380d1340 0x7f3b380b2880 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.574+0000 7f3b2cff9640 1 --2- 172.21.15.94:0/2798038019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b380cee20 0x7f3b3815c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.574+0000 7f3b2cff9640 1 --2- 172.21.15.94:0/2798038019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b38070570 0x7f3b381447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.574+0000 7f3b2cff9640 1 -- 172.21.15.94:0/2798038019 >> 172.21.15.94:0/2798038019 conn(0x7f3b38071150 msgr2=0x7f3b3813f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:15.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.574+0000 7f3b2cff9640 1 -- 172.21.15.94:0/2798038019 shutdown_connections 2024-04-26T02:28:15.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.574+0000 7f3b2cff9640 1 -- 172.21.15.94:0/2798038019 wait complete. 2024-04-26T02:28:15.612 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T02:28:15.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fd4d3f640 1 Processor -- start 2024-04-26T02:28:15.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fd4d3f640 1 -- start start 2024-04-26T02:28:15.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fd4d3f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5fd00b6b80 0x7f5fd0160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:15.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fd4d3f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5fd01607b0 0x7f5fd0161040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:15.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fd4d3f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fd00b0850 con 0x7f5fd00b1890 2024-04-26T02:28:15.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fd4d3f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f5fd0070990 con 0x7f5fd00b6b80 2024-04-26T02:28:15.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fd4d3f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5fd00b0ad0 con 0x7f5fd01607b0 2024-04-26T02:28:15.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fcdd9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5fd00b6b80 0x7f5fd0160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:15.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fcdd9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5fd00b6b80 0x7f5fd0160270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54408/0 (socket says 172.21.15.94:54408) 2024-04-26T02:28:15.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fcdd9b640 1 -- 172.21.15.94:0/3705206932 learned_addr learned my addr 172.21.15.94:0/3705206932 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:15.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fced9d640 1 --2- 172.21.15.94:0/3705206932 >> v2:172.21.15.138:3300/0 conn(0x7f5fd01607b0 0x7f5fd0161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:15.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fcdd9b640 1 -- 172.21.15.94:0/3705206932 >> v2:172.21.15.138:3300/0 conn(0x7f5fd01607b0 msgr2=0x7f5fd0161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:15.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fcdd9b640 1 --2- 172.21.15.94:0/3705206932 >> v2:172.21.15.138:3300/0 conn(0x7f5fd01607b0 0x7f5fd0161040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fcdd9b640 1 -- 172.21.15.94:0/3705206932 >> v1:172.21.15.138:6789/0 conn(0x7f5fd00b1890 legacy=0x7f5fd00b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:15.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.790+0000 7f5fcdd9b640 1 -- 172.21.15.94:0/3705206932 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd00b0db0 con 0x7f5fd00b6b80 2024-04-26T02:28:15.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.794+0000 7f5fcdd9b640 1 --2- 172.21.15.94:0/3705206932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd00b6b80 0x7f5fd0160270 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0070b10 tx=0x7f5fc401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16dec22519bd66ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:15.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.794+0000 7f5fcd59a640 1 -- 172.21.15.94:0/3705206932 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40ab660 con 0x7f5fd00b6b80 2024-04-26T02:28:15.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.794+0000 7f5fcd59a640 1 -- 172.21.15.94:0/3705206932 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc40b0070 con 0x7f5fd00b6b80 2024-04-26T02:28:15.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.794+0000 7f5fcd59a640 1 -- 172.21.15.94:0/3705206932 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40aba20 con 0x7f5fd00b6b80 2024-04-26T02:28:15.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.798+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/3705206932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd00b6b80 msgr2=0x7f5fd0160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:15.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.798+0000 7f5fd4d3f640 1 --2- 172.21.15.94:0/3705206932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd00b6b80 0x7f5fd0160270 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0070b10 tx=0x7f5fc401c460 comp rx=0 tx=0).stop 2024-04-26T02:28:15.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.798+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/3705206932 shutdown_connections 2024-04-26T02:28:15.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.798+0000 7f5fd4d3f640 1 --2- 172.21.15.94:0/3705206932 >> v2:172.21.15.138:3300/0 conn(0x7f5fd01607b0 0x7f5fd0161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.798+0000 7f5fd4d3f640 1 --2- 172.21.15.94:0/3705206932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd00b6b80 0x7f5fd0160270 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.798+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/3705206932 >> 172.21.15.94:0/3705206932 conn(0x7f5fd0071f00 msgr2=0x7f5fd0073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:15.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.798+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/3705206932 shutdown_connections 2024-04-26T02:28:15.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.798+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/3705206932 wait complete. 2024-04-26T02:28:15.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.798+0000 7f5fd4d3f640 1 Processor -- start 2024-04-26T02:28:15.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.798+0000 7f5fd4d3f640 1 -- start start 2024-04-26T02:28:15.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fd4d3f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd00b1890 0x7f5fd01419a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:15.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fd4d3f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fd01607b0 0x7f5fd0159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:15.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fd4d3f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fd00b3350 0x7f5fd00b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:15.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fd4d3f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd0160d40 con 0x7f5fd00b1890 2024-04-26T02:28:15.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fd4d3f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd0160bc0 con 0x7f5fd01607b0 2024-04-26T02:28:15.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fd4d3f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5fd016c790 con 0x7f5fd00b3350 2024-04-26T02:28:15.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fce59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd00b1890 0x7f5fd01419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:15.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fce59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd00b1890 0x7f5fd01419a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54422/0 (socket says 172.21.15.94:54422) 2024-04-26T02:28:15.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fce59c640 1 -- 172.21.15.94:0/829604293 learned_addr learned my addr 172.21.15.94:0/829604293 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:15.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fced9d640 1 --2- 172.21.15.94:0/829604293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fd00b3350 0x7f5fd00b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:15.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fce59c640 1 -- 172.21.15.94:0/829604293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fd00b3350 msgr2=0x7f5fd00b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:15.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fce59c640 1 --2- 172.21.15.94:0/829604293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fd00b3350 0x7f5fd00b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fce59c640 1 -- 172.21.15.94:0/829604293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fd01607b0 msgr2=0x7f5fd0159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:15.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fce59c640 1 --2- 172.21.15.94:0/829604293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fd01607b0 0x7f5fd0159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:15.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fce59c640 1 -- 172.21.15.94:0/829604293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd015fba0 con 0x7f5fd00b1890 2024-04-26T02:28:15.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fce59c640 1 --2- 172.21.15.94:0/829604293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd00b1890 0x7f5fd01419a0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000c760 tx=0x7f5fc000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:15.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fbeffd640 1 -- 172.21.15.94:0/829604293 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0016020 con 0x7f5fd00b1890 2024-04-26T02:28:15.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fbeffd640 1 -- 172.21.15.94:0/829604293 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc0006d60 con 0x7f5fd00b1890 2024-04-26T02:28:15.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fbeffd640 1 -- 172.21.15.94:0/829604293 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0003bc0 con 0x7f5fd00b1890 2024-04-26T02:28:15.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/829604293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd0069b60 con 0x7f5fd00b1890 2024-04-26T02:28:15.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/829604293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd00cf250 con 0x7f5fd00b1890 2024-04-26T02:28:15.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.802+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/829604293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd006a730 con 0x7f5fd00b1890 2024-04-26T02:28:15.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.814+0000 7f5fbeffd640 1 -- 172.21.15.94:0/829604293 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5fc000a850 con 0x7f5fd00b1890 2024-04-26T02:28:15.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.814+0000 7f5fbeffd640 1 --2- 172.21.15.94:0/829604293 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5fb006f400 0x7f5fb0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:15.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.814+0000 7f5fbeffd640 1 -- 172.21.15.94:0/829604293 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f5fc008fd70 con 0x7f5fd00b1890 2024-04-26T02:28:15.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.814+0000 7f5fcdd9b640 1 --2- 172.21.15.94:0/829604293 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5fb006f400 0x7f5fb0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:15.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.814+0000 7f5fbeffd640 1 -- 172.21.15.94:0/829604293 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5fc0020020 con 0x7f5fd00b1890 2024-04-26T02:28:15.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:15.818+0000 7f5fcdd9b640 1 --2- 172.21.15.94:0/829604293 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5fb006f400 0x7f5fb0071910 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40af000 tx=0x7f5fc4004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:16.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.102+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/829604293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5fd0162f60 con 0x7f5fd00b1890 2024-04-26T02:28:16.195 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-04-26T02:28:16.190+0000 7fb03286f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T02:28:16.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.190+0000 7f5fbeffd640 1 -- 172.21.15.94:0/829604293 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f5fd00b0850 con 0x7f5fd00b1890 2024-04-26T02:28:16.195 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T02:28:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.194+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/829604293 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5fb006f400 msgr2=0x7f5fb0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.194+0000 7f5fd4d3f640 1 --2- 172.21.15.94:0/829604293 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5fb006f400 0x7f5fb0071910 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40af000 tx=0x7f5fc4004740 comp rx=0 tx=0).stop 2024-04-26T02:28:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.194+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/829604293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd00b1890 msgr2=0x7f5fd01419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.194+0000 7f5fd4d3f640 1 --2- 172.21.15.94:0/829604293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd00b1890 0x7f5fd01419a0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000c760 tx=0x7f5fc000cc20 comp rx=0 tx=0).stop 2024-04-26T02:28:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.194+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/829604293 shutdown_connections 2024-04-26T02:28:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.194+0000 7f5fd4d3f640 1 --2- 172.21.15.94:0/829604293 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f5fb006f400 0x7f5fb0071910 secure :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40af000 tx=0x7f5fc4004740 comp rx=0 tx=0).stop 2024-04-26T02:28:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.194+0000 7f5fd4d3f640 1 --2- 172.21.15.94:0/829604293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5fd00b3350 0x7f5fd00b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.194+0000 7f5fd4d3f640 1 --2- 172.21.15.94:0/829604293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fd01607b0 0x7f5fd0159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.194+0000 7f5fd4d3f640 1 --2- 172.21.15.94:0/829604293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd00b1890 0x7f5fd01419a0 secure :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000c760 tx=0x7f5fc000cc20 comp rx=0 tx=0).stop 2024-04-26T02:28:16.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.194+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/829604293 >> 172.21.15.94:0/829604293 conn(0x7f5fd0071f00 msgr2=0x7f5fd00b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.206+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/829604293 shutdown_connections 2024-04-26T02:28:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.206+0000 7f5fd4d3f640 1 -- 172.21.15.94:0/829604293 wait complete. 2024-04-26T02:28:16.243 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T02:28:16.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.390+0000 7fd11723f640 1 Processor -- start 2024-04-26T02:28:16.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.390+0000 7fd11723f640 1 -- start start 2024-04-26T02:28:16.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.390+0000 7fd11723f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd110072b60 0x7fd11015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:16.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.390+0000 7fd11723f640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd110160900 con 0x7fd11015cd60 2024-04-26T02:28:16.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.390+0000 7fd11723f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd11006f360 con 0x7fd1100afa50 2024-04-26T02:28:16.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.390+0000 7fd11723f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd1100703e0 con 0x7fd110072b60 2024-04-26T02:28:16.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.390+0000 7fd10ffff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd11015cd60 0x7fd11015fae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:52460/0 (socket says 172.21.15.94:52460) 2024-04-26T02:28:16.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.390+0000 7fd114fdb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd110072b60 0x7fd11015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:16.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.390+0000 7fd10ffff640 1 -- 172.21.15.94:0/3482621258 learned_addr learned my addr 172.21.15.94:0/3482621258 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:16.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.390+0000 7fd114fdb640 1 -- 172.21.15.94:0/3482621258 >> v1:172.21.15.138:6789/0 conn(0x7fd1100afa50 legacy=0x7fd1101601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:16.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.390+0000 7fd114fdb640 1 -- 172.21.15.94:0/3482621258 >> v1:172.21.15.94:6789/0 conn(0x7fd11015cd60 legacy=0x7fd11015fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:16.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.390+0000 7fd10f7fe640 1 -- 172.21.15.94:0/3482621258 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3594817826 0 0) 0x7fd110160900 con 0x7fd11015cd60 2024-04-26T02:28:16.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.390+0000 7fd114fdb640 1 -- 172.21.15.94:0/3482621258 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11006f620 con 0x7fd110072b60 2024-04-26T02:28:16.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.394+0000 7fd114fdb640 1 --2- 172.21.15.94:0/3482621258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd110072b60 0x7fd11015c820 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fd1040097c0 tx=0x7fd10401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1914b01ed659f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:16.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.394+0000 7fd10f7fe640 1 -- 172.21.15.94:0/3482621258 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1040b1070 con 0x7fd110072b60 2024-04-26T02:28:16.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.394+0000 7fd10f7fe640 1 -- 172.21.15.94:0/3482621258 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1040a8050 con 0x7fd110072b60 2024-04-26T02:28:16.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.394+0000 7fd10f7fe640 1 -- 172.21.15.94:0/3482621258 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1040ac860 con 0x7fd110072b60 2024-04-26T02:28:16.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.394+0000 7fd11723f640 1 -- 172.21.15.94:0/3482621258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd110072b60 msgr2=0x7fd11015c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:16.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.394+0000 7fd11723f640 1 --2- 172.21.15.94:0/3482621258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd110072b60 0x7fd11015c820 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fd1040097c0 tx=0x7fd10401c3e0 comp rx=0 tx=0).stop 2024-04-26T02:28:16.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.394+0000 7fd11723f640 1 -- 172.21.15.94:0/3482621258 shutdown_connections 2024-04-26T02:28:16.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.394+0000 7fd11723f640 1 --2- 172.21.15.94:0/3482621258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd110072b60 0x7fd11015c820 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:16.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.394+0000 7fd11723f640 1 -- 172.21.15.94:0/3482621258 >> 172.21.15.94:0/3482621258 conn(0x7fd1100b47f0 msgr2=0x7fd1100b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:16.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.394+0000 7fd11723f640 1 -- 172.21.15.94:0/3482621258 shutdown_connections 2024-04-26T02:28:16.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.394+0000 7fd11723f640 1 -- 172.21.15.94:0/3482621258 wait complete. 2024-04-26T02:28:16.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.394+0000 7fd11723f640 1 Processor -- start 2024-04-26T02:28:16.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.394+0000 7fd11723f640 1 -- start start 2024-04-26T02:28:16.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd11723f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd110072b60 0x7fd11009a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:16.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd11723f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11015cd60 0x7fd11014ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:16.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd11723f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1101568c0 0x7fd11014d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd11723f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1100730f0 con 0x7fd11015cd60 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd11723f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd110072f70 con 0x7fd1101568c0 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd11723f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd110159cc0 con 0x7fd110072b60 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd10ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11015cd60 0x7fd11014ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd10ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11015cd60 0x7fd11014ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54442/0 (socket says 172.21.15.94:54442) 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd10ffff640 1 -- 172.21.15.94:0/3676364562 learned_addr learned my addr 172.21.15.94:0/3676364562 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd1157dc640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1101568c0 0x7fd11014d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd114fdb640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd110072b60 0x7fd11009a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd10ffff640 1 -- 172.21.15.94:0/3676364562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd110072b60 msgr2=0x7fd11009a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd10ffff640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd110072b60 0x7fd11009a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd10ffff640 1 -- 172.21.15.94:0/3676364562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1101568c0 msgr2=0x7fd11014d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd10ffff640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1101568c0 0x7fd11014d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd10ffff640 1 -- 172.21.15.94:0/3676364562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd110158f50 con 0x7fd11015cd60 2024-04-26T02:28:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd10ffff640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11015cd60 0x7fd11014ced0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00c440 tx=0x7fd0fc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:16.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd10d7fa640 1 -- 172.21.15.94:0/3676364562 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc018070 con 0x7fd11015cd60 2024-04-26T02:28:16.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd10d7fa640 1 -- 172.21.15.94:0/3676364562 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0fc0074e0 con 0x7fd11015cd60 2024-04-26T02:28:16.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd11723f640 1 -- 172.21.15.94:0/3676364562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd110158340 con 0x7fd11015cd60 2024-04-26T02:28:16.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd10d7fa640 1 -- 172.21.15.94:0/3676364562 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc004a10 con 0x7fd11015cd60 2024-04-26T02:28:16.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd11723f640 1 -- 172.21.15.94:0/3676364562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd110158900 con 0x7fd11015cd60 2024-04-26T02:28:16.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd11723f640 1 -- 172.21.15.94:0/3676364562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd110076830 con 0x7fd11015cd60 2024-04-26T02:28:16.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.398+0000 7fd10d7fa640 1 -- 172.21.15.94:0/3676364562 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd0fc01b070 con 0x7fd11015cd60 2024-04-26T02:28:16.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.402+0000 7fd10d7fa640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd0f406f400 0x7fd0f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:16.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.402+0000 7fd114fdb640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd0f406f400 0x7fd0f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:16.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.402+0000 7fd10d7fa640 1 -- 172.21.15.94:0/3676364562 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fd0fc08e6d0 con 0x7fd11015cd60 2024-04-26T02:28:16.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.402+0000 7fd114fdb640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd0f406f400 0x7fd0f4071910 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd1040ae000 tx=0x7fd104008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:16.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.406+0000 7fd10d7fa640 1 -- 172.21.15.94:0/3676364562 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd0fc05b450 con 0x7fd11015cd60 2024-04-26T02:28:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:16.594+0000 7fd11723f640 1 -- 172.21.15.94:0/3676364562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd110158090 con 0x7fd11015cd60 2024-04-26T02:28:16.752 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-04-26T02:28:17.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.194+0000 7fd10d7fa640 1 -- 172.21.15.94:0/3676364562 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7fd0fc08ce20 con 0x7fd11015cd60 2024-04-26T02:28:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.198+0000 7fd11723f640 1 -- 172.21.15.94:0/3676364562 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd0f406f400 msgr2=0x7fd0f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.198+0000 7fd11723f640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd0f406f400 0x7fd0f4071910 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd1040ae000 tx=0x7fd104008040 comp rx=0 tx=0).stop 2024-04-26T02:28:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.198+0000 7fd11723f640 1 -- 172.21.15.94:0/3676364562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11015cd60 msgr2=0x7fd11014ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.198+0000 7fd11723f640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11015cd60 0x7fd11014ced0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00c440 tx=0x7fd0fc00c900 comp rx=0 tx=0).stop 2024-04-26T02:28:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.198+0000 7fd11723f640 1 -- 172.21.15.94:0/3676364562 shutdown_connections 2024-04-26T02:28:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.198+0000 7fd11723f640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1101568c0 0x7fd11014d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.198+0000 7fd11723f640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd11015cd60 0x7fd11014ced0 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.198+0000 7fd11723f640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fd0f406f400 0x7fd0f4071910 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.198+0000 7fd11723f640 1 --2- 172.21.15.94:0/3676364562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd110072b60 0x7fd11009a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.198+0000 7fd11723f640 1 -- 172.21.15.94:0/3676364562 >> 172.21.15.94:0/3676364562 conn(0x7fd1100b47f0 msgr2=0x7fd11015f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.198+0000 7fd11723f640 1 -- 172.21.15.94:0/3676364562 shutdown_connections 2024-04-26T02:28:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.198+0000 7fd11723f640 1 -- 172.21.15.94:0/3676364562 wait complete. 2024-04-26T02:28:17.231 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T02:28:17.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22ab1a0640 1 Processor -- start 2024-04-26T02:28:17.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22ab1a0640 1 -- start start 2024-04-26T02:28:17.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22ab1a0640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f22a4073dc0 0x7f22a40741d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:17.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22ab1a0640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22a4073350 con 0x7f22a406d710 2024-04-26T02:28:17.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22ab1a0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22a40cfbf0 con 0x7f22a40cf3c0 2024-04-26T02:28:17.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22ab1a0640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f22a40d1210 con 0x7f22a4073dc0 2024-04-26T02:28:17.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22a973d640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f22a406d710 0x7f22a406db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:52484/0 (socket says 172.21.15.94:52484) 2024-04-26T02:28:17.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22a973d640 1 -- 172.21.15.94:0/4244845884 learned_addr learned my addr 172.21.15.94:0/4244845884 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:17.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2791078865 0 0) 0x7f22a4073350 con 0x7f22a406d710 2024-04-26T02:28:17.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22a3fff640 1 --2- 172.21.15.94:0/4244845884 >> v2:172.21.15.138:3300/0 conn(0x7f22a4073dc0 0x7f22a40741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f228c003530 con 0x7f22a406d710 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1728431686 0 0) 0x7f228c003530 con 0x7f22a406d710 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 >> v2:172.21.15.138:3300/0 conn(0x7f22a4073dc0 msgr2=0x7f22a40741d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22a37fe640 1 --2- 172.21.15.94:0/4244845884 >> v2:172.21.15.138:3300/0 conn(0x7f22a4073dc0 0x7f22a40741d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 >> v1:172.21.15.138:6789/0 conn(0x7f22a40cf3c0 legacy=0x7f22a40ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a40735c0 con 0x7f22a406d710 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f229c001a00 con 0x7f22a406d710 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.370+0000 7f22a3fff640 1 --2- 172.21.15.94:0/4244845884 >> v2:172.21.15.138:3300/0 conn(0x7f22a4073dc0 0x7f22a40741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 >> v1:172.21.15.94:6789/0 conn(0x7f22a406d710 legacy=0x7f22a406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a37fe640 1 --2- 172.21.15.94:0/4244845884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f228c003a30 0x7f228c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a37fe640 1 --2- 172.21.15.94:0/4244845884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f228c004810 0x7f228c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a37fe640 1 --2- 172.21.15.94:0/4244845884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f228c005360 0x7f228c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22a40d1210 con 0x7f228c003a30 2024-04-26T02:28:17.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f228c00c290 con 0x7f228c005360 2024-04-26T02:28:17.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f228c00c490 con 0x7f228c004810 2024-04-26T02:28:17.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a8f3c640 1 --2- 172.21.15.94:0/4244845884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f228c003a30 0x7f228c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:17.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a3fff640 1 --2- 172.21.15.94:0/4244845884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f228c004810 0x7f228c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:17.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a973d640 1 --2- 172.21.15.94:0/4244845884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f228c005360 0x7f228c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:17.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a8f3c640 1 -- 172.21.15.94:0/4244845884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f228c004810 msgr2=0x7f228c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:17.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a8f3c640 1 --2- 172.21.15.94:0/4244845884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f228c004810 0x7f228c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:17.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a8f3c640 1 -- 172.21.15.94:0/4244845884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f228c005360 msgr2=0x7f228c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:17.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a8f3c640 1 --2- 172.21.15.94:0/4244845884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f228c005360 0x7f228c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:17.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a973d640 1 -- 172.21.15.94:0/4244845884 reap_dead start 2024-04-26T02:28:17.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a8f3c640 1 -- 172.21.15.94:0/4244845884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c003530 con 0x7f228c003a30 2024-04-26T02:28:17.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a8f3c640 1 --2- 172.21.15.94:0/4244845884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f228c003a30 0x7f228c003ea0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f22a40c8440 tx=0x7f229401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf5e0531ae11737 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:17.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22940013a0 con 0x7f228c003a30 2024-04-26T02:28:17.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22940052f0 con 0x7f228c003a30 2024-04-26T02:28:17.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22a37fe640 1 -- 172.21.15.94:0/4244845884 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294003840 con 0x7f228c003a30 2024-04-26T02:28:17.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22ab1a0640 1 -- 172.21.15.94:0/4244845884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f228c003a30 msgr2=0x7f228c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:17.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22ab1a0640 1 --2- 172.21.15.94:0/4244845884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f228c003a30 0x7f228c003ea0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f22a40c8440 tx=0x7f229401c370 comp rx=0 tx=0).stop 2024-04-26T02:28:17.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22ab1a0640 1 -- 172.21.15.94:0/4244845884 shutdown_connections 2024-04-26T02:28:17.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22ab1a0640 1 --2- 172.21.15.94:0/4244845884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f228c003a30 0x7f228c003ea0 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:17.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22ab1a0640 1 -- 172.21.15.94:0/4244845884 >> 172.21.15.94:0/4244845884 conn(0x7f22a40d13c0 msgr2=0x7f22a40d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:17.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22ab1a0640 1 -- 172.21.15.94:0/4244845884 shutdown_connections 2024-04-26T02:28:17.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.374+0000 7f22ab1a0640 1 -- 172.21.15.94:0/4244845884 wait complete. 2024-04-26T02:28:17.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22ab1a0640 1 Processor -- start 2024-04-26T02:28:17.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22ab1a0640 1 -- start start 2024-04-26T02:28:17.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22ab1a0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f228c003a30 0x7f22a40caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:17.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22ab1a0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a41faab0 0x7f22a41faec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:17.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22ab1a0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a4159bf0 0x7f22a415a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:17.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22ab1a0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22a406df30 con 0x7f22a41faab0 2024-04-26T02:28:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22ab1a0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22a40745e0 con 0x7f22a4159bf0 2024-04-26T02:28:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22ab1a0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f22a4155fc0 con 0x7f228c003a30 2024-04-26T02:28:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a41faab0 0x7f22a41faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a41faab0 0x7f22a41faec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54470/0 (socket says 172.21.15.94:54470) 2024-04-26T02:28:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a3fff640 1 -- 172.21.15.94:0/1200107044 learned_addr learned my addr 172.21.15.94:0/1200107044 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a8f3c640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f228c003a30 0x7f22a40caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a973d640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a4159bf0 0x7f22a415a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a3fff640 1 -- 172.21.15.94:0/1200107044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f228c003a30 msgr2=0x7f22a40caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a3fff640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f228c003a30 0x7f22a40caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a3fff640 1 -- 172.21.15.94:0/1200107044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a4159bf0 msgr2=0x7f22a415a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a3fff640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a4159bf0 0x7f22a415a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a3fff640 1 -- 172.21.15.94:0/1200107044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a415a540 con 0x7f22a41faab0 2024-04-26T02:28:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a3fff640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a41faab0 0x7f22a41faec0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f2298010810 tx=0x7f2298010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a17fa640 1 -- 172.21.15.94:0/1200107044 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229801a020 con 0x7f22a41faab0 2024-04-26T02:28:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a17fa640 1 -- 172.21.15.94:0/1200107044 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f229800eb00 con 0x7f22a41faab0 2024-04-26T02:28:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22a17fa640 1 -- 172.21.15.94:0/1200107044 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229800f040 con 0x7f22a41faab0 2024-04-26T02:28:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22ab1a0640 1 -- 172.21.15.94:0/1200107044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a41f74c0 con 0x7f22a41faab0 2024-04-26T02:28:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.378+0000 7f22ab1a0640 1 -- 172.21.15.94:0/1200107044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a41f7780 con 0x7f22a41faab0 2024-04-26T02:28:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.382+0000 7f22ab1a0640 1 -- 172.21.15.94:0/1200107044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a41fb500 con 0x7f22a41faab0 2024-04-26T02:28:17.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.382+0000 7f22a17fa640 1 -- 172.21.15.94:0/1200107044 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f229801d020 con 0x7f22a41faab0 2024-04-26T02:28:17.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.382+0000 7f22a17fa640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f229006f430 0x7f2290071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:17.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.382+0000 7f22a8f3c640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f229006f430 0x7f2290071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:17.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.382+0000 7f22a17fa640 1 -- 172.21.15.94:0/1200107044 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f2298090260 con 0x7f22a41faab0 2024-04-26T02:28:17.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.382+0000 7f22a8f3c640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f229006f430 0x7f2290071940 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f22a40b42c0 tx=0x7f22940011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:17.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.390+0000 7f22a17fa640 1 -- 172.21.15.94:0/1200107044 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2298024020 con 0x7f22a41faab0 2024-04-26T02:28:17.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:17.542+0000 7f22ab1a0640 1 -- 172.21.15.94:0/1200107044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f22a40cb420 con 0x7f22a41faab0 2024-04-26T02:28:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.206+0000 7f22a17fa640 1 -- 172.21.15.94:0/1200107044 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) ==== 181+0+0 (secure 0 0 0) 0x7f22a415a540 con 0x7f22a41faab0 2024-04-26T02:28:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-04-26T02:28:18.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.210+0000 7f22ab1a0640 1 -- 172.21.15.94:0/1200107044 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f229006f430 msgr2=0x7f2290071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:18.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.210+0000 7f22ab1a0640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f229006f430 0x7f2290071940 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f22a40b42c0 tx=0x7f22940011a0 comp rx=0 tx=0).stop 2024-04-26T02:28:18.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.210+0000 7f22ab1a0640 1 -- 172.21.15.94:0/1200107044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a41faab0 msgr2=0x7f22a41faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:18.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.210+0000 7f22ab1a0640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a41faab0 0x7f22a41faec0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f2298010810 tx=0x7f2298010cd0 comp rx=0 tx=0).stop 2024-04-26T02:28:18.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.210+0000 7f22ab1a0640 1 -- 172.21.15.94:0/1200107044 shutdown_connections 2024-04-26T02:28:18.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.210+0000 7f22ab1a0640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a4159bf0 0x7f22a415a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:18.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.210+0000 7f22ab1a0640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a41faab0 0x7f22a41faec0 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:18.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.210+0000 7f22ab1a0640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f229006f430 0x7f2290071940 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:18.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.210+0000 7f22ab1a0640 1 --2- 172.21.15.94:0/1200107044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f228c003a30 0x7f22a40caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:18.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.210+0000 7f22ab1a0640 1 -- 172.21.15.94:0/1200107044 >> 172.21.15.94:0/1200107044 conn(0x7f22a40d13c0 msgr2=0x7f22a415f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:18.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.210+0000 7f22ab1a0640 1 -- 172.21.15.94:0/1200107044 shutdown_connections 2024-04-26T02:28:18.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.210+0000 7f22ab1a0640 1 -- 172.21.15.94:0/1200107044 wait complete. 2024-04-26T02:28:18.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T02:28:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.482+0000 7f3766dfe640 1 Processor -- start 2024-04-26T02:28:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.482+0000 7f3766dfe640 1 -- start start 2024-04-26T02:28:18.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.482+0000 7f3766dfe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f376015c870 0x7f37600b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:18.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.486+0000 7f3766dfe640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f37600ce150 0x7f37600ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:18.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.486+0000 7f3766dfe640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f376006e060 con 0x7f37601603f0 2024-04-26T02:28:18.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.486+0000 7f3766dfe640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f37600ceaa0 con 0x7f37600ce150 2024-04-26T02:28:18.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.486+0000 7f3766dfe640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f376015cc80 con 0x7f376015c870 2024-04-26T02:28:18.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.486+0000 7f375ffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f37600ce150 0x7f37600ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:18.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.486+0000 7f375ffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f37600ce150 0x7f37600ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50164/0 (socket says 172.21.15.94:50164) 2024-04-26T02:28:18.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.486+0000 7f375ffff640 1 -- 172.21.15.94:0/240093439 learned_addr learned my addr 172.21.15.94:0/240093439 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:18.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.486+0000 7f3764b9a640 1 --2- 172.21.15.94:0/240093439 >> v2:172.21.15.138:3300/0 conn(0x7f376015c870 0x7f37600b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:18.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.486+0000 7f375ffff640 1 -- 172.21.15.94:0/240093439 >> v2:172.21.15.138:3300/0 conn(0x7f376015c870 msgr2=0x7f37600b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:18.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.486+0000 7f375ffff640 1 --2- 172.21.15.94:0/240093439 >> v2:172.21.15.138:3300/0 conn(0x7f376015c870 0x7f37600b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:18.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.486+0000 7f375ffff640 1 -- 172.21.15.94:0/240093439 >> v1:172.21.15.94:6789/0 conn(0x7f37601603f0 legacy=0x7f3760160800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:28:18.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.486+0000 7f375ffff640 1 -- 172.21.15.94:0/240093439 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37600b4530 con 0x7f37600ce150 2024-04-26T02:28:18.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.490+0000 7f375ffff640 1 --2- 172.21.15.94:0/240093439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37600ce150 0x7f37600ce560 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f37600d2230 tx=0x7f375801eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2114718706018b41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:18.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.494+0000 7f375f7fe640 1 -- 172.21.15.94:0/240093439 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375800b2d0 con 0x7f37600ce150 2024-04-26T02:28:18.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.494+0000 7f375f7fe640 1 -- 172.21.15.94:0/240093439 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375800b470 con 0x7f37600ce150 2024-04-26T02:28:18.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.494+0000 7f375f7fe640 1 -- 172.21.15.94:0/240093439 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37580a7a00 con 0x7f37600ce150 2024-04-26T02:28:18.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.498+0000 7f3766dfe640 1 -- 172.21.15.94:0/240093439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37600ce150 msgr2=0x7f37600ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:18.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.498+0000 7f3766dfe640 1 --2- 172.21.15.94:0/240093439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37600ce150 0x7f37600ce560 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f37600d2230 tx=0x7f375801eb20 comp rx=0 tx=0).stop 2024-04-26T02:28:18.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.498+0000 7f3766dfe640 1 -- 172.21.15.94:0/240093439 shutdown_connections 2024-04-26T02:28:18.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.498+0000 7f3766dfe640 1 --2- 172.21.15.94:0/240093439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37600ce150 0x7f37600ce560 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:18.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.498+0000 7f3766dfe640 1 --2- 172.21.15.94:0/240093439 >> v2:172.21.15.138:3300/0 conn(0x7f376015c870 0x7f37600b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:18.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.498+0000 7f3766dfe640 1 -- 172.21.15.94:0/240093439 >> 172.21.15.94:0/240093439 conn(0x7f37600b2cf0 msgr2=0x7f37600b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:18.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.502+0000 7f3766dfe640 1 -- 172.21.15.94:0/240093439 shutdown_connections 2024-04-26T02:28:18.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.506+0000 7f3766dfe640 1 -- 172.21.15.94:0/240093439 wait complete. 2024-04-26T02:28:18.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.510+0000 7f3766dfe640 1 Processor -- start 2024-04-26T02:28:18.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.510+0000 7f3766dfe640 1 -- start start 2024-04-26T02:28:18.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.510+0000 7f3766dfe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37600ce150 0x7f376014d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:18.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.510+0000 7f3766dfe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376015c870 0x7f37600735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:18.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.510+0000 7f3766dfe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37601603f0 0x7f3760073b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:18.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.510+0000 7f3766dfe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f37600ae770 con 0x7f37600ce150 2024-04-26T02:28:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.510+0000 7f3766dfe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f376006d5b0 con 0x7f376015c870 2024-04-26T02:28:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.510+0000 7f3766dfe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f37600ae8f0 con 0x7f37601603f0 2024-04-26T02:28:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.514+0000 7f376539b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37601603f0 0x7f3760073b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.514+0000 7f376539b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37601603f0 0x7f3760073b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:56016/0 (socket says 172.21.15.94:56016) 2024-04-26T02:28:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.514+0000 7f376539b640 1 -- 172.21.15.94:0/3386373920 learned_addr learned my addr 172.21.15.94:0/3386373920 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.514+0000 7f375ffff640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376015c870 0x7f37600735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:18.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.514+0000 7f375ffff640 1 -- 172.21.15.94:0/3386373920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37601603f0 msgr2=0x7f3760073b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:18.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.514+0000 7f375ffff640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37601603f0 0x7f3760073b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:18.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.514+0000 7f375ffff640 1 -- 172.21.15.94:0/3386373920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37600ce150 msgr2=0x7f376014d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:18.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.514+0000 7f375ffff640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37600ce150 0x7f376014d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:18.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.514+0000 7f375ffff640 1 -- 172.21.15.94:0/3386373920 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376014ff70 con 0x7f376015c870 2024-04-26T02:28:18.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.514+0000 7f376539b640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37601603f0 0x7f3760073b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:28:18.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.514+0000 7f375ffff640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376015c870 0x7f37600735d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f37600b39a0 tx=0x7f375801b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:18.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.522+0000 7f375d7fa640 1 -- 172.21.15.94:0/3386373920 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37580b8a50 con 0x7f376015c870 2024-04-26T02:28:18.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.522+0000 7f3766dfe640 1 -- 172.21.15.94:0/3386373920 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3760150210 con 0x7f376015c870 2024-04-26T02:28:18.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.522+0000 7f3766dfe640 1 -- 172.21.15.94:0/3386373920 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3760151820 con 0x7f376015c870 2024-04-26T02:28:18.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.526+0000 7f3766dfe640 1 -- 172.21.15.94:0/3386373920 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3760076830 con 0x7f376015c870 2024-04-26T02:28:18.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.526+0000 7f375d7fa640 1 -- 172.21.15.94:0/3386373920 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37580c0070 con 0x7f376015c870 2024-04-26T02:28:18.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.526+0000 7f375d7fa640 1 -- 172.21.15.94:0/3386373920 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37580b8bf0 con 0x7f376015c870 2024-04-26T02:28:18.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.530+0000 7f375d7fa640 1 -- 172.21.15.94:0/3386373920 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3758009600 con 0x7f376015c870 2024-04-26T02:28:18.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.530+0000 7f375d7fa640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f374806f560 0x7f3748071a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:18.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.530+0000 7f375d7fa640 1 -- 172.21.15.94:0/3386373920 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5778+0+0 (secure 0 0 0) 0x7f375812e6a0 con 0x7f376015c870 2024-04-26T02:28:18.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.530+0000 7f3764b9a640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f374806f560 0x7f3748071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:18.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.530+0000 7f3764b9a640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f374806f560 0x7f3748071a70 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3754006aa0 tx=0x7f3754006a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:18.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.606+0000 7f375d7fa640 1 -- 172.21.15.94:0/3386373920 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f37580187a0 con 0x7f376015c870 2024-04-26T02:28:18.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:18.774+0000 7f3766dfe640 1 -- 172.21.15.94:0/3386373920 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f376014fc90 con 0x7f376015c870 2024-04-26T02:28:19.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.250+0000 7f375d7fa640 1 -- 172.21.15.94:0/3386373920 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) ==== 169+0+0 (secure 0 0 0) 0x7f37580fb4a0 con 0x7f376015c870 2024-04-26T02:28:19.257 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-04-26T02:28:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.254+0000 7f3766dfe640 1 -- 172.21.15.94:0/3386373920 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f374806f560 msgr2=0x7f3748071a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.254+0000 7f3766dfe640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f374806f560 0x7f3748071a70 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3754006aa0 tx=0x7f3754006a30 comp rx=0 tx=0).stop 2024-04-26T02:28:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.254+0000 7f3766dfe640 1 -- 172.21.15.94:0/3386373920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376015c870 msgr2=0x7f37600735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.254+0000 7f3766dfe640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376015c870 0x7f37600735d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f37600b39a0 tx=0x7f375801b8c0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.254+0000 7f3766dfe640 1 -- 172.21.15.94:0/3386373920 shutdown_connections 2024-04-26T02:28:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.254+0000 7f3766dfe640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f374806f560 0x7f3748071a70 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.254+0000 7f3766dfe640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37601603f0 0x7f3760073b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.254+0000 7f3766dfe640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376015c870 0x7f37600735d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.254+0000 7f3766dfe640 1 --2- 172.21.15.94:0/3386373920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37600ce150 0x7f376014d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.254+0000 7f3766dfe640 1 -- 172.21.15.94:0/3386373920 >> 172.21.15.94:0/3386373920 conn(0x7f37600b2cf0 msgr2=0x7f3760160800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:19.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.258+0000 7f3766dfe640 1 -- 172.21.15.94:0/3386373920 shutdown_connections 2024-04-26T02:28:19.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.258+0000 7f3766dfe640 1 -- 172.21.15.94:0/3386373920 wait complete. 2024-04-26T02:28:19.293 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T02:28:19.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.458+0000 7f1c8f31c640 1 Processor -- start 2024-04-26T02:28:19.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.458+0000 7f1c8f31c640 1 -- start start 2024-04-26T02:28:19.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.458+0000 7f1c8f31c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c880b7ae0 0x7f1c881644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:19.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.458+0000 7f1c8f31c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1c881649f0 0x7f1c88164e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:19.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.458+0000 7f1c8f31c640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c880cfe80 con 0x7f1c880d2b00 2024-04-26T02:28:19.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.458+0000 7f1c8f31c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1c880d0160 con 0x7f1c881649f0 2024-04-26T02:28:19.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.458+0000 7f1c8f31c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c880d0440 con 0x7f1c880b7ae0 2024-04-26T02:28:19.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.458+0000 7f1c8d0b8640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f1c880d2b00 0x7f1c880b73d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56062/0 (socket says 172.21.15.94:56062) 2024-04-26T02:28:19.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.458+0000 7f1c8d0b8640 1 -- 172.21.15.94:0/2881193676 learned_addr learned my addr 172.21.15.94:0/2881193676 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:19.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 321040385 0 0) 0x7f1c880cfe80 con 0x7f1c880d2b00 2024-04-26T02:28:19.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c74003530 con 0x7f1c880d2b00 2024-04-26T02:28:19.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2726282567 0 0) 0x7f1c74003530 con 0x7f1c880d2b00 2024-04-26T02:28:19.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c880b7ae0 msgr2=0x7f1c881644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:19.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 --2- 172.21.15.94:0/2881193676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c880b7ae0 0x7f1c881644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 >> v2:172.21.15.138:3300/0 conn(0x7f1c881649f0 msgr2=0x7f1c88164e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:19.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 --2- 172.21.15.94:0/2881193676 >> v2:172.21.15.138:3300/0 conn(0x7f1c881649f0 0x7f1c88164e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c880d0720 con 0x7f1c880d2b00 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f1c84001680 con 0x7f1c880d2b00 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 >> v1:172.21.15.94:6789/0 conn(0x7f1c880d2b00 legacy=0x7f1c880b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 --2- 172.21.15.94:0/2881193676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c880b7ae0 0x7f1c740039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 --2- 172.21.15.94:0/2881193676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c740040d0 0x7f1c74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 --2- 172.21.15.94:0/2881193676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c74004e70 0x7f1c740052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c880d0160 con 0x7f1c740040d0 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c880d0440 con 0x7f1c74004e70 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c7400bd30 con 0x7f1c880b7ae0 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c8c8b7640 1 --2- 172.21.15.94:0/2881193676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c740040d0 0x7f1c74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c8c8b7640 1 -- 172.21.15.94:0/2881193676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c880b7ae0 msgr2=0x7f1c740039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c8c8b7640 1 --2- 172.21.15.94:0/2881193676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c880b7ae0 0x7f1c740039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c8c8b7640 1 -- 172.21.15.94:0/2881193676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c74004e70 msgr2=0x7f1c740052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c8c8b7640 1 --2- 172.21.15.94:0/2881193676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c74004e70 0x7f1c740052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.462+0000 7f1c8c8b7640 1 -- 172.21.15.94:0/2881193676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c74003530 con 0x7f1c740040d0 2024-04-26T02:28:19.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.466+0000 7f1c8c8b7640 1 --2- 172.21.15.94:0/2881193676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c740040d0 0x7f1c74004810 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009ca0 tx=0x7f1c8001cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654d65a963668e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:19.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.466+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c800b2070 con 0x7f1c740040d0 2024-04-26T02:28:19.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.466+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c800a5ca0 con 0x7f1c740040d0 2024-04-26T02:28:19.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.466+0000 7f1c7ffff640 1 -- 172.21.15.94:0/2881193676 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80006e60 con 0x7f1c740040d0 2024-04-26T02:28:19.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.466+0000 7f1c8f31c640 1 -- 172.21.15.94:0/2881193676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c740040d0 msgr2=0x7f1c74004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:19.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.466+0000 7f1c8f31c640 1 --2- 172.21.15.94:0/2881193676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c740040d0 0x7f1c74004810 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009ca0 tx=0x7f1c8001cc70 comp rx=0 tx=0).stop 2024-04-26T02:28:19.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.466+0000 7f1c8d8b9640 1 -- 172.21.15.94:0/2881193676 reap_dead start 2024-04-26T02:28:19.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.466+0000 7f1c8f31c640 1 -- 172.21.15.94:0/2881193676 shutdown_connections 2024-04-26T02:28:19.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.466+0000 7f1c8f31c640 1 -- 172.21.15.94:0/2881193676 >> 172.21.15.94:0/2881193676 conn(0x7f1c880d33d0 msgr2=0x7f1c880d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:19.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.466+0000 7f1c8f31c640 1 -- 172.21.15.94:0/2881193676 shutdown_connections 2024-04-26T02:28:19.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.466+0000 7f1c8f31c640 1 -- 172.21.15.94:0/2881193676 wait complete. 2024-04-26T02:28:19.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.466+0000 7f1c8f31c640 1 Processor -- start 2024-04-26T02:28:19.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.466+0000 7f1c8f31c640 1 -- start start 2024-04-26T02:28:19.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8f31c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c740040d0 0x7f1c88155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:19.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8f31c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c880ca950 0x7f1c880cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:19.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8f31c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88160390 0x7f1c881607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:19.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8f31c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8806ed20 con 0x7f1c88160390 2024-04-26T02:28:19.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8f31c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8806e9b0 con 0x7f1c880ca950 2024-04-26T02:28:19.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8f31c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c88165340 con 0x7f1c740040d0 2024-04-26T02:28:19.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8d8b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88160390 0x7f1c881607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:19.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8d8b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88160390 0x7f1c881607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50194/0 (socket says 172.21.15.94:50194) 2024-04-26T02:28:19.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8d8b9640 1 -- 172.21.15.94:0/3802306676 learned_addr learned my addr 172.21.15.94:0/3802306676 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:19.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8d8b9640 1 -- 172.21.15.94:0/3802306676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c740040d0 msgr2=0x7f1c88155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:19.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8d8b9640 1 --2- 172.21.15.94:0/3802306676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c740040d0 0x7f1c88155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8d8b9640 1 -- 172.21.15.94:0/3802306676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c880ca950 msgr2=0x7f1c880cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:19.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8d8b9640 1 --2- 172.21.15.94:0/3802306676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c880ca950 0x7f1c880cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8d8b9640 1 -- 172.21.15.94:0/3802306676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c88160ce0 con 0x7f1c88160390 2024-04-26T02:28:19.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8d8b9640 1 --2- 172.21.15.94:0/3802306676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88160390 0x7f1c881607a0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f1c880d32b0 tx=0x7f1c7800a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:19.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c7dffb640 1 -- 172.21.15.94:0/3802306676 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78005de0 con 0x7f1c88160390 2024-04-26T02:28:19.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8f31c640 1 -- 172.21.15.94:0/3802306676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c880c9020 con 0x7f1c88160390 2024-04-26T02:28:19.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c8f31c640 1 -- 172.21.15.94:0/3802306676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c880c95e0 con 0x7f1c88160390 2024-04-26T02:28:19.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c7dffb640 1 -- 172.21.15.94:0/3802306676 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c78007070 con 0x7f1c88160390 2024-04-26T02:28:19.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.470+0000 7f1c7dffb640 1 -- 172.21.15.94:0/3802306676 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c780074a0 con 0x7f1c88160390 2024-04-26T02:28:19.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.474+0000 7f1c8f31c640 1 -- 172.21.15.94:0/3802306676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c680052a0 con 0x7f1c88160390 2024-04-26T02:28:19.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.474+0000 7f1c7dffb640 1 -- 172.21.15.94:0/3802306676 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1c78011070 con 0x7f1c88160390 2024-04-26T02:28:19.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.474+0000 7f1c7dffb640 1 --2- 172.21.15.94:0/3802306676 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1c6c06f400 0x7f1c6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:19.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.474+0000 7f1c7dffb640 1 -- 172.21.15.94:0/3802306676 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f1c7808e530 con 0x7f1c88160390 2024-04-26T02:28:19.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.474+0000 7f1c8d0b8640 1 --2- 172.21.15.94:0/3802306676 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1c6c06f400 0x7f1c6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:19.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.474+0000 7f1c8d0b8640 1 --2- 172.21.15.94:0/3802306676 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1c6c06f400 0x7f1c6c071910 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1c84001060 tx=0x7f1c840014c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:19.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.478+0000 7f1c7dffb640 1 -- 172.21.15.94:0/3802306676 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c7801a060 con 0x7f1c88160390 2024-04-26T02:28:19.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.778+0000 7f1c8f31c640 1 -- 172.21.15.94:0/3802306676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1c68005860 con 0x7f1c88160390 2024-04-26T02:28:19.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.802+0000 7f1c7dffb640 1 -- 172.21.15.94:0/3802306676 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f1c88160ce0 con 0x7f1c88160390 2024-04-26T02:28:19.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.806+0000 7f1c737fe640 1 -- 172.21.15.94:0/3802306676 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1c6c06f400 msgr2=0x7f1c6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:19.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.806+0000 7f1c737fe640 1 --2- 172.21.15.94:0/3802306676 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1c6c06f400 0x7f1c6c071910 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1c84001060 tx=0x7f1c840014c0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.806+0000 7f1c737fe640 1 -- 172.21.15.94:0/3802306676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88160390 msgr2=0x7f1c881607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:19.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.806+0000 7f1c737fe640 1 --2- 172.21.15.94:0/3802306676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88160390 0x7f1c881607a0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f1c880d32b0 tx=0x7f1c7800a800 comp rx=0 tx=0).stop 2024-04-26T02:28:19.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.806+0000 7f1c737fe640 1 -- 172.21.15.94:0/3802306676 shutdown_connections 2024-04-26T02:28:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.806+0000 7f1c737fe640 1 --2- 172.21.15.94:0/3802306676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88160390 0x7f1c881607a0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.806+0000 7f1c737fe640 1 --2- 172.21.15.94:0/3802306676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c880ca950 0x7f1c880cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.806+0000 7f1c737fe640 1 --2- 172.21.15.94:0/3802306676 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1c6c06f400 0x7f1c6c071910 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.806+0000 7f1c737fe640 1 --2- 172.21.15.94:0/3802306676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c740040d0 0x7f1c88155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.806+0000 7f1c737fe640 1 -- 172.21.15.94:0/3802306676 >> 172.21.15.94:0/3802306676 conn(0x7f1c880d33d0 msgr2=0x7f1c880b7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.806+0000 7f1c737fe640 1 -- 172.21.15.94:0/3802306676 shutdown_connections 2024-04-26T02:28:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:19.806+0000 7f1c737fe640 1 -- 172.21.15.94:0/3802306676 wait complete. 2024-04-26T02:28:19.841 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-04-26T02:28:20.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.062+0000 7fa3235c3640 1 Processor -- start 2024-04-26T02:28:20.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.062+0000 7fa3235c3640 1 -- start start 2024-04-26T02:28:20.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa3235c3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa31c073150 0x7fa31c167c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:20.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa3235c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c073560 0x7fa31c168180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:20.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa3235c3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa31c06d1b0 con 0x7fa31c164610 2024-04-26T02:28:20.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa3235c3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa31c0ae170 con 0x7fa31c073150 2024-04-26T02:28:20.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa3235c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa31c06ddc0 con 0x7fa31c073560 2024-04-26T02:28:20.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa32135f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa31c073150 0x7fa31c167c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:20.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa32135f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa31c073150 0x7fa31c167c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:36538/0 (socket says 172.21.15.94:36538) 2024-04-26T02:28:20.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa32135f640 1 -- 172.21.15.94:0/3592607087 learned_addr learned my addr 172.21.15.94:0/3592607087 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:20.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa320b5e640 1 --2- 172.21.15.94:0/3592607087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c073560 0x7fa31c168180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa31bfff640 1 -- 172.21.15.94:0/3592607087 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 12085456 0 0) 0x7fa31c06d1b0 con 0x7fa31c164610 2024-04-26T02:28:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa31bfff640 1 -- 172.21.15.94:0/3592607087 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa304003530 con 0x7fa31c164610 2024-04-26T02:28:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa32135f640 1 -- 172.21.15.94:0/3592607087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c073560 msgr2=0x7fa31c168180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:20.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa32135f640 1 --2- 172.21.15.94:0/3592607087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c073560 0x7fa31c168180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:20.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa32135f640 1 -- 172.21.15.94:0/3592607087 >> v1:172.21.15.138:6789/0 conn(0x7fa31c164610 legacy=0x7fa31c164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:20.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.066+0000 7fa32135f640 1 -- 172.21.15.94:0/3592607087 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c06d490 con 0x7fa31c073150 2024-04-26T02:28:20.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.070+0000 7fa32135f640 1 --2- 172.21.15.94:0/3592607087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c073150 0x7fa31c167c40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009e50 tx=0x7fa30c01f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c407b9649e22c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.074+0000 7fa31bfff640 1 -- 172.21.15.94:0/3592607087 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0b4070 con 0x7fa31c073150 2024-04-26T02:28:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.074+0000 7fa31bfff640 1 -- 172.21.15.94:0/3592607087 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30c0af400 con 0x7fa31c073150 2024-04-26T02:28:20.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.074+0000 7fa31bfff640 1 -- 172.21.15.94:0/3592607087 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0af7d0 con 0x7fa31c073150 2024-04-26T02:28:20.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.074+0000 7fa3235c3640 1 -- 172.21.15.94:0/3592607087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c073150 msgr2=0x7fa31c167c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:20.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.074+0000 7fa3235c3640 1 --2- 172.21.15.94:0/3592607087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c073150 0x7fa31c167c40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009e50 tx=0x7fa30c01f1e0 comp rx=0 tx=0).stop 2024-04-26T02:28:20.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.078+0000 7fa3235c3640 1 -- 172.21.15.94:0/3592607087 shutdown_connections 2024-04-26T02:28:20.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.078+0000 7fa3235c3640 1 --2- 172.21.15.94:0/3592607087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c073560 0x7fa31c168180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:20.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.078+0000 7fa3235c3640 1 --2- 172.21.15.94:0/3592607087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c073150 0x7fa31c167c40 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:20.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.078+0000 7fa3235c3640 1 -- 172.21.15.94:0/3592607087 >> 172.21.15.94:0/3592607087 conn(0x7fa31c0b47f0 msgr2=0x7fa31c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:20.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.078+0000 7fa3235c3640 1 -- 172.21.15.94:0/3592607087 shutdown_connections 2024-04-26T02:28:20.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.078+0000 7fa3235c3640 1 -- 172.21.15.94:0/3592607087 wait complete. 2024-04-26T02:28:20.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.078+0000 7fa3235c3640 1 Processor -- start 2024-04-26T02:28:20.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.078+0000 7fa3235c3640 1 -- start start 2024-04-26T02:28:20.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa3235c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c073150 0x7fa31c1f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:20.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa3235c3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c073560 0x7fa31c0af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:20.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa3235c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c164610 0x7fa31c0af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:20.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa3235c3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1689a0 con 0x7fa31c073560 2024-04-26T02:28:20.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa3235c3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c06da50 con 0x7fa31c164610 2024-04-26T02:28:20.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa3235c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa31c06fcf0 con 0x7fa31c073150 2024-04-26T02:28:20.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa320b5e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c073560 0x7fa31c0af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:20.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa320b5e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c073560 0x7fa31c0af350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50208/0 (socket says 172.21.15.94:50208) 2024-04-26T02:28:20.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa320b5e640 1 -- 172.21.15.94:0/4159120815 learned_addr learned my addr 172.21.15.94:0/4159120815 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:20.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa32135f640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c073150 0x7fa31c1f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:20.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa321b60640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c164610 0x7fa31c0af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:20.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa320b5e640 1 -- 172.21.15.94:0/4159120815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c073150 msgr2=0x7fa31c1f9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:20.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa320b5e640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c073150 0x7fa31c1f9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:20.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa320b5e640 1 -- 172.21.15.94:0/4159120815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c164610 msgr2=0x7fa31c0af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:20.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa320b5e640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c164610 0x7fa31c0af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:20.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa320b5e640 1 -- 172.21.15.94:0/4159120815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c09f150 con 0x7fa31c073560 2024-04-26T02:28:20.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa32135f640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c073150 0x7fa31c1f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:28:20.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa321b60640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c164610 0x7fa31c0af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:28:20.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.082+0000 7fa320b5e640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c073560 0x7fa31c0af350 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fa31000d620 tx=0x7fa31000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:20.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.086+0000 7fa319ffb640 1 -- 172.21.15.94:0/4159120815 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31000b620 con 0x7fa31c073560 2024-04-26T02:28:20.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.086+0000 7fa319ffb640 1 -- 172.21.15.94:0/4159120815 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa310019070 con 0x7fa31c073560 2024-04-26T02:28:20.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.086+0000 7fa319ffb640 1 -- 172.21.15.94:0/4159120815 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31000b9c0 con 0x7fa31c073560 2024-04-26T02:28:20.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.086+0000 7fa3235c3640 1 -- 172.21.15.94:0/4159120815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31c09dee0 con 0x7fa31c073560 2024-04-26T02:28:20.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.086+0000 7fa3235c3640 1 -- 172.21.15.94:0/4159120815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31c09e460 con 0x7fa31c073560 2024-04-26T02:28:20.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.086+0000 7fa319ffb640 1 -- 172.21.15.94:0/4159120815 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa31000bb60 con 0x7fa31c073560 2024-04-26T02:28:20.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.086+0000 7fa3235c3640 1 -- 172.21.15.94:0/4159120815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2fc0052a0 con 0x7fa31c073560 2024-04-26T02:28:20.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.090+0000 7fa319ffb640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa30806f450 0x7fa308071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:20.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.090+0000 7fa319ffb640 1 -- 172.21.15.94:0/4159120815 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fa310058140 con 0x7fa31c073560 2024-04-26T02:28:20.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.090+0000 7fa32135f640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa30806f450 0x7fa308071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:20.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.090+0000 7fa32135f640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa30806f450 0x7fa308071960 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0b1000 tx=0x7fa30c0aa000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.098+0000 7fa319ffb640 1 -- 172.21.15.94:0/4159120815 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa31c09dee0 con 0x7fa31c073560 2024-04-26T02:28:20.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:20.746+0000 7fa3235c3640 1 -- 172.21.15.94:0/4159120815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) -- 0x7fa2fc005590 con 0x7fa31c073560 2024-04-26T02:28:21.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:21.242+0000 7fa319ffb640 1 -- 172.21.15.94:0/4159120815 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v17429) ==== 144+0+0 (secure 0 0 0) 0x7fa31005c5b0 con 0x7fa31c073560 2024-04-26T02:28:21.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:21.246+0000 7fa3235c3640 1 -- 172.21.15.94:0/4159120815 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa30806f450 msgr2=0x7fa308071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:21.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:21.246+0000 7fa3235c3640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa30806f450 0x7fa308071960 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0b1000 tx=0x7fa30c0aa000 comp rx=0 tx=0).stop 2024-04-26T02:28:21.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:21.246+0000 7fa3235c3640 1 -- 172.21.15.94:0/4159120815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c073560 msgr2=0x7fa31c0af350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:21.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:21.246+0000 7fa3235c3640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c073560 0x7fa31c0af350 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fa31000d620 tx=0x7fa31000dae0 comp rx=0 tx=0).stop 2024-04-26T02:28:21.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:21.250+0000 7fa3235c3640 1 -- 172.21.15.94:0/4159120815 shutdown_connections 2024-04-26T02:28:21.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:21.250+0000 7fa3235c3640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c164610 0x7fa31c0af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:21.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:21.250+0000 7fa3235c3640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c073560 0x7fa31c0af350 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:21.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:21.250+0000 7fa3235c3640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fa30806f450 0x7fa308071960 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:21.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:21.250+0000 7fa3235c3640 1 --2- 172.21.15.94:0/4159120815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31c073150 0x7fa31c1f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:21.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:21.250+0000 7fa3235c3640 1 -- 172.21.15.94:0/4159120815 >> 172.21.15.94:0/4159120815 conn(0x7fa31c0b47f0 msgr2=0x7fa31c15c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:21.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:21.250+0000 7fa3235c3640 1 -- 172.21.15.94:0/4159120815 shutdown_connections 2024-04-26T02:28:21.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:21.250+0000 7fa3235c3640 1 -- 172.21.15.94:0/4159120815 wait complete. 2024-04-26T02:28:21.597 INFO:tasks.cephfs_test_runner:test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) ... FAIL 2024-04-26T02:28:21.598 INFO:tasks.cephfs_test_runner: 2024-04-26T02:28:21.598 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-26T02:28:21.598 INFO:tasks.cephfs_test_runner:FAIL: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-04-26T02:28:21.598 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T02:28:21.598 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-26T02:28:21.598 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/tests_from_xfstests_dev.py", line 12, in test_generic 2024-04-26T02:28:21.598 INFO:tasks.cephfs_test_runner: self.run_generic_tests() 2024-04-26T02:28:21.599 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/xfstests_dev.py", line 407, in run_generic_tests 2024-04-26T02:28:21.599 INFO:tasks.cephfs_test_runner: return self.run_testdir('generic') 2024-04-26T02:28:21.599 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/xfstests_dev.py", line 399, in run_testdir 2024-04-26T02:28:21.599 INFO:tasks.cephfs_test_runner: self.assertEqual(self.total_tests_failed, 0) 2024-04-26T02:28:21.599 INFO:tasks.cephfs_test_runner:AssertionError: 17 != 0 2024-04-26T02:28:21.599 INFO:tasks.cephfs_test_runner: 2024-04-26T02:28:21.599 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T02:28:21.599 INFO:tasks.cephfs_test_runner:Ran 1 test in 34590.955s 2024-04-26T02:28:21.599 INFO:tasks.cephfs_test_runner: 2024-04-26T02:28:21.599 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2024-04-26T02:28:21.599 INFO:tasks.cephfs_test_runner: 2024-04-26T02:28:21.599 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-26T02:28:21.600 INFO:tasks.cephfs_test_runner:FAIL: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-04-26T02:28:21.600 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T02:28:21.600 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-26T02:28:21.600 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/tests_from_xfstests_dev.py", line 12, in test_generic 2024-04-26T02:28:21.600 INFO:tasks.cephfs_test_runner: self.run_generic_tests() 2024-04-26T02:28:21.600 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/xfstests_dev.py", line 407, in run_generic_tests 2024-04-26T02:28:21.600 INFO:tasks.cephfs_test_runner: return self.run_testdir('generic') 2024-04-26T02:28:21.600 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/xfstests_dev.py", line 399, in run_testdir 2024-04-26T02:28:21.600 INFO:tasks.cephfs_test_runner: self.assertEqual(self.total_tests_failed, 0) 2024-04-26T02:28:21.600 INFO:tasks.cephfs_test_runner:AssertionError: 17 != 0 2024-04-26T02:28:21.600 INFO:tasks.cephfs_test_runner: 2024-04-26T02:28:21.601 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-04-26T02:28:21.856 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a0837c9ee4e241a5b972501401419688 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-04-26T02:28:21.858 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-26T02:28:21.867 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-26T02:28:21.920 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-26T02:28:21.920 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T02:28:21.920 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:28:21.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16115 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T02:28:21.983 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-26T02:28:21.983 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:28:22.069 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16115 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T02:28:22.069 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-26T02:28:22.069 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:28:22.075 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16120 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T02:28:22.075 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-26T02:28:22.075 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:28:22.122 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16120 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T02:28:22.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:28:22.122 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-26T02:28:22.152 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T02:28:22.153 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T02:28:22.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:28:22.153 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T02:28:22.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:28:22.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T02:28:22.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:28:22.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-04-26T02:28:22.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:28:22.550 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-26T02:28:22.555 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T02:28:22.556 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T02:28:22.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:28:22.556 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T02:28:22.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:28:22.608 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T02:28:22.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:28:22.664 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-04-26T02:28:22.858 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T02:28:22.869 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T02:28:22.869 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T02:28:23.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49bdf6640 1 Processor -- start 2024-04-26T02:28:23.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49bdf6640 1 -- start start 2024-04-26T02:28:23.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49bdf6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff494161530 0x7ff4940b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49bdf6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4940d1390 0x7ff4940ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49bdf6640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4940b4b50 con 0x7ff49406e770 2024-04-26T02:28:23.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49bdf6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff4940b1230 con 0x7ff494161530 2024-04-26T02:28:23.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49bdf6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff4940ce220 con 0x7ff4940d1390 2024-04-26T02:28:23.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff499b92640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ff49406e770 0x7ff4940d0c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56068/0 (socket says 172.21.15.94:56068) 2024-04-26T02:28:23.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff499b92640 1 -- 172.21.15.94:0/2421833481 learned_addr learned my addr 172.21.15.94:0/2421833481 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:23.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff499391640 1 --2- 172.21.15.94:0/2421833481 >> v2:172.21.15.138:3300/0 conn(0x7ff494161530 0x7ff4940b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:23.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2953913809 0 0) 0x7ff4940b4b50 con 0x7ff49406e770 2024-04-26T02:28:23.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49a393640 1 --2- 172.21.15.94:0/2421833481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4940d1390 0x7ff4940ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:23.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff480003530 con 0x7ff49406e770 2024-04-26T02:28:23.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1758571289 0 0) 0x7ff480003530 con 0x7ff49406e770 2024-04-26T02:28:23.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4940d1390 msgr2=0x7ff4940ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:23.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 --2- 172.21.15.94:0/2421833481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4940d1390 0x7ff4940ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 >> v2:172.21.15.138:3300/0 conn(0x7ff494161530 msgr2=0x7ff4940b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:23.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 --2- 172.21.15.94:0/2421833481 >> v2:172.21.15.138:3300/0 conn(0x7ff494161530 0x7ff4940b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49415e440 con 0x7ff49406e770 2024-04-26T02:28:23.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7ff48c001a00 con 0x7ff49406e770 2024-04-26T02:28:23.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 >> v1:172.21.15.94:6789/0 conn(0x7ff49406e770 legacy=0x7ff4940d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:23.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 --2- 172.21.15.94:0/2421833481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4800039c0 0x7ff480003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 --2- 172.21.15.94:0/2421833481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4940d1390 0x7ff480004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 --2- 172.21.15.94:0/2421833481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff480004d50 0x7ff480005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4940ce220 con 0x7ff480004d50 2024-04-26T02:28:23.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff48000b9d0 con 0x7ff4800039c0 2024-04-26T02:28:23.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff48000bbd0 con 0x7ff4940d1390 2024-04-26T02:28:23.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49a393640 1 --2- 172.21.15.94:0/2421833481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff480004d50 0x7ff480005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:23.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff499391640 1 --2- 172.21.15.94:0/2421833481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4940d1390 0x7ff480004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:23.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff499b92640 1 --2- 172.21.15.94:0/2421833481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4800039c0 0x7ff480003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:23.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49a393640 1 -- 172.21.15.94:0/2421833481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4940d1390 msgr2=0x7ff480004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:23.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49a393640 1 --2- 172.21.15.94:0/2421833481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4940d1390 0x7ff480004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49a393640 1 -- 172.21.15.94:0/2421833481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4800039c0 msgr2=0x7ff480003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:23.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49a393640 1 --2- 172.21.15.94:0/2421833481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4800039c0 0x7ff480003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.018+0000 7ff49a393640 1 -- 172.21.15.94:0/2421833481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff480003530 con 0x7ff480004d50 2024-04-26T02:28:23.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.022+0000 7ff49a393640 1 --2- 172.21.15.94:0/2421833481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff480004d50 0x7ff480005160 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7ff490009ae0 tx=0x7ff49001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f41939c19cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:23.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.022+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4900b2070 con 0x7ff480004d50 2024-04-26T02:28:23.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.022+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4900a5680 con 0x7ff480004d50 2024-04-26T02:28:23.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.022+0000 7ff498b90640 1 -- 172.21.15.94:0/2421833481 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4900096b0 con 0x7ff480004d50 2024-04-26T02:28:23.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.022+0000 7ff49bdf6640 1 -- 172.21.15.94:0/2421833481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff480004d50 msgr2=0x7ff480005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:23.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.022+0000 7ff49bdf6640 1 --2- 172.21.15.94:0/2421833481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff480004d50 0x7ff480005160 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7ff490009ae0 tx=0x7ff49001ca90 comp rx=0 tx=0).stop 2024-04-26T02:28:23.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.022+0000 7ff49a393640 1 -- 172.21.15.94:0/2421833481 reap_dead start 2024-04-26T02:28:23.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.022+0000 7ff49bdf6640 1 -- 172.21.15.94:0/2421833481 shutdown_connections 2024-04-26T02:28:23.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.022+0000 7ff49bdf6640 1 -- 172.21.15.94:0/2421833481 >> 172.21.15.94:0/2421833481 conn(0x7ff4940d1d30 msgr2=0x7ff4940d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:23.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.022+0000 7ff49bdf6640 1 -- 172.21.15.94:0/2421833481 shutdown_connections 2024-04-26T02:28:23.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.022+0000 7ff49bdf6640 1 -- 172.21.15.94:0/2421833481 wait complete. 2024-04-26T02:28:23.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff49bdf6640 1 Processor -- start 2024-04-26T02:28:23.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff49bdf6640 1 -- start start 2024-04-26T02:28:23.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff49bdf6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4941495f0 0x7ff49415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff49bdf6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff494149a00 0x7ff49415cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff49bdf6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff494070580 0x7ff494070990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff49bdf6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff49415ece0 con 0x7ff494149a00 2024-04-26T02:28:23.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff49bdf6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff49415ea00 con 0x7ff494070580 2024-04-26T02:28:23.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff49bdf6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff4940ce6b0 con 0x7ff4941495f0 2024-04-26T02:28:23.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff499391640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff494149a00 0x7ff49415cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:23.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff499391640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff494149a00 0x7ff49415cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50230/0 (socket says 172.21.15.94:50230) 2024-04-26T02:28:23.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff499391640 1 -- 172.21.15.94:0/3010912396 learned_addr learned my addr 172.21.15.94:0/3010912396 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:23.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff499b92640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4941495f0 0x7ff49415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:23.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff499391640 1 -- 172.21.15.94:0/3010912396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4941495f0 msgr2=0x7ff49415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:23.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff499391640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4941495f0 0x7ff49415ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff499391640 1 -- 172.21.15.94:0/3010912396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff494070580 msgr2=0x7ff494070990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:23.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff499391640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff494070580 0x7ff494070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff499391640 1 -- 172.21.15.94:0/3010912396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff494070ed0 con 0x7ff494149a00 2024-04-26T02:28:23.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff499b92640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4941495f0 0x7ff49415ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:23.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff499391640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff494149a00 0x7ff49415cf50 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7ff484011600 tx=0x7ff484011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:23.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff48a7fc640 1 -- 172.21.15.94:0/3010912396 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484015870 con 0x7ff494149a00 2024-04-26T02:28:23.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff48a7fc640 1 -- 172.21.15.94:0/3010912396 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff484015a10 con 0x7ff494149a00 2024-04-26T02:28:23.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff49bdf6640 1 -- 172.21.15.94:0/3010912396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49406f0c0 con 0x7ff494149a00 2024-04-26T02:28:23.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff48a7fc640 1 -- 172.21.15.94:0/3010912396 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484015dc0 con 0x7ff494149a00 2024-04-26T02:28:23.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff49bdf6640 1 -- 172.21.15.94:0/3010912396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49406f680 con 0x7ff494149a00 2024-04-26T02:28:23.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.026+0000 7ff49bdf6640 1 -- 172.21.15.94:0/3010912396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49406a730 con 0x7ff494149a00 2024-04-26T02:28:23.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.030+0000 7ff48a7fc640 1 -- 172.21.15.94:0/3010912396 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff48400f490 con 0x7ff494149a00 2024-04-26T02:28:23.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.030+0000 7ff48a7fc640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff47806f400 0x7ff478071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.030+0000 7ff48a7fc640 1 -- 172.21.15.94:0/3010912396 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7ff48408f980 con 0x7ff494149a00 2024-04-26T02:28:23.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.030+0000 7ff499b92640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff47806f400 0x7ff478071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:23.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.030+0000 7ff499b92640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff47806f400 0x7ff478071910 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff48c001dc0 tx=0x7ff48c001300 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:23.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.034+0000 7ff48a7fc640 1 -- 172.21.15.94:0/3010912396 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff484019030 con 0x7ff494149a00 2024-04-26T02:28:23.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.186+0000 7ff49bdf6640 1 -- 172.21.15.94:0/3010912396 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff49406cd50 con 0x7ff47806f400 2024-04-26T02:28:23.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.626+0000 7ff48a7fc640 1 -- 172.21.15.94:0/3010912396 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13672 (secure 0 0 0) 0x7ff49406cd50 con 0x7ff47806f400 2024-04-26T02:28:23.633 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:23.634 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-26T02:28:23.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.630+0000 7ff49bdf6640 1 -- 172.21.15.94:0/3010912396 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff47806f400 msgr2=0x7ff478071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:23.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.630+0000 7ff49bdf6640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff47806f400 0x7ff478071910 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff48c001dc0 tx=0x7ff48c001300 comp rx=0 tx=0).stop 2024-04-26T02:28:23.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.630+0000 7ff49bdf6640 1 -- 172.21.15.94:0/3010912396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff494149a00 msgr2=0x7ff49415cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:23.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.630+0000 7ff49bdf6640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff494149a00 0x7ff49415cf50 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7ff484011600 tx=0x7ff484011ac0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.630+0000 7ff49bdf6640 1 -- 172.21.15.94:0/3010912396 shutdown_connections 2024-04-26T02:28:23.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.630+0000 7ff49bdf6640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff494070580 0x7ff494070990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.630+0000 7ff49bdf6640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff494149a00 0x7ff49415cf50 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.630+0000 7ff49bdf6640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ff47806f400 0x7ff478071910 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.630+0000 7ff49bdf6640 1 --2- 172.21.15.94:0/3010912396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4941495f0 0x7ff49415ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.630+0000 7ff49bdf6640 1 -- 172.21.15.94:0/3010912396 >> 172.21.15.94:0/3010912396 conn(0x7ff4940d1d30 msgr2=0x7ff494073010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:23.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.630+0000 7ff49bdf6640 1 -- 172.21.15.94:0/3010912396 shutdown_connections 2024-04-26T02:28:23.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.630+0000 7ff49bdf6640 1 -- 172.21.15.94:0/3010912396 wait complete. 2024-04-26T02:28:23.665 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":17360,"stamp":"2024-04-26T02:28:21.223319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116,"ondisk_log_size":116,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":59922576,"kb_used_data":47391888,"kb_used_omap":3838,"kb_used_meta":12526785,"kb_avail":695052144,"statfs":{"total":773094113280,"available":711733395456,"internally_reserved":0,"allocated":48529293312,"data_stored":48515348890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3930321,"internal_metadata":12827428655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,6,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26633280872,"num_objects":-15220,"num_object_clones":0,"num_object_copies":-30440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15220,"num_whiteouts":0,"num_read":-1861898,"num_read_kb":-130434489,"num_write":-7559234,"num_write_kb":-203981983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-411,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.004114"},"pg_stats":[{"pgid":"1.0","version":"40'116","reported_seq":487,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T02:25:06.032055+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-26T02:25:06.032055+0000","last_peered":"2024-04-26T02:25:06.032055+0000","last_clean":"2024-04-26T02:25:06.032055+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-26T02:25:06.032055+0000","last_undegraded":"2024-04-26T02:25:06.032055+0000","last_fullsized":"2024-04-26T02:25:06.032055+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116,"ondisk_log_size":116,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834581783,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7934292,"kb_used_data":6303948,"kb_used_omap":249,"kb_used_meta":1630086,"kb_avail":86437548,"statfs":{"total":96636764160,"available":88512049152,"internally_reserved":0,"allocated":6455242752,"data_stored":6453277722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":254984,"internal_metadata":1669209080},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834581781,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7333340,"kb_used_data":5618132,"kb_used_omap":137,"kb_used_meta":1715062,"kb_avail":87038500,"statfs":{"total":96636764160,"available":89127424000,"internally_reserved":0,"allocated":5752967168,"data_stored":5751371468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":140568,"internal_metadata":1756224232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834581782,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8782168,"kb_used_data":7656656,"kb_used_omap":829,"kb_used_meta":1124674,"kb_avail":85589672,"statfs":{"total":96636764160,"available":87643824128,"internally_reserved":0,"allocated":7840415744,"data_stored":7838134788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":849456,"internal_metadata":1151666640},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834581782,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6675772,"kb_used_data":5295924,"kb_used_omap":820,"kb_used_meta":1379019,"kb_avail":87696068,"statfs":{"total":96636764160,"available":89800773632,"internally_reserved":0,"allocated":5423026176,"data_stored":5421589664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":840213,"internal_metadata":1412115947},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834581764,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8256040,"kb_used_data":6717344,"kb_used_omap":308,"kb_used_meta":1538379,"kb_avail":86115800,"statfs":{"total":96636764160,"available":88182579200,"internally_reserved":0,"allocated":6878560256,"data_stored":6876569414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":315597,"internal_metadata":1575300915},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834581769,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7903396,"kb_used_data":6172700,"kb_used_omap":628,"kb_used_meta":1730059,"kb_avail":86468444,"statfs":{"total":96636764160,"available":88543686656,"internally_reserved":0,"allocated":6320844800,"data_stored":6319113557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":643298,"internal_metadata":1771581214},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834581772,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6570084,"kb_used_data":4878556,"kb_used_omap":239,"kb_used_meta":1691280,"kb_avail":87801756,"statfs":{"total":96636764160,"available":89908998144,"internally_reserved":0,"allocated":4995641344,"data_stored":4994128296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":244932,"internal_metadata":1731871548},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834581773,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6467484,"kb_used_data":4748628,"kb_used_omap":626,"kb_used_meta":1718221,"kb_avail":87904356,"statfs":{"total":96636764160,"available":90014060544,"internally_reserved":0,"allocated":4862595072,"data_stored":4861163981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":641273,"internal_metadata":1759459079},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T02:28:23.666 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T02:28:23.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e85f18640 1 Processor -- start 2024-04-26T02:28:23.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e85f18640 1 -- start start 2024-04-26T02:28:23.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e85f18640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8e8006ce90 0x7f8e800d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e85f18640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8e8006d2a0 0x7f8e800d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e85f18640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e8006d850 con 0x7f8e801615a0 2024-04-26T02:28:23.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e85f18640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f8e8006db30 con 0x7f8e8006d2a0 2024-04-26T02:28:23.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e85f18640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8e8006de10 con 0x7f8e8006ce90 2024-04-26T02:28:23.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e77fff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8e8006d2a0 0x7f8e800d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:23.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e7ffff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f8e801615a0 0x7f8e801619b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56084/0 (socket says 172.21.15.94:56084) 2024-04-26T02:28:23.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e7ffff640 1 -- 172.21.15.94:0/1461585260 learned_addr learned my addr 172.21.15.94:0/1461585260 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:23.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e77fff640 1 -- 172.21.15.94:0/1461585260 >> v2:172.21.15.138:3300/0 conn(0x7f8e8006ce90 msgr2=0x7f8e800d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:23.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e7f7fe640 1 --2- 172.21.15.94:0/1461585260 >> v2:172.21.15.138:3300/0 conn(0x7f8e8006ce90 0x7f8e800d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:23.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e77fff640 1 --2- 172.21.15.94:0/1461585260 >> v2:172.21.15.138:3300/0 conn(0x7f8e8006ce90 0x7f8e800d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e7effd640 1 -- 172.21.15.94:0/1461585260 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3492865267 0 0) 0x7f8e8006d850 con 0x7f8e801615a0 2024-04-26T02:28:23.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e77fff640 1 -- 172.21.15.94:0/1461585260 >> v1:172.21.15.94:6789/0 conn(0x7f8e801615a0 legacy=0x7f8e801619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:23.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e77fff640 1 -- 172.21.15.94:0/1461585260 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8006e0f0 con 0x7f8e8006d2a0 2024-04-26T02:28:23.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.806+0000 7f8e7f7fe640 1 --2- 172.21.15.94:0/1461585260 >> v2:172.21.15.138:3300/0 conn(0x7f8e8006ce90 0x7f8e800d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:23.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.810+0000 7f8e77fff640 1 --2- 172.21.15.94:0/1461585260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8006d2a0 0x7f8e800d1880 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f8e800730c0 tx=0x7f8e6801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f627220d2dc08f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:23.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.810+0000 7f8e7effd640 1 -- 172.21.15.94:0/1461585260 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e680b0070 con 0x7f8e8006d2a0 2024-04-26T02:28:23.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.810+0000 7f8e7effd640 1 -- 172.21.15.94:0/1461585260 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e680ab600 con 0x7f8e8006d2a0 2024-04-26T02:28:23.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.810+0000 7f8e7effd640 1 -- 172.21.15.94:0/1461585260 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e680ab9d0 con 0x7f8e8006d2a0 2024-04-26T02:28:23.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.810+0000 7f8e85f18640 1 -- 172.21.15.94:0/1461585260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8006d2a0 msgr2=0x7f8e800d1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:23.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.810+0000 7f8e85f18640 1 --2- 172.21.15.94:0/1461585260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8006d2a0 0x7f8e800d1880 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f8e800730c0 tx=0x7f8e6801c3f0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.810+0000 7f8e85f18640 1 -- 172.21.15.94:0/1461585260 shutdown_connections 2024-04-26T02:28:23.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.810+0000 7f8e85f18640 1 --2- 172.21.15.94:0/1461585260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8006d2a0 0x7f8e800d1880 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.810+0000 7f8e85f18640 1 --2- 172.21.15.94:0/1461585260 >> v2:172.21.15.138:3300/0 conn(0x7f8e8006ce90 0x7f8e800d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.810+0000 7f8e85f18640 1 -- 172.21.15.94:0/1461585260 >> 172.21.15.94:0/1461585260 conn(0x7f8e80071150 msgr2=0x7f8e80071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:23.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.810+0000 7f8e85f18640 1 -- 172.21.15.94:0/1461585260 shutdown_connections 2024-04-26T02:28:23.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.810+0000 7f8e85f18640 1 -- 172.21.15.94:0/1461585260 wait complete. 2024-04-26T02:28:23.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e85f18640 1 Processor -- start 2024-04-26T02:28:23.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e85f18640 1 -- start start 2024-04-26T02:28:23.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e85f18640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8006ce90 0x7f8e8013e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e85f18640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8006d2a0 0x7f8e80157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e85f18640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e801615a0 0x7f8e8015c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e85f18640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80147520 con 0x7f8e8006d2a0 2024-04-26T02:28:23.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e85f18640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e800d0040 con 0x7f8e8006ce90 2024-04-26T02:28:23.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e85f18640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8e801620c0 con 0x7f8e801615a0 2024-04-26T02:28:23.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e77fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8006d2a0 0x7f8e80157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:23.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e77fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8006d2a0 0x7f8e80157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50240/0 (socket says 172.21.15.94:50240) 2024-04-26T02:28:23.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e77fff640 1 -- 172.21.15.94:0/1544764230 learned_addr learned my addr 172.21.15.94:0/1544764230 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:23.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e77fff640 1 -- 172.21.15.94:0/1544764230 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e801615a0 msgr2=0x7f8e8015c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:23.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e77fff640 1 --2- 172.21.15.94:0/1544764230 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e801615a0 0x7f8e8015c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e7f7fe640 1 --2- 172.21.15.94:0/1544764230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8006ce90 0x7f8e8013e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:23.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e77fff640 1 -- 172.21.15.94:0/1544764230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8006ce90 msgr2=0x7f8e8013e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:23.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e77fff640 1 --2- 172.21.15.94:0/1544764230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8006ce90 0x7f8e8013e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e77fff640 1 -- 172.21.15.94:0/1544764230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e80157f90 con 0x7f8e8006d2a0 2024-04-26T02:28:23.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e77fff640 1 --2- 172.21.15.94:0/1544764230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8006d2a0 0x7f8e80157a50 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f8e680ae040 tx=0x7f8e68019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:23.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e7cff9640 1 -- 172.21.15.94:0/1544764230 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e680b0040 con 0x7f8e8006d2a0 2024-04-26T02:28:23.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e7cff9640 1 -- 172.21.15.94:0/1544764230 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e680b9070 con 0x7f8e8006d2a0 2024-04-26T02:28:23.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e85f18640 1 -- 172.21.15.94:0/1544764230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e80160070 con 0x7f8e8006d2a0 2024-04-26T02:28:23.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e7cff9640 1 -- 172.21.15.94:0/1544764230 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e680b4830 con 0x7f8e8006d2a0 2024-04-26T02:28:23.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e85f18640 1 -- 172.21.15.94:0/1544764230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e80160630 con 0x7f8e8006d2a0 2024-04-26T02:28:23.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.814+0000 7f8e85f18640 1 -- 172.21.15.94:0/1544764230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e8006a730 con 0x7f8e8006d2a0 2024-04-26T02:28:23.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.818+0000 7f8e7cff9640 1 -- 172.21.15.94:0/1544764230 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8e680ab4c0 con 0x7f8e8006d2a0 2024-04-26T02:28:23.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.818+0000 7f8e7cff9640 1 --2- 172.21.15.94:0/1544764230 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8e6006f430 0x7f8e60071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:23.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.818+0000 7f8e7cff9640 1 -- 172.21.15.94:0/1544764230 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f8e68128f30 con 0x7f8e8006d2a0 2024-04-26T02:28:23.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.818+0000 7f8e7f7fe640 1 --2- 172.21.15.94:0/1544764230 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8e6006f430 0x7f8e60071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:23.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.818+0000 7f8e7f7fe640 1 --2- 172.21.15.94:0/1544764230 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8e6006f430 0x7f8e60071940 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8e70006fd0 tx=0x7f8e70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:23.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.822+0000 7f8e7cff9640 1 -- 172.21.15.94:0/1544764230 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8e68016070 con 0x7f8e8006d2a0 2024-04-26T02:28:23.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.974+0000 7f8e85f18640 1 -- 172.21.15.94:0/1544764230 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8e800d0c10 con 0x7f8e6006f430 2024-04-26T02:28:23.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.974+0000 7f8e7cff9640 1 -- 172.21.15.94:0/1544764230 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13705 (secure 0 0 0) 0x7f8e800d0c10 con 0x7f8e6006f430 2024-04-26T02:28:23.980 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:23.980 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-26T02:28:23.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.978+0000 7f8e85f18640 1 -- 172.21.15.94:0/1544764230 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8e6006f430 msgr2=0x7f8e60071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:23.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.978+0000 7f8e85f18640 1 --2- 172.21.15.94:0/1544764230 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8e6006f430 0x7f8e60071940 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8e70006fd0 tx=0x7f8e70008040 comp rx=0 tx=0).stop 2024-04-26T02:28:23.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.978+0000 7f8e85f18640 1 -- 172.21.15.94:0/1544764230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8006d2a0 msgr2=0x7f8e80157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:23.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.978+0000 7f8e85f18640 1 --2- 172.21.15.94:0/1544764230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8006d2a0 0x7f8e80157a50 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f8e680ae040 tx=0x7f8e68019190 comp rx=0 tx=0).stop 2024-04-26T02:28:23.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.978+0000 7f8e85f18640 1 -- 172.21.15.94:0/1544764230 shutdown_connections 2024-04-26T02:28:23.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.978+0000 7f8e85f18640 1 --2- 172.21.15.94:0/1544764230 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8e6006f430 0x7f8e60071940 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.978+0000 7f8e85f18640 1 --2- 172.21.15.94:0/1544764230 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e801615a0 0x7f8e8015c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.978+0000 7f8e85f18640 1 --2- 172.21.15.94:0/1544764230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8006d2a0 0x7f8e80157a50 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.978+0000 7f8e85f18640 1 --2- 172.21.15.94:0/1544764230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8006ce90 0x7f8e8013e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.978+0000 7f8e85f18640 1 -- 172.21.15.94:0/1544764230 >> 172.21.15.94:0/1544764230 conn(0x7f8e80071150 msgr2=0x7f8e800cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.978+0000 7f8e85f18640 1 -- 172.21.15.94:0/1544764230 shutdown_connections 2024-04-26T02:28:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:23.978+0000 7f8e85f18640 1 -- 172.21.15.94:0/1544764230 wait complete. 2024-04-26T02:28:24.001 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":17361,"stamp":"2024-04-26T02:28:23.634578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116,"ondisk_log_size":116,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":19446584,"kb_used_data":6741860,"kb_used_omap":3913,"kb_used_meta":12581366,"kb_avail":735528136,"statfs":{"total":773094113280,"available":753180811264,"internally_reserved":0,"allocated":6903664640,"data_stored":6899424091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4007562,"internal_metadata":12883319158},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":646,"apply_latency_ms":646,"commit_latency_ns":646000000,"apply_latency_ns":646000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26595903353,"num_objects":-15193,"num_object_clones":0,"num_object_copies":-30386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15193,"num_whiteouts":0,"num_read":-1861898,"num_read_kb":-130434489,"num_write":-7559261,"num_write_kb":-203981983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-408,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.266404"},"pg_stats":[{"pgid":"1.0","version":"40'116","reported_seq":493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T02:28:19.453534+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-26T02:28:19.453534+0000","last_peered":"2024-04-26T02:28:19.453534+0000","last_clean":"2024-04-26T02:28:19.453534+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-26T02:28:19.453534+0000","last_undegraded":"2024-04-26T02:28:19.453534+0000","last_fullsized":"2024-04-26T02:28:19.453534+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116,"ondisk_log_size":116,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834581784,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2486180,"kb_used_data":855832,"kb_used_omap":249,"kb_used_meta":1630086,"kb_avail":91885660,"statfs":{"total":96636764160,"available":94090915840,"internally_reserved":0,"allocated":876371968,"data_stored":875818111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":254984,"internal_metadata":1669209080},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":82,"apply_latency_ms":82,"commit_latency_ns":82000000,"apply_latency_ns":82000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834581782,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1715704,"kb_used_data":496,"kb_used_omap":137,"kb_used_meta":1715062,"kb_avail":92656136,"statfs":{"total":96636764160,"available":94879883264,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":140568,"internal_metadata":1756224232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":87,"apply_latency_ms":87,"commit_latency_ns":87000000,"apply_latency_ns":87000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834581783,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3119988,"kb_used_data":1994476,"kb_used_omap":829,"kb_used_meta":1124674,"kb_avail":91251852,"statfs":{"total":96636764160,"available":93441896448,"internally_reserved":0,"allocated":2042343424,"data_stored":2041484860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":849456,"internal_metadata":1151666640},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":97,"apply_latency_ms":97,"commit_latency_ns":97000000,"apply_latency_ns":97000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834581783,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1398396,"kb_used_data":496,"kb_used_omap":820,"kb_used_meta":1397067,"kb_avail":92973444,"statfs":{"total":96636764160,"available":95204806656,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":840213,"internal_metadata":1430597099},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834581765,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1642140,"kb_used_data":496,"kb_used_omap":308,"kb_used_meta":1538379,"kb_avail":92729700,"statfs":{"total":96636764160,"available":94955212800,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":315597,"internal_metadata":1575300915},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":93,"apply_latency_ms":93,"commit_latency_ns":93000000,"apply_latency_ns":93000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834581770,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1749752,"kb_used_data":496,"kb_used_omap":703,"kb_used_meta":1748544,"kb_avail":92622088,"statfs":{"total":96636764160,"available":94845018112,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":720539,"internal_metadata":1790509413},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":81,"apply_latency_ms":81,"commit_latency_ns":81000000,"apply_latency_ns":81000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834581773,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4614132,"kb_used_data":2904552,"kb_used_omap":239,"kb_used_meta":1709328,"kb_avail":89757708,"statfs":{"total":96636764160,"available":91911892992,"internally_reserved":0,"allocated":2974261248,"data_stored":2973180976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":244932,"internal_metadata":1750352700},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":71,"apply_latency_ms":71,"commit_latency_ns":71000000,"apply_latency_ns":71000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834581774,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2720292,"kb_used_data":985016,"kb_used_omap":626,"kb_used_meta":1718221,"kb_avail":91651548,"statfs":{"total":96636764160,"available":93851185152,"internally_reserved":0,"allocated":1008656384,"data_stored":1008053644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":641273,"internal_metadata":1759459079},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":83,"apply_latency_ms":83,"commit_latency_ns":83000000,"apply_latency_ns":83000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T02:28:24.002 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T02:28:24.002 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T02:28:24.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb7af9640 1 Processor -- start 2024-04-26T02:28:24.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb7af9640 1 -- start start 2024-04-26T02:28:24.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb7af9640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcbb00ce480 0x7fcbb00ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:24.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb7af9640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcbb00b7990 0x7fcbb0169750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:24.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb7af9640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbb00ccf30 con 0x7fcbb00d2030 2024-04-26T02:28:24.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb7af9640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fcbb00ca590 con 0x7fcbb00ce480 2024-04-26T02:28:24.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb7af9640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcbb00cd210 con 0x7fcbb00b7990 2024-04-26T02:28:24.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb5895640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcbb00ce480 0x7fcbb00ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:24.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb5895640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcbb00ce480 0x7fcbb00ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50248/0 (socket says 172.21.15.94:50248) 2024-04-26T02:28:24.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb5895640 1 -- 172.21.15.94:0/407200252 learned_addr learned my addr 172.21.15.94:0/407200252 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:24.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb5895640 1 -- 172.21.15.94:0/407200252 >> v2:172.21.15.138:3300/0 conn(0x7fcbb00b7990 msgr2=0x7fcbb0169750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:24.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb5895640 1 --2- 172.21.15.94:0/407200252 >> v2:172.21.15.138:3300/0 conn(0x7fcbb00b7990 0x7fcbb0169750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb5895640 1 -- 172.21.15.94:0/407200252 >> v1:172.21.15.138:6789/0 conn(0x7fcbb00d2030 legacy=0x7fcbb00d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:24.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.146+0000 7fcbb5895640 1 -- 172.21.15.94:0/407200252 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb00cd4f0 con 0x7fcbb00ce480 2024-04-26T02:28:24.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb5895640 1 --2- 172.21.15.94:0/407200252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbb00ce480 0x7fcbb00ce890 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00cf390 tx=0x7fcba001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f4106204f2a471 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:24.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb4893640 1 -- 172.21.15.94:0/407200252 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba00b0070 con 0x7fcbb00ce480 2024-04-26T02:28:24.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb4893640 1 -- 172.21.15.94:0/407200252 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba00ab600 con 0x7fcbb00ce480 2024-04-26T02:28:24.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb4893640 1 -- 172.21.15.94:0/407200252 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba00ab9d0 con 0x7fcbb00ce480 2024-04-26T02:28:24.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb7af9640 1 -- 172.21.15.94:0/407200252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbb00ce480 msgr2=0x7fcbb00ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:24.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb7af9640 1 --2- 172.21.15.94:0/407200252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbb00ce480 0x7fcbb00ce890 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00cf390 tx=0x7fcba001c4e0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb7af9640 1 -- 172.21.15.94:0/407200252 shutdown_connections 2024-04-26T02:28:24.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb7af9640 1 --2- 172.21.15.94:0/407200252 >> v2:172.21.15.138:3300/0 conn(0x7fcbb00b7990 0x7fcbb0169750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb7af9640 1 --2- 172.21.15.94:0/407200252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbb00ce480 0x7fcbb00ce890 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb7af9640 1 -- 172.21.15.94:0/407200252 >> 172.21.15.94:0/407200252 conn(0x7fcbb00cfed0 msgr2=0x7fcbb00d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:24.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb7af9640 1 -- 172.21.15.94:0/407200252 shutdown_connections 2024-04-26T02:28:24.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb7af9640 1 -- 172.21.15.94:0/407200252 wait complete. 2024-04-26T02:28:24.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb7af9640 1 Processor -- start 2024-04-26T02:28:24.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.150+0000 7fcbb7af9640 1 -- start start 2024-04-26T02:28:24.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb7af9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbb00b7990 0x7fcbb00c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:24.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb7af9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbb00ce480 0x7fcbb00ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:24.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb7af9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbb00d2030 0x7fcbb00c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:24.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb7af9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb016c840 con 0x7fcbb00b7990 2024-04-26T02:28:24.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb7af9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb016c6c0 con 0x7fcbb00ce480 2024-04-26T02:28:24.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb7af9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb016c9c0 con 0x7fcbb00d2030 2024-04-26T02:28:24.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb5895640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbb00b7990 0x7fcbb00c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:24.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb5895640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbb00b7990 0x7fcbb00c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50252/0 (socket says 172.21.15.94:50252) 2024-04-26T02:28:24.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb5895640 1 -- 172.21.15.94:0/212918887 learned_addr learned my addr 172.21.15.94:0/212918887 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:24.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb5895640 1 -- 172.21.15.94:0/212918887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbb00d2030 msgr2=0x7fcbb00c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:24.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb5895640 1 --2- 172.21.15.94:0/212918887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbb00d2030 0x7fcbb00c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb5895640 1 -- 172.21.15.94:0/212918887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbb00ce480 msgr2=0x7fcbb00ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:24.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb5895640 1 --2- 172.21.15.94:0/212918887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbb00ce480 0x7fcbb00ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb5895640 1 -- 172.21.15.94:0/212918887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb00aee30 con 0x7fcbb00b7990 2024-04-26T02:28:24.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb5895640 1 --2- 172.21.15.94:0/212918887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbb00b7990 0x7fcbb00c8040 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fcba001ab90 tx=0x7fcba001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:24.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcba67fc640 1 -- 172.21.15.94:0/212918887 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba00b0040 con 0x7fcbb00b7990 2024-04-26T02:28:24.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcba67fc640 1 -- 172.21.15.94:0/212918887 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba00bc070 con 0x7fcbb00b7990 2024-04-26T02:28:24.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb7af9640 1 -- 172.21.15.94:0/212918887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb00c8940 con 0x7fcbb00b7990 2024-04-26T02:28:24.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcba67fc640 1 -- 172.21.15.94:0/212918887 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba00b6880 con 0x7fcbb00b7990 2024-04-26T02:28:24.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb7af9640 1 -- 172.21.15.94:0/212918887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb00c8f00 con 0x7fcbb00b7990 2024-04-26T02:28:24.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb7af9640 1 -- 172.21.15.94:0/212918887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb0076830 con 0x7fcbb00b7990 2024-04-26T02:28:24.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcba67fc640 1 -- 172.21.15.94:0/212918887 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcba00b6a20 con 0x7fcbb00b7990 2024-04-26T02:28:24.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcba67fc640 1 --2- 172.21.15.94:0/212918887 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcb9c06f430 0x7fcb9c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:24.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcba67fc640 1 -- 172.21.15.94:0/212918887 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fcba0129540 con 0x7fcbb00b7990 2024-04-26T02:28:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.154+0000 7fcbb5094640 1 --2- 172.21.15.94:0/212918887 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcb9c06f430 0x7fcb9c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.158+0000 7fcbb5094640 1 --2- 172.21.15.94:0/212918887 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcb9c06f430 0x7fcb9c071940 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0157eb0 tx=0x7fcbac009250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.162+0000 7fcba67fc640 1 -- 172.21.15.94:0/212918887 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcba00160d0 con 0x7fcbb00b7990 2024-04-26T02:28:24.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.318+0000 7fcbb7af9640 1 -- 172.21.15.94:0/212918887 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcbb00cc9a0 con 0x7fcb9c06f430 2024-04-26T02:28:24.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.318+0000 7fcba67fc640 1 -- 172.21.15.94:0/212918887 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13705 (secure 0 0 0) 0x7fcbb00cc9a0 con 0x7fcb9c06f430 2024-04-26T02:28:24.324 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:24.324 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-26T02:28:24.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.322+0000 7fcbb7af9640 1 -- 172.21.15.94:0/212918887 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcb9c06f430 msgr2=0x7fcb9c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:24.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.322+0000 7fcbb7af9640 1 --2- 172.21.15.94:0/212918887 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcb9c06f430 0x7fcb9c071940 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0157eb0 tx=0x7fcbac009250 comp rx=0 tx=0).stop 2024-04-26T02:28:24.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.322+0000 7fcbb7af9640 1 -- 172.21.15.94:0/212918887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbb00b7990 msgr2=0x7fcbb00c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:24.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.322+0000 7fcbb7af9640 1 --2- 172.21.15.94:0/212918887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbb00b7990 0x7fcbb00c8040 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fcba001ab90 tx=0x7fcba001ad20 comp rx=0 tx=0).stop 2024-04-26T02:28:24.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.322+0000 7fcbb7af9640 1 -- 172.21.15.94:0/212918887 shutdown_connections 2024-04-26T02:28:24.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.322+0000 7fcbb7af9640 1 --2- 172.21.15.94:0/212918887 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fcb9c06f430 0x7fcb9c071940 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.322+0000 7fcbb7af9640 1 --2- 172.21.15.94:0/212918887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcbb00d2030 0x7fcbb00c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.322+0000 7fcbb7af9640 1 --2- 172.21.15.94:0/212918887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbb00ce480 0x7fcbb00ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.322+0000 7fcbb7af9640 1 --2- 172.21.15.94:0/212918887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcbb00b7990 0x7fcbb00c8040 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.322+0000 7fcbb7af9640 1 -- 172.21.15.94:0/212918887 >> 172.21.15.94:0/212918887 conn(0x7fcbb00cfed0 msgr2=0x7fcbb00d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:24.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.322+0000 7fcbb7af9640 1 -- 172.21.15.94:0/212918887 shutdown_connections 2024-04-26T02:28:24.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.322+0000 7fcbb7af9640 1 -- 172.21.15.94:0/212918887 wait complete. 2024-04-26T02:28:24.347 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":17361,"stamp":"2024-04-26T02:28:23.634578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116,"ondisk_log_size":116,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":19446584,"kb_used_data":6741860,"kb_used_omap":3913,"kb_used_meta":12581366,"kb_avail":735528136,"statfs":{"total":773094113280,"available":753180811264,"internally_reserved":0,"allocated":6903664640,"data_stored":6899424091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4007562,"internal_metadata":12883319158},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":646,"apply_latency_ms":646,"commit_latency_ns":646000000,"apply_latency_ns":646000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26595903353,"num_objects":-15193,"num_object_clones":0,"num_object_copies":-30386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15193,"num_whiteouts":0,"num_read":-1861898,"num_read_kb":-130434489,"num_write":-7559261,"num_write_kb":-203981983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-408,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.266404"},"pg_stats":[{"pgid":"1.0","version":"40'116","reported_seq":493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T02:28:19.453534+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-26T02:28:19.453534+0000","last_peered":"2024-04-26T02:28:19.453534+0000","last_clean":"2024-04-26T02:28:19.453534+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-26T02:28:19.453534+0000","last_undegraded":"2024-04-26T02:28:19.453534+0000","last_fullsized":"2024-04-26T02:28:19.453534+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116,"ondisk_log_size":116,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834581784,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2486180,"kb_used_data":855832,"kb_used_omap":249,"kb_used_meta":1630086,"kb_avail":91885660,"statfs":{"total":96636764160,"available":94090915840,"internally_reserved":0,"allocated":876371968,"data_stored":875818111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":254984,"internal_metadata":1669209080},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":82,"apply_latency_ms":82,"commit_latency_ns":82000000,"apply_latency_ns":82000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834581782,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1715704,"kb_used_data":496,"kb_used_omap":137,"kb_used_meta":1715062,"kb_avail":92656136,"statfs":{"total":96636764160,"available":94879883264,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":140568,"internal_metadata":1756224232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":87,"apply_latency_ms":87,"commit_latency_ns":87000000,"apply_latency_ns":87000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834581783,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3119988,"kb_used_data":1994476,"kb_used_omap":829,"kb_used_meta":1124674,"kb_avail":91251852,"statfs":{"total":96636764160,"available":93441896448,"internally_reserved":0,"allocated":2042343424,"data_stored":2041484860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":849456,"internal_metadata":1151666640},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":97,"apply_latency_ms":97,"commit_latency_ns":97000000,"apply_latency_ns":97000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834581783,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1398396,"kb_used_data":496,"kb_used_omap":820,"kb_used_meta":1397067,"kb_avail":92973444,"statfs":{"total":96636764160,"available":95204806656,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":840213,"internal_metadata":1430597099},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834581765,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1642140,"kb_used_data":496,"kb_used_omap":308,"kb_used_meta":1538379,"kb_avail":92729700,"statfs":{"total":96636764160,"available":94955212800,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":315597,"internal_metadata":1575300915},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":93,"apply_latency_ms":93,"commit_latency_ns":93000000,"apply_latency_ns":93000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834581770,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1749752,"kb_used_data":496,"kb_used_omap":703,"kb_used_meta":1748544,"kb_avail":92622088,"statfs":{"total":96636764160,"available":94845018112,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":720539,"internal_metadata":1790509413},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":81,"apply_latency_ms":81,"commit_latency_ns":81000000,"apply_latency_ns":81000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834581773,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4614132,"kb_used_data":2904552,"kb_used_omap":239,"kb_used_meta":1709328,"kb_avail":89757708,"statfs":{"total":96636764160,"available":91911892992,"internally_reserved":0,"allocated":2974261248,"data_stored":2973180976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":244932,"internal_metadata":1750352700},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":71,"apply_latency_ms":71,"commit_latency_ns":71000000,"apply_latency_ns":71000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834581774,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":2720292,"kb_used_data":985016,"kb_used_omap":626,"kb_used_meta":1718221,"kb_avail":91651548,"statfs":{"total":96636764160,"available":93851185152,"internally_reserved":0,"allocated":1008656384,"data_stored":1008053644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":641273,"internal_metadata":1759459079},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":83,"apply_latency_ms":83,"commit_latency_ns":83000000,"apply_latency_ns":83000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T02:28:24.348 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T02:28:24.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f4355c72640 1 Processor -- start 2024-04-26T02:28:24.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f4355c72640 1 -- start start 2024-04-26T02:28:24.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f4355c72640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f435006e770 0x7f43500b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:24.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f4355c72640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4350161530 0x7f43500d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:24.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f4355c72640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f43500d11c0 0x7f435006d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:24.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f4355c72640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f43500b1230 con 0x7f43500d11c0 2024-04-26T02:28:24.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f4355c72640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f43500ce220 con 0x7f435006e770 2024-04-26T02:28:24.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f4355c72640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f43500b4b50 con 0x7f4350161530 2024-04-26T02:28:24.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f434ffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f43500d11c0 0x7f435006d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:24.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f434ffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f43500d11c0 0x7f435006d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50256/0 (socket says 172.21.15.94:50256) 2024-04-26T02:28:24.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f434ffff640 1 -- 172.21.15.94:0/1505127701 learned_addr learned my addr 172.21.15.94:0/1505127701 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:24.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f434f7fe640 1 --2- 172.21.15.94:0/1505127701 >> v2:172.21.15.138:3300/0 conn(0x7f435006e770 0x7f43500b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:24.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f434effd640 1 --2- 172.21.15.94:0/1505127701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4350161530 0x7f43500d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:24.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f434ffff640 1 -- 172.21.15.94:0/1505127701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4350161530 msgr2=0x7f43500d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:24.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f434ffff640 1 --2- 172.21.15.94:0/1505127701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4350161530 0x7f43500d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f434ffff640 1 -- 172.21.15.94:0/1505127701 >> v2:172.21.15.138:3300/0 conn(0x7f435006e770 msgr2=0x7f43500b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:24.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f434ffff640 1 --2- 172.21.15.94:0/1505127701 >> v2:172.21.15.138:3300/0 conn(0x7f435006e770 0x7f43500b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.498+0000 7f434ffff640 1 -- 172.21.15.94:0/1505127701 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435015e440 con 0x7f43500d11c0 2024-04-26T02:28:24.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.502+0000 7f434ffff640 1 --2- 172.21.15.94:0/1505127701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43500d11c0 0x7f435006d4c0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f43500727e0 tx=0x7f434801c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37de0e7ec7cd26e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:24.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.502+0000 7f434e7fc640 1 -- 172.21.15.94:0/1505127701 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43480a5510 con 0x7f43500d11c0 2024-04-26T02:28:24.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.502+0000 7f434e7fc640 1 -- 172.21.15.94:0/1505127701 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43480a56b0 con 0x7f43500d11c0 2024-04-26T02:28:24.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.502+0000 7f434e7fc640 1 -- 172.21.15.94:0/1505127701 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43480a5a80 con 0x7f43500d11c0 2024-04-26T02:28:24.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.502+0000 7f4355c72640 1 -- 172.21.15.94:0/1505127701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43500d11c0 msgr2=0x7f435006d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:24.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.502+0000 7f4355c72640 1 --2- 172.21.15.94:0/1505127701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43500d11c0 0x7f435006d4c0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f43500727e0 tx=0x7f434801c670 comp rx=0 tx=0).stop 2024-04-26T02:28:24.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.502+0000 7f4355c72640 1 -- 172.21.15.94:0/1505127701 shutdown_connections 2024-04-26T02:28:24.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.502+0000 7f4355c72640 1 --2- 172.21.15.94:0/1505127701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43500d11c0 0x7f435006d4c0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.502+0000 7f4355c72640 1 --2- 172.21.15.94:0/1505127701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4350161530 0x7f43500d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.502+0000 7f4355c72640 1 --2- 172.21.15.94:0/1505127701 >> v2:172.21.15.138:3300/0 conn(0x7f435006e770 0x7f43500b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.502+0000 7f4355c72640 1 -- 172.21.15.94:0/1505127701 >> 172.21.15.94:0/1505127701 conn(0x7f43500d1d30 msgr2=0x7f43500d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:24.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.502+0000 7f4355c72640 1 -- 172.21.15.94:0/1505127701 shutdown_connections 2024-04-26T02:28:24.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.502+0000 7f4355c72640 1 -- 172.21.15.94:0/1505127701 wait complete. 2024-04-26T02:28:24.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f4355c72640 1 Processor -- start 2024-04-26T02:28:24.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f4355c72640 1 -- start start 2024-04-26T02:28:24.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f4355c72640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f435006e770 0x7f43501564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:24.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f4355c72640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43500d11c0 0x7f43500b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:24.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f4355c72640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4350161530 0x7f43500b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:24.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f4355c72640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43500ce6b0 con 0x7f4350161530 2024-04-26T02:28:24.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f4355c72640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43500d15d0 con 0x7f43500d11c0 2024-04-26T02:28:24.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f4355c72640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f43500ce830 con 0x7f435006e770 2024-04-26T02:28:24.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f434ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4350161530 0x7f43500b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:24.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f434ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4350161530 0x7f43500b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50266/0 (socket says 172.21.15.94:50266) 2024-04-26T02:28:24.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f434ffff640 1 -- 172.21.15.94:0/4086063123 learned_addr learned my addr 172.21.15.94:0/4086063123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:24.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f434f7fe640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f435006e770 0x7f43501564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:24.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f434effd640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43500d11c0 0x7f43500b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:24.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f434ffff640 1 -- 172.21.15.94:0/4086063123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f435006e770 msgr2=0x7f43501564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:24.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f434ffff640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f435006e770 0x7f43501564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f434ffff640 1 -- 172.21.15.94:0/4086063123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43500d11c0 msgr2=0x7f43500b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:24.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f434ffff640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43500d11c0 0x7f43500b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f434ffff640 1 -- 172.21.15.94:0/4086063123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435014e460 con 0x7f4350161530 2024-04-26T02:28:24.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f434effd640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43500d11c0 0x7f43500b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:24.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f434f7fe640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f435006e770 0x7f43501564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:24.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f434ffff640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4350161530 0x7f43500b1a00 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f434801ad20 tx=0x7f4348019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:24.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f4347fff640 1 -- 172.21.15.94:0/4086063123 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43480bc020 con 0x7f4350161530 2024-04-26T02:28:24.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f4347fff640 1 -- 172.21.15.94:0/4086063123 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43480b8670 con 0x7f4350161530 2024-04-26T02:28:24.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f4347fff640 1 -- 172.21.15.94:0/4086063123 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348009510 con 0x7f4350161530 2024-04-26T02:28:24.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f4355c72640 1 -- 172.21.15.94:0/4086063123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435014dbc0 con 0x7f4350161530 2024-04-26T02:28:24.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.506+0000 7f4355c72640 1 -- 172.21.15.94:0/4086063123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435014e180 con 0x7f4350161530 2024-04-26T02:28:24.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.510+0000 7f4355c72640 1 -- 172.21.15.94:0/4086063123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f435014d930 con 0x7f4350161530 2024-04-26T02:28:24.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.510+0000 7f4347fff640 1 -- 172.21.15.94:0/4086063123 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f43480bf020 con 0x7f4350161530 2024-04-26T02:28:24.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.510+0000 7f4347fff640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f433006f400 0x7f4330071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:24.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.510+0000 7f4347fff640 1 -- 172.21.15.94:0/4086063123 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f4348129e70 con 0x7f4350161530 2024-04-26T02:28:24.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.510+0000 7f434f7fe640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f433006f400 0x7f4330071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:24.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.510+0000 7f434f7fe640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f433006f400 0x7f4330071910 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f43400097a0 tx=0x7f4340008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:24.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.514+0000 7f4347fff640 1 -- 172.21.15.94:0/4086063123 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f43480c6030 con 0x7f4350161530 2024-04-26T02:28:24.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.678+0000 7f4355c72640 1 -- 172.21.15.94:0/4086063123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f435006a730 con 0x7f4350161530 2024-04-26T02:28:24.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.682+0000 7f4347fff640 1 -- 172.21.15.94:0/4086063123 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+13305 (secure 0 0 0) 0x7f43480165e0 con 0x7f4350161530 2024-04-26T02:28:24.687 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:24.688 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":43,"fsid":"cc26689a-ed7a-405e-9abc-b7fa2c1906cf","created":"2024-04-25T16:50:28.948173+0000","modified":"2024-04-26T02:28:19.211001+0000","last_up_change":"2024-04-25T16:50:40.356281+0000","last_in_change":"2024-04-25T16:50:31.570598+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:50:41.716256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"394e2fce-26fe-473e-930b-ce703137119c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6801","nonce":3354844003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6803","nonce":3354844003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6807","nonce":3354844003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3354844003},{"type":"v1","addr":"172.21.15.94:6805","nonce":3354844003}]},"public_addr":"172.21.15.94:6801/3354844003","cluster_addr":"172.21.15.94:6803/3354844003","heartbeat_back_addr":"172.21.15.94:6807/3354844003","heartbeat_front_addr":"172.21.15.94:6805/3354844003","state":["exists","up"]},{"osd":1,"uuid":"47b97498-86d9-4503-897a-96333028e6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6809","nonce":3897403281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6811","nonce":3897403281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6815","nonce":3897403281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3897403281},{"type":"v1","addr":"172.21.15.94:6813","nonce":3897403281}]},"public_addr":"172.21.15.94:6809/3897403281","cluster_addr":"172.21.15.94:6811/3897403281","heartbeat_back_addr":"172.21.15.94:6815/3897403281","heartbeat_front_addr":"172.21.15.94:6813/3897403281","state":["exists","up"]},{"osd":2,"uuid":"606ece18-5501-48bd-b735-45a4c435c03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6825","nonce":1121746691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6827","nonce":1121746691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6831","nonce":1121746691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1121746691},{"type":"v1","addr":"172.21.15.94:6829","nonce":1121746691}]},"public_addr":"172.21.15.94:6825/1121746691","cluster_addr":"172.21.15.94:6827/1121746691","heartbeat_back_addr":"172.21.15.94:6831/1121746691","heartbeat_front_addr":"172.21.15.94:6829/1121746691","state":["exists","up"]},{"osd":3,"uuid":"8463d045-b8bb-493a-a1fb-7b3029fbe685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6817","nonce":1777163555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6819","nonce":1777163555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6823","nonce":1777163555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1777163555},{"type":"v1","addr":"172.21.15.94:6821","nonce":1777163555}]},"public_addr":"172.21.15.94:6817/1777163555","cluster_addr":"172.21.15.94:6819/1777163555","heartbeat_back_addr":"172.21.15.94:6823/1777163555","heartbeat_front_addr":"172.21.15.94:6821/1777163555","state":["exists","up"]},{"osd":4,"uuid":"1768cc78-6c07-42e8-b4df-a24f2b58540c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6817","nonce":4237363180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6819","nonce":4237363180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6824","nonce":4237363180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":4237363180},{"type":"v1","addr":"172.21.15.138:6821","nonce":4237363180}]},"public_addr":"172.21.15.138:6817/4237363180","cluster_addr":"172.21.15.138:6819/4237363180","heartbeat_back_addr":"172.21.15.138:6824/4237363180","heartbeat_front_addr":"172.21.15.138:6821/4237363180","state":["exists","up"]},{"osd":5,"uuid":"8ee52e85-51e1-4444-92ba-696df3573c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6823","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6825","nonce":2199406473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6827","nonce":2199406473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6831","nonce":2199406473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2199406473},{"type":"v1","addr":"172.21.15.138:6829","nonce":2199406473}]},"public_addr":"172.21.15.138:6825/2199406473","cluster_addr":"172.21.15.138:6827/2199406473","heartbeat_back_addr":"172.21.15.138:6831/2199406473","heartbeat_front_addr":"172.21.15.138:6829/2199406473","state":["exists","up"]},{"osd":6,"uuid":"8df799bb-47cd-4458-b347-bed7f533b6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6804","nonce":3744891971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6808","nonce":3744891971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6815","nonce":3744891971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3744891971},{"type":"v1","addr":"172.21.15.138:6812","nonce":3744891971}]},"public_addr":"172.21.15.138:6804/3744891971","cluster_addr":"172.21.15.138:6808/3744891971","heartbeat_back_addr":"172.21.15.138:6815/3744891971","heartbeat_front_addr":"172.21.15.138:6812/3744891971","state":["exists","up"]},{"osd":7,"uuid":"27e6f89e-2d85-4888-b0d3-bbb58d15544b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6801","nonce":3634138036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6805","nonce":3634138036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6813","nonce":3634138036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3634138036},{"type":"v1","addr":"172.21.15.138:6809","nonce":3634138036}]},"public_addr":"172.21.15.138:6801/3634138036","cluster_addr":"172.21.15.138:6805/3634138036","heartbeat_back_addr":"172.21.15.138:6813/3634138036","heartbeat_front_addr":"172.21.15.138:6809/3634138036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.074506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.184970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.235792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.101011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.118492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.104935+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.202183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:50:37.234644+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6837/2235039329":"2024-04-27T02:28:16.106006+0000","172.21.15.94:6836/2235039329":"2024-04-27T02:28:16.106006+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T02:28:24.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.686+0000 7f4355c72640 1 -- 172.21.15.94:0/4086063123 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f433006f400 msgr2=0x7f4330071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:24.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.686+0000 7f4355c72640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f433006f400 0x7f4330071910 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f43400097a0 tx=0x7f4340008040 comp rx=0 tx=0).stop 2024-04-26T02:28:24.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.686+0000 7f4355c72640 1 -- 172.21.15.94:0/4086063123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4350161530 msgr2=0x7f43500b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:24.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.686+0000 7f4355c72640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4350161530 0x7f43500b1a00 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f434801ad20 tx=0x7f4348019410 comp rx=0 tx=0).stop 2024-04-26T02:28:24.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.686+0000 7f4355c72640 1 -- 172.21.15.94:0/4086063123 shutdown_connections 2024-04-26T02:28:24.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.686+0000 7f4355c72640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4350161530 0x7f43500b1a00 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.686+0000 7f4355c72640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43500d11c0 0x7f43500b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.686+0000 7f4355c72640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f433006f400 0x7f4330071910 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.686+0000 7f4355c72640 1 --2- 172.21.15.94:0/4086063123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f435006e770 0x7f43501564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:24.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.686+0000 7f4355c72640 1 -- 172.21.15.94:0/4086063123 >> 172.21.15.94:0/4086063123 conn(0x7f43500d1d30 msgr2=0x7f43500d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:24.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.686+0000 7f4355c72640 1 -- 172.21.15.94:0/4086063123 shutdown_connections 2024-04-26T02:28:24.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:24.686+0000 7f4355c72640 1 -- 172.21.15.94:0/4086063123 wait complete. 2024-04-26T02:28:25.715 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T02:28:25.715 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T02:28:25.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.878+0000 7f058dd21640 1 Processor -- start 2024-04-26T02:28:25.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.878+0000 7f058dd21640 1 -- start start 2024-04-26T02:28:25.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.878+0000 7f058dd21640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f058806fa60 0x7f0588164510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.878+0000 7f058dd21640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05880d50c0 con 0x7f05880d3350 2024-04-26T02:28:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.878+0000 7f058dd21640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05880d53a0 con 0x7f05880d59d0 2024-04-26T02:28:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.878+0000 7f058dd21640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0588163a30 con 0x7f058806fa60 2024-04-26T02:28:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.878+0000 7f0587fff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f05880d3350 0x7f05880d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56090/0 (socket says 172.21.15.94:56090) 2024-04-26T02:28:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.878+0000 7f0587fff640 1 -- 172.21.15.94:0/3514046260 learned_addr learned my addr 172.21.15.94:0/3514046260 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05877fe640 1 --2- 172.21.15.94:0/3514046260 >> v2:172.21.15.138:3300/0 conn(0x7f058806fa60 0x7f0588164510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728410636 0 0) 0x7f05880d50c0 con 0x7f05880d3350 2024-04-26T02:28:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0570003530 con 0x7f05880d3350 2024-04-26T02:28:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2000069055 0 0) 0x7f0570003530 con 0x7f05880d3350 2024-04-26T02:28:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 >> v2:172.21.15.138:3300/0 conn(0x7f058806fa60 msgr2=0x7f0588164510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 --2- 172.21.15.94:0/3514046260 >> v2:172.21.15.138:3300/0 conn(0x7f058806fa60 0x7f0588164510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 >> v1:172.21.15.138:6789/0 conn(0x7f05880d59d0 legacy=0x7f05880d5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0588163d10 con 0x7f05880d3350 2024-04-26T02:28:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f0580001a00 con 0x7f05880d3350 2024-04-26T02:28:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 >> v1:172.21.15.94:6789/0 conn(0x7f05880d3350 legacy=0x7f05880d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 --2- 172.21.15.94:0/3514046260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0570003a30 0x7f0570003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 --2- 172.21.15.94:0/3514046260 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0570004810 0x7f0570004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 --2- 172.21.15.94:0/3514046260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05700053b0 0x7f057000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0588163a30 con 0x7f05700053b0 2024-04-26T02:28:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f057000c310 con 0x7f0570003a30 2024-04-26T02:28:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f057000c510 con 0x7f0570004810 2024-04-26T02:28:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f0587fff640 1 --2- 172.21.15.94:0/3514046260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05700053b0 0x7f057000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05877fe640 1 --2- 172.21.15.94:0/3514046260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0570003a30 0x7f0570003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f0586ffd640 1 --2- 172.21.15.94:0/3514046260 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0570004810 0x7f0570004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f0587fff640 1 -- 172.21.15.94:0/3514046260 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0570004810 msgr2=0x7f0570004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f0587fff640 1 --2- 172.21.15.94:0/3514046260 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0570004810 0x7f0570004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f0587fff640 1 -- 172.21.15.94:0/3514046260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0570003a30 msgr2=0x7f0570003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f0587fff640 1 --2- 172.21.15.94:0/3514046260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0570003a30 0x7f0570003ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f0587fff640 1 -- 172.21.15.94:0/3514046260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0570003530 con 0x7f05700053b0 2024-04-26T02:28:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f0587fff640 1 -- 172.21.15.94:0/3514046260 reap_dead start 2024-04-26T02:28:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f0587fff640 1 --2- 172.21.15.94:0/3514046260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05700053b0 0x7f057000ba30 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f058000a810 tx=0x7f058001faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c14db946a662d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058000a460 con 0x7f05700053b0 2024-04-26T02:28:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05800aa6f0 con 0x7f05700053b0 2024-04-26T02:28:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.882+0000 7f05867fc640 1 -- 172.21.15.94:0/3514046260 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05800aaae0 con 0x7f05700053b0 2024-04-26T02:28:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.886+0000 7f058dd21640 1 -- 172.21.15.94:0/3514046260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05700053b0 msgr2=0x7f057000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.886+0000 7f058dd21640 1 --2- 172.21.15.94:0/3514046260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05700053b0 0x7f057000ba30 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f058000a810 tx=0x7f058001faa0 comp rx=0 tx=0).stop 2024-04-26T02:28:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.886+0000 7f058dd21640 1 -- 172.21.15.94:0/3514046260 shutdown_connections 2024-04-26T02:28:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.886+0000 7f058dd21640 1 --2- 172.21.15.94:0/3514046260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05700053b0 0x7f057000ba30 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.886+0000 7f058dd21640 1 -- 172.21.15.94:0/3514046260 >> 172.21.15.94:0/3514046260 conn(0x7f058809eaf0 msgr2=0x7f058808d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.886+0000 7f058dd21640 1 -- 172.21.15.94:0/3514046260 shutdown_connections 2024-04-26T02:28:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.886+0000 7f058dd21640 1 -- 172.21.15.94:0/3514046260 wait complete. 2024-04-26T02:28:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.886+0000 7f058dd21640 1 Processor -- start 2024-04-26T02:28:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.886+0000 7f058dd21640 1 -- start start 2024-04-26T02:28:25.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f058dd21640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05700053b0 0x7f05880a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:25.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f058dd21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f058815d820 0x7f05880b6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:25.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f058dd21640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880b6b70 0x7f058806d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:25.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f058dd21640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f058808dc40 con 0x7f058815d820 2024-04-26T02:28:25.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f058dd21640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05880d4f20 con 0x7f05880b6b70 2024-04-26T02:28:25.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f058dd21640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05880d1e60 con 0x7f05700053b0 2024-04-26T02:28:25.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f0586ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f058815d820 0x7f05880b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:25.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f0586ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f058815d820 0x7f05880b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50274/0 (socket says 172.21.15.94:50274) 2024-04-26T02:28:25.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f0586ffd640 1 -- 172.21.15.94:0/187382493 learned_addr learned my addr 172.21.15.94:0/187382493 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:25.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f0587fff640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880b6b70 0x7f058806d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:25.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f0586ffd640 1 -- 172.21.15.94:0/187382493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05700053b0 msgr2=0x7f05880a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:25.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f05877fe640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05700053b0 0x7f05880a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:25.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f0586ffd640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05700053b0 0x7f05880a2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:25.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f0586ffd640 1 -- 172.21.15.94:0/187382493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880b6b70 msgr2=0x7f058806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:25.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f0586ffd640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880b6b70 0x7f058806d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:25.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f0586ffd640 1 -- 172.21.15.94:0/187382493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05880a1d60 con 0x7f058815d820 2024-04-26T02:28:25.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f0587fff640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880b6b70 0x7f058806d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:25.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f0586ffd640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f058815d820 0x7f05880b6630 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f057c00d4f0 tx=0x7f057c00d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:25.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f056ffff640 1 -- 172.21.15.94:0/187382493 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c005480 con 0x7f058815d820 2024-04-26T02:28:25.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f056ffff640 1 -- 172.21.15.94:0/187382493 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057c017070 con 0x7f058815d820 2024-04-26T02:28:25.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f058dd21640 1 -- 172.21.15.94:0/187382493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05880a2040 con 0x7f058815d820 2024-04-26T02:28:25.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f056ffff640 1 -- 172.21.15.94:0/187382493 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c003820 con 0x7f058815d820 2024-04-26T02:28:25.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f058dd21640 1 -- 172.21.15.94:0/187382493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05880a2600 con 0x7f058815d820 2024-04-26T02:28:25.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f058dd21640 1 -- 172.21.15.94:0/187382493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f05880d26e0 con 0x7f058815d820 2024-04-26T02:28:25.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f056ffff640 1 -- 172.21.15.94:0/187382493 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f057c026430 con 0x7f058815d820 2024-04-26T02:28:25.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f056ffff640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f057406f400 0x7f0574071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:25.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f056ffff640 1 -- 172.21.15.94:0/187382493 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f057c08f970 con 0x7f058815d820 2024-04-26T02:28:25.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.890+0000 7f05877fe640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f057406f400 0x7f0574071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:25.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.894+0000 7f056ffff640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] conn(0x7f05740758f0 0x7f0574077d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:25.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.894+0000 7f056ffff640 1 -- 172.21.15.94:0/187382493 --> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0574078450 con 0x7f05740758f0 2024-04-26T02:28:25.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.894+0000 7f056ffff640 1 -- 172.21.15.94:0/187382493 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f057c08fe00 con 0x7f058815d820 2024-04-26T02:28:25.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.894+0000 7f0587fff640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] conn(0x7f05740758f0 0x7f0574077d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:25.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.894+0000 7f05877fe640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f057406f400 0x7f0574071910 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f05880c8f20 tx=0x7f0578008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:25.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.894+0000 7f0587fff640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] conn(0x7f05740758f0 0x7f0574077d80 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:25.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.910+0000 7f056ffff640 1 -- 172.21.15.94:0/187382493 <== osd.0 v2:172.21.15.94:6800/3354844003 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0574078450 con 0x7f05740758f0 2024-04-26T02:28:25.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.926+0000 7f058dd21640 1 -- 172.21.15.94:0/187382493 --> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f05881ea610 con 0x7f05740758f0 2024-04-26T02:28:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.938+0000 7f056ffff640 1 -- 172.21.15.94:0/187382493 <== osd.0 v2:172.21.15.94:6800/3354844003 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f05881ea610 con 0x7f05740758f0 2024-04-26T02:28:25.941 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T02:28:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T02:28:25.942 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T02:28:25.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.938+0000 7f058dd21640 1 -- 172.21.15.94:0/187382493 >> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] conn(0x7f05740758f0 msgr2=0x7f0574077d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:25.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.938+0000 7f058dd21640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.94:6800/3354844003,v1:172.21.15.94:6801/3354844003] conn(0x7f05740758f0 0x7f0574077d80 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:25.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.938+0000 7f058dd21640 1 -- 172.21.15.94:0/187382493 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f057406f400 msgr2=0x7f0574071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:25.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.938+0000 7f058dd21640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f057406f400 0x7f0574071910 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f05880c8f20 tx=0x7f0578008040 comp rx=0 tx=0).stop 2024-04-26T02:28:25.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.938+0000 7f058dd21640 1 -- 172.21.15.94:0/187382493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f058815d820 msgr2=0x7f05880b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:25.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.938+0000 7f058dd21640 1 --2- 172.21.15.94:0/187382493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f058815d820 0x7f05880b6630 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f057c00d4f0 tx=0x7f057c00d9b0 comp rx=0 tx=0).stop 2024-04-26T02:28:25.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.938+0000 7f0587fff640 1 -- 172.21.15.94:0/187382493 reap_dead start 2024-04-26T02:28:25.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.938+0000 7f058dd21640 1 -- 172.21.15.94:0/187382493 shutdown_connections 2024-04-26T02:28:25.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.938+0000 7f058dd21640 1 -- 172.21.15.94:0/187382493 >> 172.21.15.94:0/187382493 conn(0x7f058809eaf0 msgr2=0x7f05880b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:25.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.938+0000 7f058dd21640 1 -- 172.21.15.94:0/187382493 shutdown_connections 2024-04-26T02:28:25.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:25.938+0000 7f058dd21640 1 -- 172.21.15.94:0/187382493 wait complete. 2024-04-26T02:28:25.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T02:28:26.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.118+0000 7f60a1ed8640 1 Processor -- start 2024-04-26T02:28:26.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f60a1ed8640 1 -- start start 2024-04-26T02:28:26.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f60a1ed8640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f609c06f470 0x7f609c15dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f60a1ed8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f609c15e160 0x7f609c15e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f60a1ed8640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f609c070630 con 0x7f609c0b0420 2024-04-26T02:28:26.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f60a1ed8640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f609c0aeaa0 con 0x7f609c06f470 2024-04-26T02:28:26.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f60a1ed8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f609c0703e0 con 0x7f609c15e160 2024-04-26T02:28:26.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f609affd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f609c06f470 0x7f609c15dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f609b7fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f609c0b0420 0x7f609c06ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56098/0 (socket says 172.21.15.94:56098) 2024-04-26T02:28:26.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f609b7fe640 1 -- 172.21.15.94:0/2782301907 learned_addr learned my addr 172.21.15.94:0/2782301907 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f609bfff640 1 --2- 172.21.15.94:0/2782301907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f609c15e160 0x7f609c15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f609affd640 1 -- 172.21.15.94:0/2782301907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f609c15e160 msgr2=0x7f609c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f609affd640 1 --2- 172.21.15.94:0/2782301907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f609c15e160 0x7f609c15e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f609affd640 1 -- 172.21.15.94:0/2782301907 >> v1:172.21.15.94:6789/0 conn(0x7f609c0b0420 legacy=0x7f609c06ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f609a7fc640 1 -- 172.21.15.94:0/2782301907 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2631924938 0 0) 0x7f609c070630 con 0x7f609c0b0420 2024-04-26T02:28:26.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.122+0000 7f609affd640 1 -- 172.21.15.94:0/2782301907 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609c0ae170 con 0x7f609c06f470 2024-04-26T02:28:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.126+0000 7f609affd640 1 --2- 172.21.15.94:0/2782301907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c06f470 0x7f609c15dc20 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f60900097c0 tx=0x7f609001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a8dc343781feb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:26.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.126+0000 7f609a7fc640 1 -- 172.21.15.94:0/2782301907 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60900a7070 con 0x7f609c06f470 2024-04-26T02:28:26.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.126+0000 7f609a7fc640 1 -- 172.21.15.94:0/2782301907 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60900b0070 con 0x7f609c06f470 2024-04-26T02:28:26.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.126+0000 7f609a7fc640 1 -- 172.21.15.94:0/2782301907 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60900ab890 con 0x7f609c06f470 2024-04-26T02:28:26.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.126+0000 7f60a1ed8640 1 -- 172.21.15.94:0/2782301907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c06f470 msgr2=0x7f609c15dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.126+0000 7f60a1ed8640 1 --2- 172.21.15.94:0/2782301907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c06f470 0x7f609c15dc20 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f60900097c0 tx=0x7f609001c470 comp rx=0 tx=0).stop 2024-04-26T02:28:26.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.126+0000 7f60a1ed8640 1 -- 172.21.15.94:0/2782301907 shutdown_connections 2024-04-26T02:28:26.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.126+0000 7f60a1ed8640 1 --2- 172.21.15.94:0/2782301907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f609c15e160 0x7f609c15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.126+0000 7f60a1ed8640 1 --2- 172.21.15.94:0/2782301907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c06f470 0x7f609c15dc20 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.126+0000 7f60a1ed8640 1 -- 172.21.15.94:0/2782301907 >> 172.21.15.94:0/2782301907 conn(0x7f609c0b47f0 msgr2=0x7f609c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.126+0000 7f60a1ed8640 1 -- 172.21.15.94:0/2782301907 shutdown_connections 2024-04-26T02:28:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.126+0000 7f60a1ed8640 1 -- 172.21.15.94:0/2782301907 wait complete. 2024-04-26T02:28:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f60a1ed8640 1 Processor -- start 2024-04-26T02:28:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f60a1ed8640 1 -- start start 2024-04-26T02:28:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f60a1ed8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f609c0b0420 0x7f609c09a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f60a1ed8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f609c15e160 0x7f609c09aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f60a1ed8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c073150 0x7f609c15ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f60a1ed8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f609c0b09f0 con 0x7f609c073150 2024-04-26T02:28:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f60a1ed8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f609c0b0870 con 0x7f609c15e160 2024-04-26T02:28:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f60a1ed8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f609c16a6f0 con 0x7f609c0b0420 2024-04-26T02:28:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f609bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c073150 0x7f609c15ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f609bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c073150 0x7f609c15ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50290/0 (socket says 172.21.15.94:50290) 2024-04-26T02:28:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f609bfff640 1 -- 172.21.15.94:0/749382546 learned_addr learned my addr 172.21.15.94:0/749382546 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f609b7fe640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f609c0b0420 0x7f609c09a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f609affd640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f609c15e160 0x7f609c09aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f609bfff640 1 -- 172.21.15.94:0/749382546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f609c0b0420 msgr2=0x7f609c09a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f609bfff640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f609c0b0420 0x7f609c09a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f609bfff640 1 -- 172.21.15.94:0/749382546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f609c15e160 msgr2=0x7f609c09aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f609bfff640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f609c15e160 0x7f609c09aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f609bfff640 1 -- 172.21.15.94:0/749382546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609c0738c0 con 0x7f609c073150 2024-04-26T02:28:26.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f609b7fe640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f609c0b0420 0x7f609c09a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:26.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f609affd640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f609c15e160 0x7f609c09aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:26.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f609bfff640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c073150 0x7f609c15ca40 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f609400d550 tx=0x7f609400da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:26.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f6083fff640 1 -- 172.21.15.94:0/749382546 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940125f0 con 0x7f609c073150 2024-04-26T02:28:26.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f6083fff640 1 -- 172.21.15.94:0/749382546 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6094019070 con 0x7f609c073150 2024-04-26T02:28:26.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f60a1ed8640 1 -- 172.21.15.94:0/749382546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609c073ba0 con 0x7f609c073150 2024-04-26T02:28:26.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f6083fff640 1 -- 172.21.15.94:0/749382546 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940129d0 con 0x7f609c073150 2024-04-26T02:28:26.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f60a1ed8640 1 -- 172.21.15.94:0/749382546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609c074160 con 0x7f609c073150 2024-04-26T02:28:26.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.130+0000 7f60a1ed8640 1 -- 172.21.15.94:0/749382546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f609c073630 con 0x7f609c073150 2024-04-26T02:28:26.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.134+0000 7f6083fff640 1 -- 172.21.15.94:0/749382546 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6094012b70 con 0x7f609c073150 2024-04-26T02:28:26.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.134+0000 7f6083fff640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f608806f400 0x7f6088071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.134+0000 7f6083fff640 1 -- 172.21.15.94:0/749382546 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f609408f560 con 0x7f609c073150 2024-04-26T02:28:26.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.134+0000 7f609b7fe640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f608806f400 0x7f6088071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.134+0000 7f609b7fe640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f608806f400 0x7f6088071910 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f608c0012f0 tx=0x7f608c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:26.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.138+0000 7f6083fff640 1 -- 172.21.15.94:0/749382546 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f609405c5c0 con 0x7f609c073150 2024-04-26T02:28:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.302+0000 7f60a1ed8640 1 -- 172.21.15.94:0/749382546 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f609c0743a0 con 0x7f608806f400 2024-04-26T02:28:26.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.302+0000 7f6083fff640 1 -- 172.21.15.94:0/749382546 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f609c0743a0 con 0x7f608806f400 2024-04-26T02:28:26.307 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T02:28:26.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.306+0000 7f60a1ed8640 1 -- 172.21.15.94:0/749382546 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f608806f400 msgr2=0x7f6088071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.306+0000 7f60a1ed8640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f608806f400 0x7f6088071910 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f608c0012f0 tx=0x7f608c009040 comp rx=0 tx=0).stop 2024-04-26T02:28:26.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.306+0000 7f60a1ed8640 1 -- 172.21.15.94:0/749382546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c073150 msgr2=0x7f609c15ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.306+0000 7f60a1ed8640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c073150 0x7f609c15ca40 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f609400d550 tx=0x7f609400da10 comp rx=0 tx=0).stop 2024-04-26T02:28:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.306+0000 7f60a1ed8640 1 -- 172.21.15.94:0/749382546 shutdown_connections 2024-04-26T02:28:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.306+0000 7f60a1ed8640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f609c073150 0x7f609c15ca40 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.306+0000 7f60a1ed8640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f609c15e160 0x7f609c09aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.306+0000 7f60a1ed8640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f608806f400 0x7f6088071910 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.306+0000 7f60a1ed8640 1 --2- 172.21.15.94:0/749382546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f609c0b0420 0x7f609c09a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.306+0000 7f60a1ed8640 1 -- 172.21.15.94:0/749382546 >> 172.21.15.94:0/749382546 conn(0x7f609c0b47f0 msgr2=0x7f609c160a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:26.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.306+0000 7f60a1ed8640 1 -- 172.21.15.94:0/749382546 shutdown_connections 2024-04-26T02:28:26.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.306+0000 7f60a1ed8640 1 -- 172.21.15.94:0/749382546 wait complete. 2024-04-26T02:28:26.331 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T02:28:26.331 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T02:28:26.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.482+0000 7f902e409640 1 Processor -- start 2024-04-26T02:28:26.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.482+0000 7f902e409640 1 -- start start 2024-04-26T02:28:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.482+0000 7f902e409640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f902806c600 0x7f902806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.482+0000 7f902e409640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f902806fcf0 0x7f9028070100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.482+0000 7f902e409640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f902806d230 con 0x7f9028161580 2024-04-26T02:28:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.482+0000 7f902e409640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f902806ec30 con 0x7f902806fcf0 2024-04-26T02:28:26.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.482+0000 7f902e409640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f902806d510 con 0x7f902806c600 2024-04-26T02:28:26.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f901ffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f902806fcf0 0x7f9028070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f901ffff640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f902806fcf0 0x7f9028070100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50304/0 (socket says 172.21.15.94:50304) 2024-04-26T02:28:26.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f901ffff640 1 -- 172.21.15.94:0/2700234231 learned_addr learned my addr 172.21.15.94:0/2700234231 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:26.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f9027fff640 1 --2- 172.21.15.94:0/2700234231 >> v2:172.21.15.138:3300/0 conn(0x7f902806c600 0x7f902806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f901ffff640 1 -- 172.21.15.94:0/2700234231 >> v2:172.21.15.138:3300/0 conn(0x7f902806c600 msgr2=0x7f902806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f901ffff640 1 --2- 172.21.15.94:0/2700234231 >> v2:172.21.15.138:3300/0 conn(0x7f902806c600 0x7f902806ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f901ffff640 1 -- 172.21.15.94:0/2700234231 >> v1:172.21.15.138:6789/0 conn(0x7f9028161580 legacy=0x7f9028161990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f901ffff640 1 -- 172.21.15.94:0/2700234231 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902806d7f0 con 0x7f902806fcf0 2024-04-26T02:28:26.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f901ffff640 1 --2- 172.21.15.94:0/2700234231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902806fcf0 0x7f9028070100 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f90280b73a0 tx=0x7f901001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac13ef2bf160c88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:26.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f90277fe640 1 -- 172.21.15.94:0/2700234231 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100aa610 con 0x7f902806fcf0 2024-04-26T02:28:26.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f90277fe640 1 -- 172.21.15.94:0/2700234231 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90100af070 con 0x7f902806fcf0 2024-04-26T02:28:26.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f90277fe640 1 -- 172.21.15.94:0/2700234231 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100aa9d0 con 0x7f902806fcf0 2024-04-26T02:28:26.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f902e409640 1 -- 172.21.15.94:0/2700234231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902806fcf0 msgr2=0x7f9028070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f902e409640 1 --2- 172.21.15.94:0/2700234231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902806fcf0 0x7f9028070100 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f90280b73a0 tx=0x7f901001c4b0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f902e409640 1 -- 172.21.15.94:0/2700234231 shutdown_connections 2024-04-26T02:28:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f902e409640 1 --2- 172.21.15.94:0/2700234231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902806fcf0 0x7f9028070100 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f902e409640 1 --2- 172.21.15.94:0/2700234231 >> v2:172.21.15.138:3300/0 conn(0x7f902806c600 0x7f902806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f902e409640 1 -- 172.21.15.94:0/2700234231 >> 172.21.15.94:0/2700234231 conn(0x7f90280d13c0 msgr2=0x7f90280d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f902e409640 1 -- 172.21.15.94:0/2700234231 shutdown_connections 2024-04-26T02:28:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.486+0000 7f902e409640 1 -- 172.21.15.94:0/2700234231 wait complete. 2024-04-26T02:28:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f902e409640 1 Processor -- start 2024-04-26T02:28:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f902e409640 1 -- start start 2024-04-26T02:28:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f902e409640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f902806c600 0x7f9028156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f902e409640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902806fcf0 0x7f902809b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f902e409640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9028161580 0x7f902809b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f902e409640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f90280ccc00 con 0x7f902806fcf0 2024-04-26T02:28:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f902e409640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90280b0b80 con 0x7f902806c600 2024-04-26T02:28:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f902e409640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f90280ccd80 con 0x7f9028161580 2024-04-26T02:28:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f901ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902806fcf0 0x7f902809b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f901ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902806fcf0 0x7f902809b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50308/0 (socket says 172.21.15.94:50308) 2024-04-26T02:28:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f901ffff640 1 -- 172.21.15.94:0/2205739785 learned_addr learned my addr 172.21.15.94:0/2205739785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f901ffff640 1 -- 172.21.15.94:0/2205739785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9028161580 msgr2=0x7f902809b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f9027fff640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f902806c600 0x7f9028156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f902c9a6640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9028161580 0x7f902809b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f901ffff640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9028161580 0x7f902809b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f901ffff640 1 -- 172.21.15.94:0/2205739785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f902806c600 msgr2=0x7f9028156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f901ffff640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f902806c600 0x7f9028156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f901ffff640 1 -- 172.21.15.94:0/2205739785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9028157ce0 con 0x7f902806fcf0 2024-04-26T02:28:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f9027fff640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f902806c600 0x7f9028156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.490+0000 7f901ffff640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902806fcf0 0x7f902809b1e0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f90100ac040 tx=0x7f901001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f90257fa640 1 -- 172.21.15.94:0/2205739785 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100bd5f0 con 0x7f902806fcf0 2024-04-26T02:28:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f90257fa640 1 -- 172.21.15.94:0/2205739785 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90100bd7b0 con 0x7f902806fcf0 2024-04-26T02:28:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f902e409640 1 -- 172.21.15.94:0/2205739785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90281580b0 con 0x7f902806fcf0 2024-04-26T02:28:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f90257fa640 1 -- 172.21.15.94:0/2205739785 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100bdba0 con 0x7f902806fcf0 2024-04-26T02:28:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f902e409640 1 -- 172.21.15.94:0/2205739785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9028158670 con 0x7f902806fcf0 2024-04-26T02:28:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f902e409640 1 -- 172.21.15.94:0/2205739785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f90280d2260 con 0x7f902806fcf0 2024-04-26T02:28:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f90257fa640 1 -- 172.21.15.94:0/2205739785 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f90100b33f0 con 0x7f902806fcf0 2024-04-26T02:28:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f90257fa640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f900806f400 0x7f9008071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f90257fa640 1 -- 172.21.15.94:0/2205739785 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f90101298c0 con 0x7f902806fcf0 2024-04-26T02:28:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f9027fff640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f900806f400 0x7f9008071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f90257fa640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] conn(0x7f90080758f0 0x7f9008077d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f90257fa640 1 -- 172.21.15.94:0/2205739785 --> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f901000af30 con 0x7f90080758f0 2024-04-26T02:28:26.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f90257fa640 1 -- 172.21.15.94:0/2205739785 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9010129d50 con 0x7f902806fcf0 2024-04-26T02:28:26.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f902c9a6640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] conn(0x7f90080758f0 0x7f9008077d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f9027fff640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f900806f400 0x7f9008071910 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f90180097a0 tx=0x7f9018008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:26.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.494+0000 7f902c9a6640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] conn(0x7f90080758f0 0x7f9008077d80 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:26.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.498+0000 7f90257fa640 1 -- 172.21.15.94:0/2205739785 <== osd.1 v2:172.21.15.94:6808/3897403281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f901000af30 con 0x7f90080758f0 2024-04-26T02:28:26.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.510+0000 7f902e409640 1 -- 172.21.15.94:0/2205739785 --> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f902809e6a0 con 0x7f90080758f0 2024-04-26T02:28:26.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.510+0000 7f90257fa640 1 -- 172.21.15.94:0/2205739785 <== osd.1 v2:172.21.15.94:6808/3897403281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f902809e6a0 con 0x7f90080758f0 2024-04-26T02:28:26.515 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T02:28:26.515 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T02:28:26.516 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T02:28:26.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.510+0000 7f902e409640 1 -- 172.21.15.94:0/2205739785 >> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] conn(0x7f90080758f0 msgr2=0x7f9008077d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.510+0000 7f902e409640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.94:6808/3897403281,v1:172.21.15.94:6809/3897403281] conn(0x7f90080758f0 0x7f9008077d80 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.510+0000 7f902e409640 1 -- 172.21.15.94:0/2205739785 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f900806f400 msgr2=0x7f9008071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.510+0000 7f902e409640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f900806f400 0x7f9008071910 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f90180097a0 tx=0x7f9018008040 comp rx=0 tx=0).stop 2024-04-26T02:28:26.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.510+0000 7f902e409640 1 -- 172.21.15.94:0/2205739785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902806fcf0 msgr2=0x7f902809b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.510+0000 7f902e409640 1 --2- 172.21.15.94:0/2205739785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902806fcf0 0x7f902809b1e0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f90100ac040 tx=0x7f901001acf0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.510+0000 7f902c9a6640 1 -- 172.21.15.94:0/2205739785 reap_dead start 2024-04-26T02:28:26.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.510+0000 7f902e409640 1 -- 172.21.15.94:0/2205739785 shutdown_connections 2024-04-26T02:28:26.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.510+0000 7f902e409640 1 -- 172.21.15.94:0/2205739785 >> 172.21.15.94:0/2205739785 conn(0x7f90280d13c0 msgr2=0x7f90280cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:26.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.510+0000 7f902e409640 1 -- 172.21.15.94:0/2205739785 shutdown_connections 2024-04-26T02:28:26.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.510+0000 7f902e409640 1 -- 172.21.15.94:0/2205739785 wait complete. 2024-04-26T02:28:26.536 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T02:28:26.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e10afa640 1 Processor -- start 2024-04-26T02:28:26.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e10afa640 1 -- start start 2024-04-26T02:28:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e10afa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8e0c0b0420 0x7f8e0c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e10afa640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e0c070630 con 0x7f8e0c06f2a0 2024-04-26T02:28:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e10afa640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e0c06f6b0 con 0x7f8e0c15ec30 2024-04-26T02:28:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e10afa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8e0c0aeaa0 con 0x7f8e0c0b0420 2024-04-26T02:28:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e09d9b640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f8e0c06f2a0 0x7f8e0c06c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56104/0 (socket says 172.21.15.94:56104) 2024-04-26T02:28:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e09d9b640 1 -- 172.21.15.94:0/3829755535 learned_addr learned my addr 172.21.15.94:0/3829755535 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e0a59c640 1 --2- 172.21.15.94:0/3829755535 >> v2:172.21.15.138:3300/0 conn(0x7f8e0c0b0420 0x7f8e0c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 736049796 0 0) 0x7f8e0c070630 con 0x7f8e0c06f2a0 2024-04-26T02:28:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8df4003530 con 0x7f8e0c06f2a0 2024-04-26T02:28:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 293989894 0 0) 0x7f8df4003530 con 0x7f8e0c06f2a0 2024-04-26T02:28:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 >> v2:172.21.15.138:3300/0 conn(0x7f8e0c0b0420 msgr2=0x7f8e0c06ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e0959a640 1 --2- 172.21.15.94:0/3829755535 >> v2:172.21.15.138:3300/0 conn(0x7f8e0c0b0420 0x7f8e0c06ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 >> v1:172.21.15.138:6789/0 conn(0x7f8e0c15ec30 legacy=0x7f8e0c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c06d420 con 0x7f8e0c06f2a0 2024-04-26T02:28:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f8e00001680 con 0x7f8e0c06f2a0 2024-04-26T02:28:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.678+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 >> v1:172.21.15.94:6789/0 conn(0x7f8e0c06f2a0 legacy=0x7f8e0c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0959a640 1 --2- 172.21.15.94:0/3829755535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df40039c0 0x7f8df4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0959a640 1 --2- 172.21.15.94:0/3829755535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df4004810 0x7f8df4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0959a640 1 --2- 172.21.15.94:0/3829755535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8df40051d0 0x7f8df400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c0aeaa0 con 0x7f8df40051d0 2024-04-26T02:28:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8df400c170 con 0x7f8df40039c0 2024-04-26T02:28:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8df400c370 con 0x7f8df4004810 2024-04-26T02:28:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0ad9d640 1 --2- 172.21.15.94:0/3829755535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8df40051d0 0x7f8df400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0a59c640 1 --2- 172.21.15.94:0/3829755535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df40039c0 0x7f8df4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e09d9b640 1 --2- 172.21.15.94:0/3829755535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df4004810 0x7f8df4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0ad9d640 1 -- 172.21.15.94:0/3829755535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df4004810 msgr2=0x7f8df4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0ad9d640 1 --2- 172.21.15.94:0/3829755535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df4004810 0x7f8df4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0ad9d640 1 -- 172.21.15.94:0/3829755535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df40039c0 msgr2=0x7f8df4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0ad9d640 1 --2- 172.21.15.94:0/3829755535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df40039c0 0x7f8df4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0ad9d640 1 -- 172.21.15.94:0/3829755535 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df4003530 con 0x7f8df40051d0 2024-04-26T02:28:26.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e09d9b640 1 --2- 172.21.15.94:0/3829755535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8df4004810 0x7f8df4004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:26.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0ad9d640 1 -- 172.21.15.94:0/3829755535 reap_dead start 2024-04-26T02:28:26.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0ad9d640 1 --2- 172.21.15.94:0/3829755535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8df40051d0 0x7f8df400b9d0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f8e04001e70 tx=0x7f8e0401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41be82f97e4d64a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:26.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04001a90 con 0x7f8df40051d0 2024-04-26T02:28:26.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e04003ba0 con 0x7f8df40051d0 2024-04-26T02:28:26.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e0959a640 1 -- 172.21.15.94:0/3829755535 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e040050d0 con 0x7f8df40051d0 2024-04-26T02:28:26.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e10afa640 1 -- 172.21.15.94:0/3829755535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8df40051d0 msgr2=0x7f8df400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e10afa640 1 --2- 172.21.15.94:0/3829755535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8df40051d0 0x7f8df400b9d0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f8e04001e70 tx=0x7f8e0401c500 comp rx=0 tx=0).stop 2024-04-26T02:28:26.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e10afa640 1 -- 172.21.15.94:0/3829755535 shutdown_connections 2024-04-26T02:28:26.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e10afa640 1 --2- 172.21.15.94:0/3829755535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8df40051d0 0x7f8df400b9d0 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e10afa640 1 -- 172.21.15.94:0/3829755535 >> 172.21.15.94:0/3829755535 conn(0x7f8e0c0b47f0 msgr2=0x7f8e0c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:26.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e10afa640 1 -- 172.21.15.94:0/3829755535 shutdown_connections 2024-04-26T02:28:26.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.682+0000 7f8e10afa640 1 -- 172.21.15.94:0/3829755535 wait complete. 2024-04-26T02:28:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e10afa640 1 Processor -- start 2024-04-26T02:28:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e10afa640 1 -- start start 2024-04-26T02:28:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e10afa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df40051d0 0x7f8e0c1f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e10afa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c09b0e0 0x7f8e0c1e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e10afa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e0c09b4f0 0x7f8e0c1e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e10afa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c0ae730 con 0x7f8e0c09b4f0 2024-04-26T02:28:26.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e10afa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c0ae450 con 0x7f8df40051d0 2024-04-26T02:28:26.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e10afa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0c0703e0 con 0x7f8e0c09b0e0 2024-04-26T02:28:26.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e0ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e0c09b4f0 0x7f8e0c1e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e0ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e0c09b4f0 0x7f8e0c1e99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50324/0 (socket says 172.21.15.94:50324) 2024-04-26T02:28:26.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e0ad9d640 1 -- 172.21.15.94:0/4095346400 learned_addr learned my addr 172.21.15.94:0/4095346400 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:26.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e0a59c640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df40051d0 0x7f8e0c1f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e09d9b640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c09b0e0 0x7f8e0c1e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e0ad9d640 1 -- 172.21.15.94:0/4095346400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c09b0e0 msgr2=0x7f8e0c1e94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e0ad9d640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c09b0e0 0x7f8e0c1e94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e0ad9d640 1 -- 172.21.15.94:0/4095346400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df40051d0 msgr2=0x7f8e0c1f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e0ad9d640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df40051d0 0x7f8e0c1f6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e0ad9d640 1 -- 172.21.15.94:0/4095346400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c1f9dd0 con 0x7f8e0c09b4f0 2024-04-26T02:28:26.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.686+0000 7f8e0ad9d640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e0c09b4f0 0x7f8e0c1e99f0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f8e0401c9e0 tx=0x7f8df4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:26.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.690+0000 7f8dfaffd640 1 -- 172.21.15.94:0/4095346400 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e040b0070 con 0x7f8e0c09b4f0 2024-04-26T02:28:26.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.690+0000 7f8dfaffd640 1 -- 172.21.15.94:0/4095346400 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e04001260 con 0x7f8e0c09b4f0 2024-04-26T02:28:26.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.690+0000 7f8dfaffd640 1 -- 172.21.15.94:0/4095346400 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04007dc0 con 0x7f8e0c09b4f0 2024-04-26T02:28:26.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.690+0000 7f8e10afa640 1 -- 172.21.15.94:0/4095346400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c1fa0b0 con 0x7f8e0c09b4f0 2024-04-26T02:28:26.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.690+0000 7f8e10afa640 1 -- 172.21.15.94:0/4095346400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c1fa670 con 0x7f8e0c09b4f0 2024-04-26T02:28:26.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.690+0000 7f8e10afa640 1 -- 172.21.15.94:0/4095346400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0c076830 con 0x7f8e0c09b4f0 2024-04-26T02:28:26.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.690+0000 7f8dfaffd640 1 -- 172.21.15.94:0/4095346400 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8e04007240 con 0x7f8e0c09b4f0 2024-04-26T02:28:26.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.690+0000 7f8dfaffd640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8dec06f400 0x7f8dec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:26.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.690+0000 7f8dfaffd640 1 -- 172.21.15.94:0/4095346400 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f8e04128a60 con 0x7f8e0c09b4f0 2024-04-26T02:28:26.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.690+0000 7f8e0a59c640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8dec06f400 0x7f8dec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:26.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.690+0000 7f8e0a59c640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8dec06f400 0x7f8dec071910 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0cd2f0 tx=0x7f8dfc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:26.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.694+0000 7f8dfaffd640 1 -- 172.21.15.94:0/4095346400 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8e040ba060 con 0x7f8e0c09b4f0 2024-04-26T02:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.850+0000 7f8e10afa640 1 -- 172.21.15.94:0/4095346400 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f8e0c157dc0 con 0x7f8dec06f400 2024-04-26T02:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.850+0000 7f8dfaffd640 1 -- 172.21.15.94:0/4095346400 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8e0c157dc0 con 0x7f8dec06f400 2024-04-26T02:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T02:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.850+0000 7f8e10afa640 1 -- 172.21.15.94:0/4095346400 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8dec06f400 msgr2=0x7f8dec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.850+0000 7f8e10afa640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8dec06f400 0x7f8dec071910 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0cd2f0 tx=0x7f8dfc008040 comp rx=0 tx=0).stop 2024-04-26T02:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.850+0000 7f8e10afa640 1 -- 172.21.15.94:0/4095346400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e0c09b4f0 msgr2=0x7f8e0c1e99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.850+0000 7f8e10afa640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e0c09b4f0 0x7f8e0c1e99f0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f8e0401c9e0 tx=0x7f8df4005840 comp rx=0 tx=0).stop 2024-04-26T02:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.854+0000 7f8e10afa640 1 -- 172.21.15.94:0/4095346400 shutdown_connections 2024-04-26T02:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.854+0000 7f8e10afa640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e0c09b4f0 0x7f8e0c1e99f0 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.854+0000 7f8e10afa640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8dec06f400 0x7f8dec071910 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.854+0000 7f8e10afa640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c09b0e0 0x7f8e0c1e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.854+0000 7f8e10afa640 1 --2- 172.21.15.94:0/4095346400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df40051d0 0x7f8e0c1f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.854+0000 7f8e10afa640 1 -- 172.21.15.94:0/4095346400 >> 172.21.15.94:0/4095346400 conn(0x7f8e0c0b47f0 msgr2=0x7f8e0c15ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.854+0000 7f8e10afa640 1 -- 172.21.15.94:0/4095346400 shutdown_connections 2024-04-26T02:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:26.854+0000 7f8e10afa640 1 -- 172.21.15.94:0/4095346400 wait complete. 2024-04-26T02:28:26.877 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T02:28:26.877 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T02:28:27.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.018+0000 7f8fc5e29640 1 Processor -- start 2024-04-26T02:28:27.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.018+0000 7f8fc5e29640 1 -- start start 2024-04-26T02:28:27.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fc5e29640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fc00cf3c0 0x7f8fc00ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fc5e29640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8fc00ae5a0 0x7f8fc0073dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fc5e29640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fc0073350 con 0x7f8fc0074300 2024-04-26T02:28:27.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fc5e29640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc00d1210 con 0x7f8fc00ae5a0 2024-04-26T02:28:27.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fc5e29640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc0074710 con 0x7f8fc00cf3c0 2024-04-26T02:28:27.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbffff640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f8fc0074300 0x7f8fc006d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56120/0 (socket says 172.21.15.94:56120) 2024-04-26T02:28:27.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbffff640 1 -- 172.21.15.94:0/446988467 learned_addr learned my addr 172.21.15.94:0/446988467 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:27.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbeffd640 1 --2- 172.21.15.94:0/446988467 >> v2:172.21.15.138:3300/0 conn(0x7f8fc00ae5a0 0x7f8fc0073dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbf7fe640 1 --2- 172.21.15.94:0/446988467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fc00cf3c0 0x7f8fc00ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3504537413 0 0) 0x7f8fc0073350 con 0x7f8fc0074300 2024-04-26T02:28:27.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fa8003530 con 0x7f8fc0074300 2024-04-26T02:28:27.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 377979903 0 0) 0x7f8fa8003530 con 0x7f8fc0074300 2024-04-26T02:28:27.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fc00cf3c0 msgr2=0x7f8fc00ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:27.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 --2- 172.21.15.94:0/446988467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fc00cf3c0 0x7f8fc00ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 >> v2:172.21.15.138:3300/0 conn(0x7f8fc00ae5a0 msgr2=0x7f8fc0073dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:27.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 --2- 172.21.15.94:0/446988467 >> v2:172.21.15.138:3300/0 conn(0x7f8fc00ae5a0 0x7f8fc0073dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc00cfbf0 con 0x7f8fc0074300 2024-04-26T02:28:27.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f8fb8001a00 con 0x7f8fc0074300 2024-04-26T02:28:27.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 >> v1:172.21.15.94:6789/0 conn(0x7f8fc0074300 legacy=0x7f8fc006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 --2- 172.21.15.94:0/446988467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fa8003a30 0x7f8fa8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 --2- 172.21.15.94:0/446988467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fc00cf3c0 0x7f8fa8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 --2- 172.21.15.94:0/446988467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa8004d50 0x7f8fa8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc00d1210 con 0x7f8fa8004d50 2024-04-26T02:28:27.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc0074710 con 0x7f8fa8003a30 2024-04-26T02:28:27.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa800ba80 con 0x7f8fc00cf3c0 2024-04-26T02:28:27.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbffff640 1 --2- 172.21.15.94:0/446988467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa8004d50 0x7f8fa8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbf7fe640 1 --2- 172.21.15.94:0/446988467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fa8003a30 0x7f8fa8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbffff640 1 -- 172.21.15.94:0/446988467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fc00cf3c0 msgr2=0x7f8fa8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:27.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbffff640 1 --2- 172.21.15.94:0/446988467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fc00cf3c0 0x7f8fa8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbffff640 1 -- 172.21.15.94:0/446988467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fa8003a30 msgr2=0x7f8fa8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:27.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbffff640 1 --2- 172.21.15.94:0/446988467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fa8003a30 0x7f8fa8003ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.022+0000 7f8fbffff640 1 -- 172.21.15.94:0/446988467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa8003530 con 0x7f8fa8004d50 2024-04-26T02:28:27.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.026+0000 7f8fbffff640 1 --2- 172.21.15.94:0/446988467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa8004d50 0x7f8fa8005160 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800a950 tx=0x7f8fb801d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff656831b7a58c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.026+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800a4c0 con 0x7f8fa8004d50 2024-04-26T02:28:27.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.026+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb80072f0 con 0x7f8fa8004d50 2024-04-26T02:28:27.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.026+0000 7f8fbe7fc640 1 -- 172.21.15.94:0/446988467 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80076e0 con 0x7f8fa8004d50 2024-04-26T02:28:27.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.026+0000 7f8fc5e29640 1 -- 172.21.15.94:0/446988467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa8004d50 msgr2=0x7f8fa8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.026+0000 7f8fc5e29640 1 --2- 172.21.15.94:0/446988467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa8004d50 0x7f8fa8005160 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800a950 tx=0x7f8fb801d440 comp rx=0 tx=0).stop 2024-04-26T02:28:27.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.026+0000 7f8fbffff640 1 -- 172.21.15.94:0/446988467 reap_dead start 2024-04-26T02:28:27.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.026+0000 7f8fc5e29640 1 -- 172.21.15.94:0/446988467 shutdown_connections 2024-04-26T02:28:27.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.026+0000 7f8fc5e29640 1 -- 172.21.15.94:0/446988467 >> 172.21.15.94:0/446988467 conn(0x7f8fc00d13c0 msgr2=0x7f8fc00d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:27.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.026+0000 7f8fc5e29640 1 -- 172.21.15.94:0/446988467 shutdown_connections 2024-04-26T02:28:27.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.026+0000 7f8fc5e29640 1 -- 172.21.15.94:0/446988467 wait complete. 2024-04-26T02:28:27.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.026+0000 7f8fc5e29640 1 Processor -- start 2024-04-26T02:28:27.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.026+0000 7f8fc5e29640 1 -- start start 2024-04-26T02:28:27.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fc5e29640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fc00cd2b0 0x7f8fc01f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fc5e29640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fc0157d40 0x7f8fc00ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fc5e29640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc00ce570 0x7f8fc00b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fc5e29640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc015cdb0 con 0x7f8fc00cd2b0 2024-04-26T02:28:27.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fc5e29640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc00738a0 con 0x7f8fc00ce570 2024-04-26T02:28:27.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fc5e29640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc00cfed0 con 0x7f8fc0157d40 2024-04-26T02:28:27.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fbf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fc00cd2b0 0x7f8fc01f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fbf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fc00cd2b0 0x7f8fc01f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50342/0 (socket says 172.21.15.94:50342) 2024-04-26T02:28:27.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fbf7fe640 1 -- 172.21.15.94:0/2868526464 learned_addr learned my addr 172.21.15.94:0/2868526464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:27.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fbeffd640 1 --2- 172.21.15.94:0/2868526464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fc0157d40 0x7f8fc00ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fbf7fe640 1 -- 172.21.15.94:0/2868526464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fc0157d40 msgr2=0x7f8fc00ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fbf7fe640 1 --2- 172.21.15.94:0/2868526464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fc0157d40 0x7f8fc00ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fbf7fe640 1 -- 172.21.15.94:0/2868526464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc00ce570 msgr2=0x7f8fc00b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fbf7fe640 1 --2- 172.21.15.94:0/2868526464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc00ce570 0x7f8fc00b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fbf7fe640 1 -- 172.21.15.94:0/2868526464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc00ce980 con 0x7f8fc00cd2b0 2024-04-26T02:28:27.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fbf7fe640 1 --2- 172.21.15.94:0/2868526464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fc00cd2b0 0x7f8fc01f6570 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0010860 tx=0x7f8fb0010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fa7fff640 1 -- 172.21.15.94:0/2868526464 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000f040 con 0x7f8fc00cd2b0 2024-04-26T02:28:27.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fa7fff640 1 -- 172.21.15.94:0/2868526464 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb000eb60 con 0x7f8fc00cd2b0 2024-04-26T02:28:27.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fa7fff640 1 -- 172.21.15.94:0/2868526464 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0010e50 con 0x7f8fc00cd2b0 2024-04-26T02:28:27.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fc5e29640 1 -- 172.21.15.94:0/2868526464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc00b1f50 con 0x7f8fc00cd2b0 2024-04-26T02:28:27.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fc5e29640 1 -- 172.21.15.94:0/2868526464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc00b2510 con 0x7f8fc00cd2b0 2024-04-26T02:28:27.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fc5e29640 1 -- 172.21.15.94:0/2868526464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8fc01589c0 con 0x7f8fc00cd2b0 2024-04-26T02:28:27.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fa7fff640 1 -- 172.21.15.94:0/2868526464 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8fb001c400 con 0x7f8fc00cd2b0 2024-04-26T02:28:27.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fa7fff640 1 --2- 172.21.15.94:0/2868526464 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8fac06f400 0x7f8fac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fa7fff640 1 -- 172.21.15.94:0/2868526464 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f8fb008f720 con 0x7f8fc00cd2b0 2024-04-26T02:28:27.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fbeffd640 1 --2- 172.21.15.94:0/2868526464 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8fac06f400 0x7f8fac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fa7fff640 1 --2- 172.21.15.94:0/2868526464 >> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] conn(0x7f8fac0758f0 0x7f8fac077d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fa7fff640 1 -- 172.21.15.94:0/2868526464 --> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8fac078450 con 0x7f8fac0758f0 2024-04-26T02:28:27.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fa7fff640 1 -- 172.21.15.94:0/2868526464 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8fb008fbb0 con 0x7f8fc00cd2b0 2024-04-26T02:28:27.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.030+0000 7f8fbffff640 1 --2- 172.21.15.94:0/2868526464 >> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] conn(0x7f8fac0758f0 0x7f8fac077d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.034+0000 7f8fbeffd640 1 --2- 172.21.15.94:0/2868526464 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8fac06f400 0x7f8fac071910 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40047c0 tx=0x7f8fb4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.034+0000 7f8fbffff640 1 --2- 172.21.15.94:0/2868526464 >> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] conn(0x7f8fac0758f0 0x7f8fac077d80 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.034+0000 7f8fa7fff640 1 -- 172.21.15.94:0/2868526464 <== osd.2 v2:172.21.15.94:6824/1121746691 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8fac078450 con 0x7f8fac0758f0 2024-04-26T02:28:27.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.046+0000 7f8fc5e29640 1 -- 172.21.15.94:0/2868526464 --> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8fc00cef90 con 0x7f8fac0758f0 2024-04-26T02:28:27.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.046+0000 7f8fa7fff640 1 -- 172.21.15.94:0/2868526464 <== osd.2 v2:172.21.15.94:6824/1121746691 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8fc00cef90 con 0x7f8fac0758f0 2024-04-26T02:28:27.051 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T02:28:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T02:28:27.052 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T02:28:27.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.046+0000 7f8fc5e29640 1 -- 172.21.15.94:0/2868526464 >> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] conn(0x7f8fac0758f0 msgr2=0x7f8fac077d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.046+0000 7f8fc5e29640 1 --2- 172.21.15.94:0/2868526464 >> [v2:172.21.15.94:6824/1121746691,v1:172.21.15.94:6825/1121746691] conn(0x7f8fac0758f0 0x7f8fac077d80 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.046+0000 7f8fc5e29640 1 -- 172.21.15.94:0/2868526464 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8fac06f400 msgr2=0x7f8fac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.046+0000 7f8fc5e29640 1 --2- 172.21.15.94:0/2868526464 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f8fac06f400 0x7f8fac071910 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40047c0 tx=0x7f8fb4008040 comp rx=0 tx=0).stop 2024-04-26T02:28:27.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.046+0000 7f8fc5e29640 1 -- 172.21.15.94:0/2868526464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fc00cd2b0 msgr2=0x7f8fc01f6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.046+0000 7f8fc5e29640 1 --2- 172.21.15.94:0/2868526464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fc00cd2b0 0x7f8fc01f6570 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0010860 tx=0x7f8fb0010d20 comp rx=0 tx=0).stop 2024-04-26T02:28:27.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.046+0000 7f8fbffff640 1 -- 172.21.15.94:0/2868526464 reap_dead start 2024-04-26T02:28:27.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.046+0000 7f8fc5e29640 1 -- 172.21.15.94:0/2868526464 shutdown_connections 2024-04-26T02:28:27.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.046+0000 7f8fc5e29640 1 -- 172.21.15.94:0/2868526464 >> 172.21.15.94:0/2868526464 conn(0x7f8fc00d13c0 msgr2=0x7f8fc006daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:27.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.046+0000 7f8fc5e29640 1 -- 172.21.15.94:0/2868526464 shutdown_connections 2024-04-26T02:28:27.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.046+0000 7f8fc5e29640 1 -- 172.21.15.94:0/2868526464 wait complete. 2024-04-26T02:28:27.070 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T02:28:27.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f017985c640 1 Processor -- start 2024-04-26T02:28:27.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f017985c640 1 -- start start 2024-04-26T02:28:27.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f017985c640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f01740b6470 0x7f01740b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f017985c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01741636a0 0x7f0174163ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f017985c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01740b6dc0 con 0x7f0174163ff0 2024-04-26T02:28:27.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f017985c640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f017415da10 con 0x7f01740b6470 2024-04-26T02:28:27.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f017985c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f01740b0c10 con 0x7f01741636a0 2024-04-26T02:28:27.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f0172ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f01740b6470 0x7f01740b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f0172ffd640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f01740b6470 0x7f01740b6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50344/0 (socket says 172.21.15.94:50344) 2024-04-26T02:28:27.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f0172ffd640 1 -- 172.21.15.94:0/2580309101 learned_addr learned my addr 172.21.15.94:0/2580309101 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:27.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f01727fc640 1 --2- 172.21.15.94:0/2580309101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01741636a0 0x7f0174163ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f0172ffd640 1 -- 172.21.15.94:0/2580309101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01741636a0 msgr2=0x7f0174163ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:27.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f0172ffd640 1 --2- 172.21.15.94:0/2580309101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01741636a0 0x7f0174163ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f0172ffd640 1 -- 172.21.15.94:0/2580309101 >> v1:172.21.15.138:6789/0 conn(0x7f0174163ff0 legacy=0x7f0174160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:27.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f0172ffd640 1 -- 172.21.15.94:0/2580309101 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01740b0ef0 con 0x7f01740b6470 2024-04-26T02:28:27.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.210+0000 7f01727fc640 1 --2- 172.21.15.94:0/2580309101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01741636a0 0x7f0174163ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:27.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.214+0000 7f0172ffd640 1 --2- 172.21.15.94:0/2580309101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01740b6470 0x7f01740b6880 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f016c009e50 tx=0x7f016c01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09973955284502d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.214+0000 7f0171ffb640 1 -- 172.21.15.94:0/2580309101 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c0ac020 con 0x7f01740b6470 2024-04-26T02:28:27.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.214+0000 7f0171ffb640 1 -- 172.21.15.94:0/2580309101 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f016c0a64a0 con 0x7f01740b6470 2024-04-26T02:28:27.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.214+0000 7f0171ffb640 1 -- 172.21.15.94:0/2580309101 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c0a6870 con 0x7f01740b6470 2024-04-26T02:28:27.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.218+0000 7f017985c640 1 -- 172.21.15.94:0/2580309101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01740b6470 msgr2=0x7f01740b6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.218+0000 7f017985c640 1 --2- 172.21.15.94:0/2580309101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01740b6470 0x7f01740b6880 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f016c009e50 tx=0x7f016c01c810 comp rx=0 tx=0).stop 2024-04-26T02:28:27.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.218+0000 7f017985c640 1 -- 172.21.15.94:0/2580309101 shutdown_connections 2024-04-26T02:28:27.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.218+0000 7f017985c640 1 --2- 172.21.15.94:0/2580309101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01741636a0 0x7f0174163ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.218+0000 7f017985c640 1 --2- 172.21.15.94:0/2580309101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01740b6470 0x7f01740b6880 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.218+0000 7f017985c640 1 -- 172.21.15.94:0/2580309101 >> 172.21.15.94:0/2580309101 conn(0x7f0174071f00 msgr2=0x7f0174073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:27.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.218+0000 7f017985c640 1 -- 172.21.15.94:0/2580309101 shutdown_connections 2024-04-26T02:28:27.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.218+0000 7f017985c640 1 -- 172.21.15.94:0/2580309101 wait complete. 2024-04-26T02:28:27.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.218+0000 7f017985c640 1 Processor -- start 2024-04-26T02:28:27.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.218+0000 7f017985c640 1 -- start start 2024-04-26T02:28:27.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f017985c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01740b6470 0x7f01741428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f017985c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01741636a0 0x7f017406d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f017985c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0174163ff0 0x7f017406dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f017985c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f01740b1e30 con 0x7f0174163ff0 2024-04-26T02:28:27.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f017985c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f01740b1cb0 con 0x7f01740b6470 2024-04-26T02:28:27.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f017985c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f01740aebf0 con 0x7f01741636a0 2024-04-26T02:28:27.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f01737fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0174163ff0 0x7f017406dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f01737fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0174163ff0 0x7f017406dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50352/0 (socket says 172.21.15.94:50352) 2024-04-26T02:28:27.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f01737fe640 1 -- 172.21.15.94:0/241958896 learned_addr learned my addr 172.21.15.94:0/241958896 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:27.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f01727fc640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01741636a0 0x7f017406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f0172ffd640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01740b6470 0x7f01741428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f01737fe640 1 -- 172.21.15.94:0/241958896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01741636a0 msgr2=0x7f017406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f01737fe640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01741636a0 0x7f017406d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f01737fe640 1 -- 172.21.15.94:0/241958896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01740b6470 msgr2=0x7f01741428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f01737fe640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01740b6470 0x7f01741428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f01737fe640 1 -- 172.21.15.94:0/241958896 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01741434b0 con 0x7f0174163ff0 2024-04-26T02:28:27.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f01737fe640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0174163ff0 0x7f017406dc50 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f016400b440 tx=0x7f016400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f016bfff640 1 -- 172.21.15.94:0/241958896 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164015020 con 0x7f0174163ff0 2024-04-26T02:28:27.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f016bfff640 1 -- 172.21.15.94:0/241958896 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0164011870 con 0x7f0174163ff0 2024-04-26T02:28:27.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f017985c640 1 -- 172.21.15.94:0/241958896 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0174143790 con 0x7f0174163ff0 2024-04-26T02:28:27.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f016bfff640 1 -- 172.21.15.94:0/241958896 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164011c60 con 0x7f0174163ff0 2024-04-26T02:28:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f017985c640 1 -- 172.21.15.94:0/241958896 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0174143d50 con 0x7f0174163ff0 2024-04-26T02:28:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f017985c640 1 -- 172.21.15.94:0/241958896 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017406a730 con 0x7f0174163ff0 2024-04-26T02:28:27.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.222+0000 7f016bfff640 1 -- 172.21.15.94:0/241958896 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0164018020 con 0x7f0174163ff0 2024-04-26T02:28:27.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.226+0000 7f016bfff640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f015406f400 0x7f0154071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.226+0000 7f0172ffd640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f015406f400 0x7f0154071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.226+0000 7f016bfff640 1 -- 172.21.15.94:0/241958896 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f016408f0a0 con 0x7f0174163ff0 2024-04-26T02:28:27.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.226+0000 7f0172ffd640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f015406f400 0x7f0154071910 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f016c0ad000 tx=0x7f016c0b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.230+0000 7f016bfff640 1 -- 172.21.15.94:0/241958896 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f016401f030 con 0x7f0174163ff0 2024-04-26T02:28:27.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f017985c640 1 -- 172.21.15.94:0/241958896 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f01741611f0 con 0x7f015406f400 2024-04-26T02:28:27.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f016bfff640 1 -- 172.21.15.94:0/241958896 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f01741611f0 con 0x7f015406f400 2024-04-26T02:28:27.386 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T02:28:27.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f017985c640 1 -- 172.21.15.94:0/241958896 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f015406f400 msgr2=0x7f0154071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f017985c640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f015406f400 0x7f0154071910 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f016c0ad000 tx=0x7f016c0b3040 comp rx=0 tx=0).stop 2024-04-26T02:28:27.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f017985c640 1 -- 172.21.15.94:0/241958896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0174163ff0 msgr2=0x7f017406dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f017985c640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0174163ff0 0x7f017406dc50 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f016400b440 tx=0x7f016400b900 comp rx=0 tx=0).stop 2024-04-26T02:28:27.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f017985c640 1 -- 172.21.15.94:0/241958896 shutdown_connections 2024-04-26T02:28:27.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f017985c640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0174163ff0 0x7f017406dc50 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f017985c640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f015406f400 0x7f0154071910 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f017985c640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f01741636a0 0x7f017406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f017985c640 1 --2- 172.21.15.94:0/241958896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01740b6470 0x7f01741428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f017985c640 1 -- 172.21.15.94:0/241958896 >> 172.21.15.94:0/241958896 conn(0x7f0174071f00 msgr2=0x7f0174160ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:27.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f017985c640 1 -- 172.21.15.94:0/241958896 shutdown_connections 2024-04-26T02:28:27.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.382+0000 7f017985c640 1 -- 172.21.15.94:0/241958896 wait complete. 2024-04-26T02:28:27.407 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T02:28:27.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T02:28:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.558+0000 7f168ec66640 1 Processor -- start 2024-04-26T02:28:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.558+0000 7f168ec66640 1 -- start start 2024-04-26T02:28:27.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f168ec66640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f168806c600 0x7f168806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f168ec66640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f16880cd500 0x7f16880cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f168ec66640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f168806d230 con 0x7f168806fcf0 2024-04-26T02:28:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f168ec66640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f168806ec30 con 0x7f168806c600 2024-04-26T02:28:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f168ec66640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f168806d510 con 0x7f16880cd500 2024-04-26T02:28:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f168ca02640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f168806c600 0x7f168806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f168ca02640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f168806c600 0x7f168806ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50356/0 (socket says 172.21.15.94:50356) 2024-04-26T02:28:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f168ca02640 1 -- 172.21.15.94:0/2721622099 learned_addr learned my addr 172.21.15.94:0/2721622099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f1687fff640 1 --1- 172.21.15.94:0/2721622099 >> v1:172.21.15.94:6789/0 conn(0x7f168806fcf0 0x7f1688070100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:56132/0 (socket says 172.21.15.94:56132) 2024-04-26T02:28:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f168ca02640 1 -- 172.21.15.94:0/2721622099 >> v2:172.21.15.138:3300/0 conn(0x7f16880cd500 msgr2=0x7f16880cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:27.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f168ca02640 1 --2- 172.21.15.94:0/2721622099 >> v2:172.21.15.138:3300/0 conn(0x7f16880cd500 0x7f16880cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f168ca02640 1 -- 172.21.15.94:0/2721622099 >> v1:172.21.15.94:6789/0 conn(0x7f168806fcf0 legacy=0x7f1688070100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f168ca02640 1 -- 172.21.15.94:0/2721622099 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168806d7f0 con 0x7f168806c600 2024-04-26T02:28:27.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f168ca02640 1 --2- 172.21.15.94:0/2721622099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f168806c600 0x7f168806ca10 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f168806e320 tx=0x7f167801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39298292c3d1b56d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f16877fe640 1 -- 172.21.15.94:0/2721622099 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16780aa650 con 0x7f168806c600 2024-04-26T02:28:27.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f16877fe640 1 -- 172.21.15.94:0/2721622099 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16780af070 con 0x7f168806c600 2024-04-26T02:28:27.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.562+0000 7f16877fe640 1 -- 172.21.15.94:0/2721622099 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16780aaa10 con 0x7f168806c600 2024-04-26T02:28:27.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.566+0000 7f168ec66640 1 -- 172.21.15.94:0/2721622099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f168806c600 msgr2=0x7f168806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.566+0000 7f168ec66640 1 --2- 172.21.15.94:0/2721622099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f168806c600 0x7f168806ca10 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f168806e320 tx=0x7f167801c4f0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.566+0000 7f168ec66640 1 -- 172.21.15.94:0/2721622099 shutdown_connections 2024-04-26T02:28:27.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.566+0000 7f168ec66640 1 --2- 172.21.15.94:0/2721622099 >> v2:172.21.15.138:3300/0 conn(0x7f16880cd500 0x7f16880cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.566+0000 7f168ec66640 1 --2- 172.21.15.94:0/2721622099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f168806c600 0x7f168806ca10 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.566+0000 7f168ec66640 1 -- 172.21.15.94:0/2721622099 >> 172.21.15.94:0/2721622099 conn(0x7f16880d13c0 msgr2=0x7f16880d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:27.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.566+0000 7f168ec66640 1 -- 172.21.15.94:0/2721622099 shutdown_connections 2024-04-26T02:28:27.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.566+0000 7f168ec66640 1 -- 172.21.15.94:0/2721622099 wait complete. 2024-04-26T02:28:27.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.566+0000 7f168ec66640 1 Processor -- start 2024-04-26T02:28:27.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.566+0000 7f168ec66640 1 -- start start 2024-04-26T02:28:27.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ec66640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f168806c600 0x7f16880ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ec66640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f168806fcf0 0x7f168809e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ec66640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16880cd500 0x7f168809e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ec66640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16880ccd80 con 0x7f168806c600 2024-04-26T02:28:27.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ec66640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16880ccc00 con 0x7f168806fcf0 2024-04-26T02:28:27.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ec66640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f168816a920 con 0x7f16880cd500 2024-04-26T02:28:27.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ca02640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f168806c600 0x7f16880ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ca02640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f168806c600 0x7f16880ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50362/0 (socket says 172.21.15.94:50362) 2024-04-26T02:28:27.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ca02640 1 -- 172.21.15.94:0/4119587611 learned_addr learned my addr 172.21.15.94:0/4119587611 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:27.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f1687fff640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f168806fcf0 0x7f168809e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168d203640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16880cd500 0x7f168809e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ca02640 1 -- 172.21.15.94:0/4119587611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16880cd500 msgr2=0x7f168809e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ca02640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16880cd500 0x7f168809e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ca02640 1 -- 172.21.15.94:0/4119587611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f168806fcf0 msgr2=0x7f168809e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ca02640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f168806fcf0 0x7f168809e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ca02640 1 -- 172.21.15.94:0/4119587611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1688158d20 con 0x7f168806c600 2024-04-26T02:28:27.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168d203640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16880cd500 0x7f168809e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:27.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ca02640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f168806c600 0x7f16880ca5e0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f167801a120 tx=0x7f167801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f16857fa640 1 -- 172.21.15.94:0/4119587611 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16780b3800 con 0x7f168806c600 2024-04-26T02:28:27.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f16857fa640 1 -- 172.21.15.94:0/4119587611 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16780a6040 con 0x7f168806c600 2024-04-26T02:28:27.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ec66640 1 -- 172.21.15.94:0/4119587611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1688159000 con 0x7f168806c600 2024-04-26T02:28:27.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f16857fa640 1 -- 172.21.15.94:0/4119587611 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16780b3c00 con 0x7f168806c600 2024-04-26T02:28:27.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ec66640 1 -- 172.21.15.94:0/4119587611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16881f9dd0 con 0x7f168806c600 2024-04-26T02:28:27.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168ec66640 1 -- 172.21.15.94:0/4119587611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1688158ad0 con 0x7f168806c600 2024-04-26T02:28:27.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f16857fa640 1 -- 172.21.15.94:0/4119587611 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f16780bd3f0 con 0x7f168806c600 2024-04-26T02:28:27.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f16857fa640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f167406f400 0x7f1674071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f16857fa640 1 -- 172.21.15.94:0/4119587611 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f16781296e0 con 0x7f168806c600 2024-04-26T02:28:27.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f1687fff640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f167406f400 0x7f1674071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f16857fa640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] conn(0x7f16740758f0 0x7f1674077d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f16857fa640 1 -- 172.21.15.94:0/4119587611 --> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16780afe70 con 0x7f16740758f0 2024-04-26T02:28:27.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f16857fa640 1 -- 172.21.15.94:0/4119587611 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1678129b70 con 0x7f168806c600 2024-04-26T02:28:27.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.570+0000 7f168d203640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] conn(0x7f16740758f0 0x7f1674077d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.574+0000 7f1687fff640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f167406f400 0x7f1674071910 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f16880c7cb0 tx=0x7f167c007cf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.574+0000 7f168d203640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] conn(0x7f16740758f0 0x7f1674077d80 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.574+0000 7f16857fa640 1 -- 172.21.15.94:0/4119587611 <== osd.3 v2:172.21.15.94:6816/1777163555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16780afe70 con 0x7f16740758f0 2024-04-26T02:28:27.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.586+0000 7f168ec66640 1 -- 172.21.15.94:0/4119587611 --> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f16880d2260 con 0x7f16740758f0 2024-04-26T02:28:27.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.586+0000 7f16857fa640 1 -- 172.21.15.94:0/4119587611 <== osd.3 v2:172.21.15.94:6816/1777163555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f16880d2260 con 0x7f16740758f0 2024-04-26T02:28:27.592 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T02:28:27.592 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T02:28:27.592 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T02:28:27.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.586+0000 7f168ec66640 1 -- 172.21.15.94:0/4119587611 >> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] conn(0x7f16740758f0 msgr2=0x7f1674077d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.586+0000 7f168ec66640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.94:6816/1777163555,v1:172.21.15.94:6817/1777163555] conn(0x7f16740758f0 0x7f1674077d80 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.586+0000 7f168ec66640 1 -- 172.21.15.94:0/4119587611 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f167406f400 msgr2=0x7f1674071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.586+0000 7f168ec66640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f167406f400 0x7f1674071910 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f16880c7cb0 tx=0x7f167c007cf0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.586+0000 7f168ec66640 1 -- 172.21.15.94:0/4119587611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f168806c600 msgr2=0x7f16880ca5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.586+0000 7f168ec66640 1 --2- 172.21.15.94:0/4119587611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f168806c600 0x7f16880ca5e0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f167801a120 tx=0x7f167801ad30 comp rx=0 tx=0).stop 2024-04-26T02:28:27.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.586+0000 7f168d203640 1 -- 172.21.15.94:0/4119587611 reap_dead start 2024-04-26T02:28:27.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.586+0000 7f168ec66640 1 -- 172.21.15.94:0/4119587611 shutdown_connections 2024-04-26T02:28:27.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.586+0000 7f168ec66640 1 -- 172.21.15.94:0/4119587611 >> 172.21.15.94:0/4119587611 conn(0x7f16880d13c0 msgr2=0x7f16880cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:27.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.586+0000 7f168ec66640 1 -- 172.21.15.94:0/4119587611 shutdown_connections 2024-04-26T02:28:27.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.586+0000 7f168ec66640 1 -- 172.21.15.94:0/4119587611 wait complete. 2024-04-26T02:28:27.612 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T02:28:27.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9f782640 1 Processor -- start 2024-04-26T02:28:27.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9f782640 1 -- start start 2024-04-26T02:28:27.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9f782640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fac980d3450 0x7fac980d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9f782640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac980d3da0 0x7fac980b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9f782640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fac980b7510 0x7fac980b7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9f782640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fac980b7e60 con 0x7fac980d3450 2024-04-26T02:28:27.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9f782640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fac98164a90 con 0x7fac980b7510 2024-04-26T02:28:27.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9f782640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fac980d51b0 con 0x7fac980d3da0 2024-04-26T02:28:27.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9d51e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fac980d3450 0x7fac980d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9d51e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fac980d3450 0x7fac980d3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50372/0 (socket says 172.21.15.94:50372) 2024-04-26T02:28:27.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9d51e640 1 -- 172.21.15.94:0/1222838153 learned_addr learned my addr 172.21.15.94:0/1222838153 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:27.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9dd1f640 1 --2- 172.21.15.94:0/1222838153 >> v2:172.21.15.138:3300/0 conn(0x7fac980b7510 0x7fac980b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9cd1d640 1 --2- 172.21.15.94:0/1222838153 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac980d3da0 0x7fac980b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9d51e640 1 -- 172.21.15.94:0/1222838153 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac980d3da0 msgr2=0x7fac980b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:27.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9d51e640 1 --2- 172.21.15.94:0/1222838153 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac980d3da0 0x7fac980b6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9d51e640 1 -- 172.21.15.94:0/1222838153 >> v2:172.21.15.138:3300/0 conn(0x7fac980b7510 msgr2=0x7fac980b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:27.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9d51e640 1 --2- 172.21.15.94:0/1222838153 >> v2:172.21.15.138:3300/0 conn(0x7fac980b7510 0x7fac980b7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.754+0000 7fac9d51e640 1 -- 172.21.15.94:0/1222838153 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac98164510 con 0x7fac980d3450 2024-04-26T02:28:27.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.758+0000 7fac9d51e640 1 --2- 172.21.15.94:0/1222838153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac980d3450 0x7fac980d3860 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fac9808d540 tx=0x7fac9401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40eea6aee47edf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.758+0000 7fac8ffff640 1 -- 172.21.15.94:0/1222838153 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940ab620 con 0x7fac980d3450 2024-04-26T02:28:27.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.758+0000 7fac8ffff640 1 -- 172.21.15.94:0/1222838153 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac940b0070 con 0x7fac980d3450 2024-04-26T02:28:27.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.758+0000 7fac8ffff640 1 -- 172.21.15.94:0/1222838153 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940ab9e0 con 0x7fac980d3450 2024-04-26T02:28:27.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.762+0000 7fac9f782640 1 -- 172.21.15.94:0/1222838153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac980d3450 msgr2=0x7fac980d3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.762+0000 7fac9f782640 1 --2- 172.21.15.94:0/1222838153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac980d3450 0x7fac980d3860 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fac9808d540 tx=0x7fac9401c460 comp rx=0 tx=0).stop 2024-04-26T02:28:27.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.762+0000 7fac9f782640 1 -- 172.21.15.94:0/1222838153 shutdown_connections 2024-04-26T02:28:27.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.762+0000 7fac9f782640 1 --2- 172.21.15.94:0/1222838153 >> v2:172.21.15.138:3300/0 conn(0x7fac980b7510 0x7fac980b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.762+0000 7fac9f782640 1 --2- 172.21.15.94:0/1222838153 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac980d3da0 0x7fac980b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.762+0000 7fac9f782640 1 --2- 172.21.15.94:0/1222838153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac980d3450 0x7fac980d3860 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.762+0000 7fac9f782640 1 -- 172.21.15.94:0/1222838153 >> 172.21.15.94:0/1222838153 conn(0x7fac9809e8d0 msgr2=0x7fac980b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:27.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.762+0000 7fac9f782640 1 -- 172.21.15.94:0/1222838153 shutdown_connections 2024-04-26T02:28:27.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.762+0000 7fac9f782640 1 -- 172.21.15.94:0/1222838153 wait complete. 2024-04-26T02:28:27.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.762+0000 7fac9f782640 1 Processor -- start 2024-04-26T02:28:27.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.762+0000 7fac9f782640 1 -- start start 2024-04-26T02:28:27.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9f782640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac980b7510 0x7fac9806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9f782640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac980d3450 0x7fac9814e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9f782640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac980d3da0 0x7fac9815fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9f782640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fac980d41b0 con 0x7fac980d3da0 2024-04-26T02:28:27.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9f782640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac980cfc10 con 0x7fac980d3450 2024-04-26T02:28:27.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9f782640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fac980b8cd0 con 0x7fac980b7510 2024-04-26T02:28:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9dd1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac980d3da0 0x7fac9815fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9dd1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac980d3da0 0x7fac9815fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50386/0 (socket says 172.21.15.94:50386) 2024-04-26T02:28:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9dd1f640 1 -- 172.21.15.94:0/2426377156 learned_addr learned my addr 172.21.15.94:0/2426377156 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9cd1d640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac980d3450 0x7fac9814e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9d51e640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac980b7510 0x7fac9806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9dd1f640 1 -- 172.21.15.94:0/2426377156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac980b7510 msgr2=0x7fac9806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9dd1f640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac980b7510 0x7fac9806c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9dd1f640 1 -- 172.21.15.94:0/2426377156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac980d3450 msgr2=0x7fac9814e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9dd1f640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac980d3450 0x7fac9814e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9dd1f640 1 -- 172.21.15.94:0/2426377156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9814ee90 con 0x7fac980d3da0 2024-04-26T02:28:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9dd1f640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac980d3da0 0x7fac9815fef0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fac90007d10 tx=0x7fac9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac8dffb640 1 -- 172.21.15.94:0/2426377156 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90014420 con 0x7fac980d3da0 2024-04-26T02:28:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac8dffb640 1 -- 172.21.15.94:0/2426377156 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac90019070 con 0x7fac980d3da0 2024-04-26T02:28:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9f782640 1 -- 172.21.15.94:0/2426377156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9806d0c0 con 0x7fac980d3da0 2024-04-26T02:28:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac8dffb640 1 -- 172.21.15.94:0/2426377156 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90014800 con 0x7fac980d3da0 2024-04-26T02:28:27.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9f782640 1 -- 172.21.15.94:0/2426377156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9806d680 con 0x7fac980d3da0 2024-04-26T02:28:27.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9f782640 1 -- 172.21.15.94:0/2426377156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac98076830 con 0x7fac980d3da0 2024-04-26T02:28:27.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac8dffb640 1 -- 172.21.15.94:0/2426377156 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fac900149a0 con 0x7fac980d3da0 2024-04-26T02:28:27.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac8dffb640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fac7c06f400 0x7fac7c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:27.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac9d51e640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fac7c06f400 0x7fac7c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:27.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.766+0000 7fac8dffb640 1 -- 172.21.15.94:0/2426377156 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fac9008f8c0 con 0x7fac980d3da0 2024-04-26T02:28:27.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.770+0000 7fac9d51e640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fac7c06f400 0x7fac7c071910 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fac940af000 tx=0x7fac940a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:27.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.774+0000 7fac8dffb640 1 -- 172.21.15.94:0/2426377156 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fac9005c920 con 0x7fac980d3da0 2024-04-26T02:28:27.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.930+0000 7fac9f782640 1 -- 172.21.15.94:0/2426377156 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fac980d2c40 con 0x7fac7c06f400 2024-04-26T02:28:27.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.930+0000 7fac8dffb640 1 -- 172.21.15.94:0/2426377156 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fac980d2c40 con 0x7fac7c06f400 2024-04-26T02:28:27.936 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T02:28:27.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.934+0000 7fac9f782640 1 -- 172.21.15.94:0/2426377156 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fac7c06f400 msgr2=0x7fac7c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.934+0000 7fac9f782640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fac7c06f400 0x7fac7c071910 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fac940af000 tx=0x7fac940a7000 comp rx=0 tx=0).stop 2024-04-26T02:28:27.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.934+0000 7fac9f782640 1 -- 172.21.15.94:0/2426377156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac980d3da0 msgr2=0x7fac9815fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:27.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.934+0000 7fac9f782640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac980d3da0 0x7fac9815fef0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fac90007d10 tx=0x7fac9000e4b0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.934+0000 7fac9f782640 1 -- 172.21.15.94:0/2426377156 shutdown_connections 2024-04-26T02:28:27.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.934+0000 7fac9f782640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac980d3da0 0x7fac9815fef0 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.934+0000 7fac9f782640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac980d3450 0x7fac9814e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.934+0000 7fac9f782640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fac7c06f400 0x7fac7c071910 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.934+0000 7fac9f782640 1 --2- 172.21.15.94:0/2426377156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac980b7510 0x7fac9806c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:27.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.934+0000 7fac9f782640 1 -- 172.21.15.94:0/2426377156 >> 172.21.15.94:0/2426377156 conn(0x7fac9809e8d0 msgr2=0x7fac980b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:27.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.934+0000 7fac9f782640 1 -- 172.21.15.94:0/2426377156 shutdown_connections 2024-04-26T02:28:27.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:27.934+0000 7fac9f782640 1 -- 172.21.15.94:0/2426377156 wait complete. 2024-04-26T02:28:27.958 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T02:28:27.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T02:28:28.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4caafd3640 1 Processor -- start 2024-04-26T02:28:28.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4caafd3640 1 -- start start 2024-04-26T02:28:28.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4caafd3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ca4073f10 0x7f4ca40d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4caafd3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca40d02f0 con 0x7f4ca40b5260 2024-04-26T02:28:28.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4caafd3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca40d05d0 con 0x7f4ca40d19e0 2024-04-26T02:28:28.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4caafd3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca40d0f00 con 0x7f4ca4073f10 2024-04-26T02:28:28.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca9570640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f4ca40b5260 0x7f4ca40b5670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51272/0 (socket says 172.21.15.94:51272) 2024-04-26T02:28:28.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca9570640 1 -- 172.21.15.94:0/4124267762 learned_addr learned my addr 172.21.15.94:0/4124267762 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca3fff640 1 --2- 172.21.15.94:0/4124267762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ca4073f10 0x7f4ca40d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2202255925 0 0) 0x7f4ca40d02f0 con 0x7f4ca40b5260 2024-04-26T02:28:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c8c003530 con 0x7f4ca40b5260 2024-04-26T02:28:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4160930680 0 0) 0x7f4c8c003530 con 0x7f4ca40b5260 2024-04-26T02:28:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ca4073f10 msgr2=0x7f4ca40d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 --2- 172.21.15.94:0/4124267762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ca4073f10 0x7f4ca40d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 >> v1:172.21.15.138:6789/0 conn(0x7f4ca40d19e0 legacy=0x7f4ca4073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca40d08b0 con 0x7f4ca40b5260 2024-04-26T02:28:28.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f4c9c001a00 con 0x7f4ca40b5260 2024-04-26T02:28:28.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 >> v1:172.21.15.94:6789/0 conn(0x7f4ca40b5260 legacy=0x7f4ca40b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 --2- 172.21.15.94:0/4124267762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c8c003a60 0x7f4c8c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 --2- 172.21.15.94:0/4124267762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8c004810 0x7f4c8c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 --2- 172.21.15.94:0/4124267762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c005160 0x7f4c8c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca40d0f00 con 0x7f4c8c003a60 2024-04-26T02:28:28.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c00bfc0 con 0x7f4c8c005160 2024-04-26T02:28:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4c8c00c190 con 0x7f4c8c004810 2024-04-26T02:28:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca8d6f640 1 --2- 172.21.15.94:0/4124267762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c8c003a60 0x7f4c8c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca9570640 1 --2- 172.21.15.94:0/4124267762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c005160 0x7f4c8c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca3fff640 1 --2- 172.21.15.94:0/4124267762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8c004810 0x7f4c8c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca8d6f640 1 -- 172.21.15.94:0/4124267762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8c004810 msgr2=0x7f4c8c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca8d6f640 1 --2- 172.21.15.94:0/4124267762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8c004810 0x7f4c8c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca8d6f640 1 -- 172.21.15.94:0/4124267762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c005160 msgr2=0x7f4c8c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca8d6f640 1 --2- 172.21.15.94:0/4124267762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c005160 0x7f4c8c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.098+0000 7f4ca8d6f640 1 -- 172.21.15.94:0/4124267762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c003530 con 0x7f4c8c003a60 2024-04-26T02:28:28.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.102+0000 7f4ca8d6f640 1 --2- 172.21.15.94:0/4124267762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c8c003a60 0x7f4c8c003ed0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f4c98001780 tx=0x7f4c9801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c9e50efe96ae83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.102+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c980013a0 con 0x7f4c8c003a60 2024-04-26T02:28:28.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.102+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c980052f0 con 0x7f4c8c003a60 2024-04-26T02:28:28.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.102+0000 7f4ca37fe640 1 -- 172.21.15.94:0/4124267762 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98003840 con 0x7f4c8c003a60 2024-04-26T02:28:28.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.102+0000 7f4caafd3640 1 -- 172.21.15.94:0/4124267762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c8c003a60 msgr2=0x7f4c8c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.102+0000 7f4caafd3640 1 --2- 172.21.15.94:0/4124267762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c8c003a60 0x7f4c8c003ed0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f4c98001780 tx=0x7f4c9801c3b0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.102+0000 7f4ca9570640 1 -- 172.21.15.94:0/4124267762 reap_dead start 2024-04-26T02:28:28.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.102+0000 7f4caafd3640 1 -- 172.21.15.94:0/4124267762 shutdown_connections 2024-04-26T02:28:28.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.102+0000 7f4caafd3640 1 -- 172.21.15.94:0/4124267762 >> 172.21.15.94:0/4124267762 conn(0x7f4ca40cedd0 msgr2=0x7f4ca40cf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:28.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.102+0000 7f4caafd3640 1 -- 172.21.15.94:0/4124267762 shutdown_connections 2024-04-26T02:28:28.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.102+0000 7f4caafd3640 1 -- 172.21.15.94:0/4124267762 wait complete. 2024-04-26T02:28:28.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4caafd3640 1 Processor -- start 2024-04-26T02:28:28.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4caafd3640 1 -- start start 2024-04-26T02:28:28.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4caafd3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8c003a60 0x7f4ca414adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4caafd3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca41500c0 0x7f4ca4157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4caafd3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca4157c80 0x7f4ca4153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4caafd3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca4073f10 con 0x7f4ca4157c80 2024-04-26T02:28:28.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4caafd3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca4070590 con 0x7f4ca41500c0 2024-04-26T02:28:28.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4caafd3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca4074090 con 0x7f4c8c003a60 2024-04-26T02:28:28.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca9570640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca4157c80 0x7f4ca4153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca9570640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca4157c80 0x7f4ca4153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40960/0 (socket says 172.21.15.94:40960) 2024-04-26T02:28:28.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca9570640 1 -- 172.21.15.94:0/2222459746 learned_addr learned my addr 172.21.15.94:0/2222459746 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:28.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca3fff640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca41500c0 0x7f4ca4157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca8d6f640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8c003a60 0x7f4ca414adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca9570640 1 -- 172.21.15.94:0/2222459746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8c003a60 msgr2=0x7f4ca414adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca9570640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8c003a60 0x7f4ca414adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca9570640 1 -- 172.21.15.94:0/2222459746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca41500c0 msgr2=0x7f4ca4157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca9570640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca41500c0 0x7f4ca4157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca9570640 1 -- 172.21.15.94:0/2222459746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca4152ad0 con 0x7f4ca4157c80 2024-04-26T02:28:28.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca8d6f640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8c003a60 0x7f4ca414adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:28.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca3fff640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca41500c0 0x7f4ca4157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:28.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca9570640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca4157c80 0x7f4ca4153b70 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4072a10 tx=0x7f4c9c005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca17fa640 1 -- 172.21.15.94:0/2222459746 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c00eb00 con 0x7f4ca4157c80 2024-04-26T02:28:28.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca17fa640 1 -- 172.21.15.94:0/2222459746 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c9c018070 con 0x7f4ca4157c80 2024-04-26T02:28:28.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4caafd3640 1 -- 172.21.15.94:0/2222459746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca4152db0 con 0x7f4ca4157c80 2024-04-26T02:28:28.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4ca17fa640 1 -- 172.21.15.94:0/2222459746 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c013420 con 0x7f4ca4157c80 2024-04-26T02:28:28.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4caafd3640 1 -- 172.21.15.94:0/2222459746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca4153370 con 0x7f4ca4157c80 2024-04-26T02:28:28.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.106+0000 7f4caafd3640 1 -- 172.21.15.94:0/2222459746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ca40cf480 con 0x7f4ca4157c80 2024-04-26T02:28:28.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.110+0000 7f4ca17fa640 1 -- 172.21.15.94:0/2222459746 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4c9c0135c0 con 0x7f4ca4157c80 2024-04-26T02:28:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.110+0000 7f4ca17fa640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4c9006f400 0x7f4c90071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.110+0000 7f4ca17fa640 1 -- 172.21.15.94:0/2222459746 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f4c9c08ff60 con 0x7f4ca4157c80 2024-04-26T02:28:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.110+0000 7f4ca8d6f640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4c9006f400 0x7f4c90071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.110+0000 7f4ca17fa640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] conn(0x7f4c900758f0 0x7f4c90077d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.110+0000 7f4ca17fa640 1 -- 172.21.15.94:0/2222459746 --> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c90078450 con 0x7f4c900758f0 2024-04-26T02:28:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.110+0000 7f4ca17fa640 1 -- 172.21.15.94:0/2222459746 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4c9c090310 con 0x7f4ca4157c80 2024-04-26T02:28:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.110+0000 7f4ca8d6f640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4c9006f400 0x7f4c90071910 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f4c98001750 tx=0x7f4c980011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.110+0000 7f4ca3fff640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] conn(0x7f4c900758f0 0x7f4c90077d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.110+0000 7f4ca3fff640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] conn(0x7f4c900758f0 0x7f4c90077d80 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.122+0000 7f4ca17fa640 1 -- 172.21.15.94:0/2222459746 <== osd.4 v2:172.21.15.138:6816/4237363180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c90078450 con 0x7f4c900758f0 2024-04-26T02:28:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.134+0000 7f4caafd3640 1 -- 172.21.15.94:0/2222459746 --> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4ca4158220 con 0x7f4c900758f0 2024-04-26T02:28:28.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.138+0000 7f4ca17fa640 1 -- 172.21.15.94:0/2222459746 <== osd.4 v2:172.21.15.138:6816/4237363180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4ca4158220 con 0x7f4c900758f0 2024-04-26T02:28:28.142 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T02:28:28.142 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T02:28:28.142 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T02:28:28.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.138+0000 7f4caafd3640 1 -- 172.21.15.94:0/2222459746 >> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] conn(0x7f4c900758f0 msgr2=0x7f4c90077d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.138+0000 7f4caafd3640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.138:6816/4237363180,v1:172.21.15.138:6817/4237363180] conn(0x7f4c900758f0 0x7f4c90077d80 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.138+0000 7f4caafd3640 1 -- 172.21.15.94:0/2222459746 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4c9006f400 msgr2=0x7f4c90071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.138+0000 7f4caafd3640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4c9006f400 0x7f4c90071910 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f4c98001750 tx=0x7f4c980011a0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.138+0000 7f4caafd3640 1 -- 172.21.15.94:0/2222459746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca4157c80 msgr2=0x7f4ca4153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.138+0000 7f4caafd3640 1 --2- 172.21.15.94:0/2222459746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca4157c80 0x7f4ca4153b70 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4072a10 tx=0x7f4c9c005f60 comp rx=0 tx=0).stop 2024-04-26T02:28:28.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.138+0000 7f4ca9570640 1 -- 172.21.15.94:0/2222459746 reap_dead start 2024-04-26T02:28:28.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.138+0000 7f4caafd3640 1 -- 172.21.15.94:0/2222459746 shutdown_connections 2024-04-26T02:28:28.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.138+0000 7f4caafd3640 1 -- 172.21.15.94:0/2222459746 >> 172.21.15.94:0/2222459746 conn(0x7f4ca40cedd0 msgr2=0x7f4ca4071570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:28.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.138+0000 7f4caafd3640 1 -- 172.21.15.94:0/2222459746 shutdown_connections 2024-04-26T02:28:28.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.138+0000 7f4caafd3640 1 -- 172.21.15.94:0/2222459746 wait complete. 2024-04-26T02:28:28.162 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T02:28:28.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.294+0000 7faa98aa3640 1 Processor -- start 2024-04-26T02:28:28.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.294+0000 7faa98aa3640 1 -- start start 2024-04-26T02:28:28.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.294+0000 7faa98aa3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faa940ce010 0x7faa941681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.294+0000 7faa98aa3640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa94168b30 con 0x7faa94164170 2024-04-26T02:28:28.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.294+0000 7faa98aa3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa940ce9f0 con 0x7faa94168720 2024-04-26T02:28:28.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.294+0000 7faa98aa3640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7faa940d10a0 con 0x7faa940ce010 2024-04-26T02:28:28.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.294+0000 7faa91d9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faa940ce010 0x7faa941681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.294+0000 7faa91d9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7faa940ce010 0x7faa941681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40968/0 (socket says 172.21.15.94:40968) 2024-04-26T02:28:28.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.294+0000 7faa9259c640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7faa94164170 0x7faa940b6e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51280/0 (socket says 172.21.15.94:51280) 2024-04-26T02:28:28.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.294+0000 7faa91d9b640 1 -- 172.21.15.94:0/1474681466 learned_addr learned my addr 172.21.15.94:0/1474681466 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.294+0000 7faa91d9b640 1 -- 172.21.15.94:0/1474681466 >> v1:172.21.15.138:6789/0 conn(0x7faa94168720 legacy=0x7faa94073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.294+0000 7faa91d9b640 1 -- 172.21.15.94:0/1474681466 >> v1:172.21.15.94:6789/0 conn(0x7faa94164170 legacy=0x7faa940b6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.294+0000 7faa91d9b640 1 -- 172.21.15.94:0/1474681466 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa940b5390 con 0x7faa940ce010 2024-04-26T02:28:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.298+0000 7faa91d9b640 1 --2- 172.21.15.94:0/1474681466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa940ce010 0x7faa941681e0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7faa84009e50 tx=0x7faa8401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51c36e3aa420568 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.298+0000 7faa9159a640 1 -- 172.21.15.94:0/1474681466 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa840a56d0 con 0x7faa940ce010 2024-04-26T02:28:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.298+0000 7faa9159a640 1 -- 172.21.15.94:0/1474681466 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa840a5870 con 0x7faa940ce010 2024-04-26T02:28:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.298+0000 7faa9159a640 1 -- 172.21.15.94:0/1474681466 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa840a5c40 con 0x7faa940ce010 2024-04-26T02:28:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.298+0000 7faa98aa3640 1 -- 172.21.15.94:0/1474681466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa940ce010 msgr2=0x7faa941681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.298+0000 7faa98aa3640 1 --2- 172.21.15.94:0/1474681466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa940ce010 0x7faa941681e0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7faa84009e50 tx=0x7faa8401c960 comp rx=0 tx=0).stop 2024-04-26T02:28:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.298+0000 7faa98aa3640 1 -- 172.21.15.94:0/1474681466 shutdown_connections 2024-04-26T02:28:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.298+0000 7faa98aa3640 1 --2- 172.21.15.94:0/1474681466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa940ce010 0x7faa941681e0 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.298+0000 7faa98aa3640 1 -- 172.21.15.94:0/1474681466 >> 172.21.15.94:0/1474681466 conn(0x7faa940d2690 msgr2=0x7faa940d2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.298+0000 7faa98aa3640 1 -- 172.21.15.94:0/1474681466 shutdown_connections 2024-04-26T02:28:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.298+0000 7faa98aa3640 1 -- 172.21.15.94:0/1474681466 wait complete. 2024-04-26T02:28:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa98aa3640 1 Processor -- start 2024-04-26T02:28:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa98aa3640 1 -- start start 2024-04-26T02:28:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa98aa3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa940ce010 0x7faa94150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa98aa3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa94164170 0x7faa94166870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa98aa3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa94168720 0x7faa9415b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa98aa3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa940b76d0 con 0x7faa94168720 2024-04-26T02:28:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa98aa3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa940b7550 con 0x7faa940ce010 2024-04-26T02:28:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa98aa3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa94074120 con 0x7faa94164170 2024-04-26T02:28:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa92d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa94168720 0x7faa9415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa92d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa94168720 0x7faa9415b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40982/0 (socket says 172.21.15.94:40982) 2024-04-26T02:28:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa92d9d640 1 -- 172.21.15.94:0/4118646242 learned_addr learned my addr 172.21.15.94:0/4118646242 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa9259c640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa940ce010 0x7faa94150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa91d9b640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa94164170 0x7faa94166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa92d9d640 1 -- 172.21.15.94:0/4118646242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa94164170 msgr2=0x7faa94166870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa92d9d640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa94164170 0x7faa94166870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa92d9d640 1 -- 172.21.15.94:0/4118646242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa940ce010 msgr2=0x7faa94150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa92d9d640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa940ce010 0x7faa94150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa92d9d640 1 -- 172.21.15.94:0/4118646242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa94202090 con 0x7faa94168720 2024-04-26T02:28:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa92d9d640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa94168720 0x7faa9415b100 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7faa8800b410 tx=0x7faa8800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa82ffd640 1 -- 172.21.15.94:0/4118646242 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88015030 con 0x7faa94168720 2024-04-26T02:28:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa82ffd640 1 -- 172.21.15.94:0/4118646242 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa88004020 con 0x7faa94168720 2024-04-26T02:28:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa98aa3640 1 -- 172.21.15.94:0/4118646242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa941608f0 con 0x7faa94168720 2024-04-26T02:28:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa82ffd640 1 -- 172.21.15.94:0/4118646242 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88004410 con 0x7faa94168720 2024-04-26T02:28:28.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.302+0000 7faa98aa3640 1 -- 172.21.15.94:0/4118646242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa94160eb0 con 0x7faa94168720 2024-04-26T02:28:28.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.306+0000 7faa98aa3640 1 -- 172.21.15.94:0/4118646242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9406a730 con 0x7faa94168720 2024-04-26T02:28:28.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.306+0000 7faa82ffd640 1 -- 172.21.15.94:0/4118646242 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faa88018070 con 0x7faa94168720 2024-04-26T02:28:28.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.306+0000 7faa82ffd640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa7406f430 0x7faa74071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.306+0000 7faa9259c640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa7406f430 0x7faa74071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.306+0000 7faa82ffd640 1 -- 172.21.15.94:0/4118646242 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7faa8808ec00 con 0x7faa94168720 2024-04-26T02:28:28.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.306+0000 7faa9259c640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa7406f430 0x7faa74071940 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7faa8c000f70 tx=0x7faa8c007cf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.310+0000 7faa82ffd640 1 -- 172.21.15.94:0/4118646242 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faa88095080 con 0x7faa94168720 2024-04-26T02:28:28.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.458+0000 7faa98aa3640 1 -- 172.21.15.94:0/4118646242 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7faa94150690 con 0x7faa7406f430 2024-04-26T02:28:28.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.458+0000 7faa82ffd640 1 -- 172.21.15.94:0/4118646242 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7faa94150690 con 0x7faa7406f430 2024-04-26T02:28:28.467 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T02:28:28.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.462+0000 7faa98aa3640 1 -- 172.21.15.94:0/4118646242 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa7406f430 msgr2=0x7faa74071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.462+0000 7faa98aa3640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa7406f430 0x7faa74071940 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7faa8c000f70 tx=0x7faa8c007cf0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.462+0000 7faa98aa3640 1 -- 172.21.15.94:0/4118646242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa94168720 msgr2=0x7faa9415b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.462+0000 7faa98aa3640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa94168720 0x7faa9415b100 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7faa8800b410 tx=0x7faa8800b8d0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.462+0000 7faa98aa3640 1 -- 172.21.15.94:0/4118646242 shutdown_connections 2024-04-26T02:28:28.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.462+0000 7faa98aa3640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa94168720 0x7faa9415b100 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.462+0000 7faa98aa3640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7faa7406f430 0x7faa74071940 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.462+0000 7faa98aa3640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa94164170 0x7faa94166870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.462+0000 7faa98aa3640 1 --2- 172.21.15.94:0/4118646242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa940ce010 0x7faa94150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.462+0000 7faa98aa3640 1 -- 172.21.15.94:0/4118646242 >> 172.21.15.94:0/4118646242 conn(0x7faa940d2690 msgr2=0x7faa940b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:28.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.462+0000 7faa98aa3640 1 -- 172.21.15.94:0/4118646242 shutdown_connections 2024-04-26T02:28:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.462+0000 7faa98aa3640 1 -- 172.21.15.94:0/4118646242 wait complete. 2024-04-26T02:28:28.486 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T02:28:28.487 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T02:28:28.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.634+0000 7f4ffb62b640 1 Processor -- start 2024-04-26T02:28:28.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.634+0000 7f4ffb62b640 1 -- start start 2024-04-26T02:28:28.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ffb62b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4ff40d0c20 0x7f4ff4163600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ffb62b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ff4163b40 0x7f4ff4160010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ffb62b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4ff4160550 0x7f4ff4160960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ffb62b640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f4ff40d1030 con 0x7f4ff40d0c20 2024-04-26T02:28:28.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ffb62b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4ff4163f50 con 0x7f4ff4160550 2024-04-26T02:28:28.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ffb62b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff4160ea0 con 0x7f4ff4163b40 2024-04-26T02:28:28.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ff93c7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4ff40d0c20 0x7f4ff4163600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ff93c7640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4ff40d0c20 0x7f4ff4163600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40984/0 (socket says 172.21.15.94:40984) 2024-04-26T02:28:28.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ff93c7640 1 -- 172.21.15.94:0/923149563 learned_addr learned my addr 172.21.15.94:0/923149563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:28.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ff8bc6640 1 --2- 172.21.15.94:0/923149563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ff4163b40 0x7f4ff4160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ff93c7640 1 -- 172.21.15.94:0/923149563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ff4163b40 msgr2=0x7f4ff4160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:28.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ff9bc8640 1 --2- 172.21.15.94:0/923149563 >> v2:172.21.15.138:3300/0 conn(0x7f4ff4160550 0x7f4ff4160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ff93c7640 1 --2- 172.21.15.94:0/923149563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ff4163b40 0x7f4ff4160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ff93c7640 1 -- 172.21.15.94:0/923149563 >> v2:172.21.15.138:3300/0 conn(0x7f4ff4160550 msgr2=0x7f4ff4160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:28.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ff93c7640 1 --2- 172.21.15.94:0/923149563 >> v2:172.21.15.138:3300/0 conn(0x7f4ff4160550 0x7f4ff4160960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ff93c7640 1 -- 172.21.15.94:0/923149563 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff40cf760 con 0x7f4ff40d0c20 2024-04-26T02:28:28.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ff8bc6640 1 --2- 172.21.15.94:0/923149563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ff4163b40 0x7f4ff4160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:28.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4ff93c7640 1 --2- 172.21.15.94:0/923149563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff40d0c20 0x7f4ff4163600 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4070a80 tx=0x7f4ff001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5f190f0953be4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4febfff640 1 -- 172.21.15.94:0/923149563 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00ab620 con 0x7f4ff40d0c20 2024-04-26T02:28:28.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4febfff640 1 -- 172.21.15.94:0/923149563 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff00b0070 con 0x7f4ff40d0c20 2024-04-26T02:28:28.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.638+0000 7f4febfff640 1 -- 172.21.15.94:0/923149563 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00ab9e0 con 0x7f4ff40d0c20 2024-04-26T02:28:28.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 -- 172.21.15.94:0/923149563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff40d0c20 msgr2=0x7f4ff4163600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 --2- 172.21.15.94:0/923149563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff40d0c20 0x7f4ff4163600 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4070a80 tx=0x7f4ff001c460 comp rx=0 tx=0).stop 2024-04-26T02:28:28.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 -- 172.21.15.94:0/923149563 shutdown_connections 2024-04-26T02:28:28.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 --2- 172.21.15.94:0/923149563 >> v2:172.21.15.138:3300/0 conn(0x7f4ff4160550 0x7f4ff4160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 --2- 172.21.15.94:0/923149563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ff4163b40 0x7f4ff4160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 --2- 172.21.15.94:0/923149563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff40d0c20 0x7f4ff4163600 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 -- 172.21.15.94:0/923149563 >> 172.21.15.94:0/923149563 conn(0x7f4ff40711c0 msgr2=0x7f4ff40715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:28.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 -- 172.21.15.94:0/923149563 shutdown_connections 2024-04-26T02:28:28.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 -- 172.21.15.94:0/923149563 wait complete. 2024-04-26T02:28:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 Processor -- start 2024-04-26T02:28:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 -- start start 2024-04-26T02:28:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff40d0c20 0x7f4ff40af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff4160550 0x7f4ff4144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ff4163b40 0x7f4ff40739b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff415d980 con 0x7f4ff40d0c20 2024-04-26T02:28:28.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff40d02e0 con 0x7f4ff4160550 2024-04-26T02:28:28.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ffb62b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff40cf3b0 con 0x7f4ff4163b40 2024-04-26T02:28:28.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ff93c7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff40d0c20 0x7f4ff40af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ff93c7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff40d0c20 0x7f4ff40af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40994/0 (socket says 172.21.15.94:40994) 2024-04-26T02:28:28.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ff93c7640 1 -- 172.21.15.94:0/3068251427 learned_addr learned my addr 172.21.15.94:0/3068251427 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:28.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ff8bc6640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff4160550 0x7f4ff4144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ff93c7640 1 -- 172.21.15.94:0/3068251427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ff4163b40 msgr2=0x7f4ff40739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.642+0000 7f4ff9bc8640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ff4163b40 0x7f4ff40739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4ff93c7640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4ff4163b40 0x7f4ff40739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4ff93c7640 1 -- 172.21.15.94:0/3068251427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff4160550 msgr2=0x7f4ff4144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4ff93c7640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff4160550 0x7f4ff4144090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4ff93c7640 1 -- 172.21.15.94:0/3068251427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff415e4d0 con 0x7f4ff40d0c20 2024-04-26T02:28:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4ff8bc6640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff4160550 0x7f4ff4144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4ff93c7640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff40d0c20 0x7f4ff40af400 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f4ff001cd00 tx=0x7f4ff0019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4fe9ffb640 1 -- 172.21.15.94:0/3068251427 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0005ba0 con 0x7f4ff40d0c20 2024-04-26T02:28:28.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4fe9ffb640 1 -- 172.21.15.94:0/3068251427 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff00044e0 con 0x7f4ff40d0c20 2024-04-26T02:28:28.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4fe9ffb640 1 -- 172.21.15.94:0/3068251427 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0003830 con 0x7f4ff40d0c20 2024-04-26T02:28:28.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4ffb62b640 1 -- 172.21.15.94:0/3068251427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff415e760 con 0x7f4ff40d0c20 2024-04-26T02:28:28.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4ffb62b640 1 -- 172.21.15.94:0/3068251427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff41554e0 con 0x7f4ff40d0c20 2024-04-26T02:28:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4ffb62b640 1 -- 172.21.15.94:0/3068251427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ff415ec80 con 0x7f4ff40d0c20 2024-04-26T02:28:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4fe9ffb640 1 -- 172.21.15.94:0/3068251427 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4ff00039d0 con 0x7f4ff40d0c20 2024-04-26T02:28:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4fe9ffb640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4fd806f400 0x7f4fd8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4ff8bc6640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4fd806f400 0x7f4fd8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4fe9ffb640 1 -- 172.21.15.94:0/3068251427 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f4ff0129460 con 0x7f4ff40d0c20 2024-04-26T02:28:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4fe9ffb640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] conn(0x7f4fd80758f0 0x7f4fd8077d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4fe9ffb640 1 -- 172.21.15.94:0/3068251427 --> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ff0001d40 con 0x7f4fd80758f0 2024-04-26T02:28:28.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4fe9ffb640 1 -- 172.21.15.94:0/3068251427 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4ff0130050 con 0x7f4ff40d0c20 2024-04-26T02:28:28.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4ff8bc6640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4fd806f400 0x7f4fd8071910 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4074550 tx=0x7f4fe4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.646+0000 7f4ff9bc8640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] conn(0x7f4fd80758f0 0x7f4fd8077d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.650+0000 7f4ff9bc8640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] conn(0x7f4fd80758f0 0x7f4fd8077d80 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.650+0000 7f4fe9ffb640 1 -- 172.21.15.94:0/3068251427 <== osd.5 v2:172.21.15.138:6823/2199406473 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ff0001d40 con 0x7f4fd80758f0 2024-04-26T02:28:28.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.662+0000 7f4ffb62b640 1 -- 172.21.15.94:0/3068251427 --> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4ff416a560 con 0x7f4fd80758f0 2024-04-26T02:28:28.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.666+0000 7f4fe9ffb640 1 -- 172.21.15.94:0/3068251427 <== osd.5 v2:172.21.15.138:6823/2199406473 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4ff416a560 con 0x7f4fd80758f0 2024-04-26T02:28:28.670 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T02:28:28.670 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T02:28:28.670 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T02:28:28.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.666+0000 7f4ffb62b640 1 -- 172.21.15.94:0/3068251427 >> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] conn(0x7f4fd80758f0 msgr2=0x7f4fd8077d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.666+0000 7f4ffb62b640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.138:6823/2199406473,v1:172.21.15.138:6825/2199406473] conn(0x7f4fd80758f0 0x7f4fd8077d80 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.666+0000 7f4ffb62b640 1 -- 172.21.15.94:0/3068251427 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4fd806f400 msgr2=0x7f4fd8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.666+0000 7f4ffb62b640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4fd806f400 0x7f4fd8071910 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4074550 tx=0x7f4fe4008040 comp rx=0 tx=0).stop 2024-04-26T02:28:28.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.666+0000 7f4ffb62b640 1 -- 172.21.15.94:0/3068251427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff40d0c20 msgr2=0x7f4ff40af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.666+0000 7f4ffb62b640 1 --2- 172.21.15.94:0/3068251427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff40d0c20 0x7f4ff40af400 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f4ff001cd00 tx=0x7f4ff0019200 comp rx=0 tx=0).stop 2024-04-26T02:28:28.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.666+0000 7f4ff9bc8640 1 -- 172.21.15.94:0/3068251427 reap_dead start 2024-04-26T02:28:28.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.666+0000 7f4ffb62b640 1 -- 172.21.15.94:0/3068251427 shutdown_connections 2024-04-26T02:28:28.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.666+0000 7f4ffb62b640 1 -- 172.21.15.94:0/3068251427 >> 172.21.15.94:0/3068251427 conn(0x7f4ff40711c0 msgr2=0x7f4ff40b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:28.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.666+0000 7f4ffb62b640 1 -- 172.21.15.94:0/3068251427 shutdown_connections 2024-04-26T02:28:28.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.666+0000 7f4ffb62b640 1 -- 172.21.15.94:0/3068251427 wait complete. 2024-04-26T02:28:28.691 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T02:28:28.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee8391e640 1 Processor -- start 2024-04-26T02:28:28.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee8391e640 1 -- start start 2024-04-26T02:28:28.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee8391e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fee7c0b0420 0x7fee7c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee8391e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee7c06f2a0 0x7fee7c06c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee8391e640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fee7c06cb40 0x7fee7c06cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee8391e640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fee7c06d490 con 0x7fee7c06cb40 2024-04-26T02:28:28.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee8391e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fee7c0aeaa0 con 0x7fee7c0b0420 2024-04-26T02:28:28.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee8391e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fee7c0703e0 con 0x7fee7c06f2a0 2024-04-26T02:28:28.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee81ebb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fee7c06cb40 0x7fee7c06cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee81ebb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fee7c06cb40 0x7fee7c06cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41004/0 (socket says 172.21.15.94:41004) 2024-04-26T02:28:28.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee81ebb640 1 -- 172.21.15.94:0/1604506181 learned_addr learned my addr 172.21.15.94:0/1604506181 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:28.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee816ba640 1 --2- 172.21.15.94:0/1604506181 >> v2:172.21.15.138:3300/0 conn(0x7fee7c0b0420 0x7fee7c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee80eb9640 1 --2- 172.21.15.94:0/1604506181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee7c06f2a0 0x7fee7c06c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee81ebb640 1 -- 172.21.15.94:0/1604506181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee7c06f2a0 msgr2=0x7fee7c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:28.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee81ebb640 1 --2- 172.21.15.94:0/1604506181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee7c06f2a0 0x7fee7c06c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee81ebb640 1 -- 172.21.15.94:0/1604506181 >> v2:172.21.15.138:3300/0 conn(0x7fee7c0b0420 msgr2=0x7fee7c06ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:28.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee81ebb640 1 --2- 172.21.15.94:0/1604506181 >> v2:172.21.15.138:3300/0 conn(0x7fee7c0b0420 0x7fee7c06ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.834+0000 7fee81ebb640 1 -- 172.21.15.94:0/1604506181 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c070630 con 0x7fee7c06cb40 2024-04-26T02:28:28.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.838+0000 7fee81ebb640 1 --2- 172.21.15.94:0/1604506181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee7c06cb40 0x7fee7c06cf50 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fee7c071670 tx=0x7fee7401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16854c60b8d8c95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.838+0000 7fee73fff640 1 -- 172.21.15.94:0/1604506181 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee740b1070 con 0x7fee7c06cb40 2024-04-26T02:28:28.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.838+0000 7fee73fff640 1 -- 172.21.15.94:0/1604506181 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee740ac630 con 0x7fee7c06cb40 2024-04-26T02:28:28.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.838+0000 7fee73fff640 1 -- 172.21.15.94:0/1604506181 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee740a8050 con 0x7fee7c06cb40 2024-04-26T02:28:28.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.838+0000 7fee8391e640 1 -- 172.21.15.94:0/1604506181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee7c06cb40 msgr2=0x7fee7c06cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.838+0000 7fee8391e640 1 --2- 172.21.15.94:0/1604506181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee7c06cb40 0x7fee7c06cf50 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fee7c071670 tx=0x7fee7401c520 comp rx=0 tx=0).stop 2024-04-26T02:28:28.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.838+0000 7fee8391e640 1 -- 172.21.15.94:0/1604506181 shutdown_connections 2024-04-26T02:28:28.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.838+0000 7fee8391e640 1 --2- 172.21.15.94:0/1604506181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee7c06cb40 0x7fee7c06cf50 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.838+0000 7fee8391e640 1 --2- 172.21.15.94:0/1604506181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee7c06f2a0 0x7fee7c06c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.838+0000 7fee8391e640 1 --2- 172.21.15.94:0/1604506181 >> v2:172.21.15.138:3300/0 conn(0x7fee7c0b0420 0x7fee7c06ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.838+0000 7fee8391e640 1 -- 172.21.15.94:0/1604506181 >> 172.21.15.94:0/1604506181 conn(0x7fee7c0b47f0 msgr2=0x7fee7c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:28.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.838+0000 7fee8391e640 1 -- 172.21.15.94:0/1604506181 shutdown_connections 2024-04-26T02:28:28.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.838+0000 7fee8391e640 1 -- 172.21.15.94:0/1604506181 wait complete. 2024-04-26T02:28:28.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee8391e640 1 Processor -- start 2024-04-26T02:28:28.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee8391e640 1 -- start start 2024-04-26T02:28:28.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee8391e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7c06cb40 0x7fee7c148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee8391e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee7c06f2a0 0x7fee7c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee8391e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee7c0b0420 0x7fee7c0b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee8391e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c0b0830 con 0x7fee7c06f2a0 2024-04-26T02:28:28.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee8391e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c0aef30 con 0x7fee7c06cb40 2024-04-26T02:28:28.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee8391e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fee7c0b09b0 con 0x7fee7c0b0420 2024-04-26T02:28:28.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee80eb9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee7c06f2a0 0x7fee7c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee80eb9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee7c06f2a0 0x7fee7c073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41020/0 (socket says 172.21.15.94:41020) 2024-04-26T02:28:28.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee80eb9640 1 -- 172.21.15.94:0/4002519217 learned_addr learned my addr 172.21.15.94:0/4002519217 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:28.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee816ba640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7c06cb40 0x7fee7c148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee81ebb640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee7c0b0420 0x7fee7c0b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee80eb9640 1 -- 172.21.15.94:0/4002519217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee7c0b0420 msgr2=0x7fee7c0b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee80eb9640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee7c0b0420 0x7fee7c0b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee80eb9640 1 -- 172.21.15.94:0/4002519217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7c06cb40 msgr2=0x7fee7c148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:28.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee80eb9640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7c06cb40 0x7fee7c148cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:28.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee80eb9640 1 -- 172.21.15.94:0/4002519217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c073b50 con 0x7fee7c06f2a0 2024-04-26T02:28:28.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee816ba640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7c06cb40 0x7fee7c148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:28:28.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee80eb9640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee7c06f2a0 0x7fee7c073f90 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7fee78007e90 tx=0x7fee7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee71ffb640 1 -- 172.21.15.94:0/4002519217 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78014420 con 0x7fee7c06f2a0 2024-04-26T02:28:28.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee71ffb640 1 -- 172.21.15.94:0/4002519217 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee78019070 con 0x7fee7c06f2a0 2024-04-26T02:28:28.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee8391e640 1 -- 172.21.15.94:0/4002519217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7c072ab0 con 0x7fee7c06f2a0 2024-04-26T02:28:28.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee71ffb640 1 -- 172.21.15.94:0/4002519217 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780147e0 con 0x7fee7c06f2a0 2024-04-26T02:28:28.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee8391e640 1 -- 172.21.15.94:0/4002519217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee7c073070 con 0x7fee7c06f2a0 2024-04-26T02:28:28.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.842+0000 7fee8391e640 1 -- 172.21.15.94:0/4002519217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee7c072820 con 0x7fee7c06f2a0 2024-04-26T02:28:28.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.846+0000 7fee71ffb640 1 -- 172.21.15.94:0/4002519217 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fee78014980 con 0x7fee7c06f2a0 2024-04-26T02:28:28.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.846+0000 7fee71ffb640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fee6006f430 0x7fee60071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:28.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.846+0000 7fee71ffb640 1 -- 172.21.15.94:0/4002519217 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fee7808f890 con 0x7fee7c06f2a0 2024-04-26T02:28:28.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.846+0000 7fee816ba640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fee6006f430 0x7fee60071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:28.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.846+0000 7fee816ba640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fee6006f430 0x7fee60071940 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0cd2f0 tx=0x7fee6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:28.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:28.850+0000 7fee71ffb640 1 -- 172.21.15.94:0/4002519217 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fee7805c980 con 0x7fee7c06f2a0 2024-04-26T02:28:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.006+0000 7fee8391e640 1 -- 172.21.15.94:0/4002519217 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fee7c076830 con 0x7fee6006f430 2024-04-26T02:28:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.006+0000 7fee71ffb640 1 -- 172.21.15.94:0/4002519217 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fee7c076830 con 0x7fee6006f430 2024-04-26T02:28:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T02:28:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.010+0000 7fee8391e640 1 -- 172.21.15.94:0/4002519217 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fee6006f430 msgr2=0x7fee60071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.010+0000 7fee8391e640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fee6006f430 0x7fee60071940 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0cd2f0 tx=0x7fee6c008040 comp rx=0 tx=0).stop 2024-04-26T02:28:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.010+0000 7fee8391e640 1 -- 172.21.15.94:0/4002519217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee7c06f2a0 msgr2=0x7fee7c073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.010+0000 7fee8391e640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee7c06f2a0 0x7fee7c073f90 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7fee78007e90 tx=0x7fee7800e4b0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.010+0000 7fee8391e640 1 -- 172.21.15.94:0/4002519217 shutdown_connections 2024-04-26T02:28:29.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.010+0000 7fee8391e640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fee6006f430 0x7fee60071940 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.010+0000 7fee8391e640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee7c0b0420 0x7fee7c0b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.010+0000 7fee8391e640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee7c06f2a0 0x7fee7c073f90 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.010+0000 7fee8391e640 1 --2- 172.21.15.94:0/4002519217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7c06cb40 0x7fee7c148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.010+0000 7fee8391e640 1 -- 172.21.15.94:0/4002519217 >> 172.21.15.94:0/4002519217 conn(0x7fee7c0b47f0 msgr2=0x7fee7c1578a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:29.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.010+0000 7fee8391e640 1 -- 172.21.15.94:0/4002519217 shutdown_connections 2024-04-26T02:28:29.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.010+0000 7fee8391e640 1 -- 172.21.15.94:0/4002519217 wait complete. 2024-04-26T02:28:29.033 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T02:28:29.034 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T02:28:29.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.170+0000 7ffbd7b6f640 1 Processor -- start 2024-04-26T02:28:29.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.170+0000 7ffbd7b6f640 1 -- start start 2024-04-26T02:28:29.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.170+0000 7ffbd7b6f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffbd00b35d0 0x7ffbd015ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.170+0000 7ffbd7b6f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbd00d2f70 0x7ffbd015e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.170+0000 7ffbd7b6f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbd00d1de0 con 0x7ffbd00d2b60 2024-04-26T02:28:29.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.170+0000 7ffbd7b6f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ffbd00b67d0 con 0x7ffbd00b35d0 2024-04-26T02:28:29.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.170+0000 7ffbd7b6f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffbd00d2080 con 0x7ffbd00d2f70 2024-04-26T02:28:29.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.170+0000 7ffbd510a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffbd00b35d0 0x7ffbd015ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.170+0000 7ffbd610c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbd00d2f70 0x7ffbd015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.170+0000 7ffbd610c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbd00d2f70 0x7ffbd015e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.94:37190/0 (socket says 172.21.15.94:37190) 2024-04-26T02:28:29.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.170+0000 7ffbd510a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffbd00b35d0 0x7ffbd015ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:39024/0 (socket says 172.21.15.94:39024) 2024-04-26T02:28:29.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.170+0000 7ffbd610c640 1 -- 172.21.15.94:0/2602292630 learned_addr learned my addr 172.21.15.94:0/2602292630 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:29.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.174+0000 7ffbd610c640 1 -- 172.21.15.94:0/2602292630 >> v2:172.21.15.138:3300/0 conn(0x7ffbd00b35d0 msgr2=0x7ffbd015ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:29.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.174+0000 7ffbd610c640 1 --2- 172.21.15.94:0/2602292630 >> v2:172.21.15.138:3300/0 conn(0x7ffbd00b35d0 0x7ffbd015ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.174+0000 7ffbd610c640 1 -- 172.21.15.94:0/2602292630 >> v1:172.21.15.138:6789/0 conn(0x7ffbd00d2b60 legacy=0x7ffbd00b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.174+0000 7ffbd610c640 1 -- 172.21.15.94:0/2602292630 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd00d2360 con 0x7ffbd00d2f70 2024-04-26T02:28:29.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.174+0000 7ffbd610c640 1 --2- 172.21.15.94:0/2602292630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbd00d2f70 0x7ffbd015e310 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009e50 tx=0x7ffbc801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e334e38fbdd2eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.174+0000 7ffbd4909640 1 -- 172.21.15.94:0/2602292630 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc80a5810 con 0x7ffbd00d2f70 2024-04-26T02:28:29.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.174+0000 7ffbd4909640 1 -- 172.21.15.94:0/2602292630 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc80a59b0 con 0x7ffbd00d2f70 2024-04-26T02:28:29.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.174+0000 7ffbd4909640 1 -- 172.21.15.94:0/2602292630 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc80a5d80 con 0x7ffbd00d2f70 2024-04-26T02:28:29.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.178+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/2602292630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbd00d2f70 msgr2=0x7ffbd015e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.178+0000 7ffbd7b6f640 1 --2- 172.21.15.94:0/2602292630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbd00d2f70 0x7ffbd015e310 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009e50 tx=0x7ffbc801ca30 comp rx=0 tx=0).stop 2024-04-26T02:28:29.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.178+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/2602292630 shutdown_connections 2024-04-26T02:28:29.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.178+0000 7ffbd7b6f640 1 --2- 172.21.15.94:0/2602292630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbd00d2f70 0x7ffbd015e310 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.178+0000 7ffbd7b6f640 1 --2- 172.21.15.94:0/2602292630 >> v2:172.21.15.138:3300/0 conn(0x7ffbd00b35d0 0x7ffbd015ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.178+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/2602292630 >> 172.21.15.94:0/2602292630 conn(0x7ffbd00d3610 msgr2=0x7ffbd00d3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:29.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.178+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/2602292630 shutdown_connections 2024-04-26T02:28:29.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.178+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/2602292630 wait complete. 2024-04-26T02:28:29.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.178+0000 7ffbd7b6f640 1 Processor -- start 2024-04-26T02:28:29.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.178+0000 7ffbd7b6f640 1 -- start start 2024-04-26T02:28:29.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd7b6f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd00b35d0 0x7ffbd00719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd7b6f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbd00d2b60 0x7ffbd0158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd7b6f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbd00d2f70 0x7ffbd00cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd7b6f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd015f1e0 con 0x7ffbd00b35d0 2024-04-26T02:28:29.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd7b6f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd015f060 con 0x7ffbd00d2b60 2024-04-26T02:28:29.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd7b6f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffbd016acc0 con 0x7ffbd00d2f70 2024-04-26T02:28:29.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd590b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd00b35d0 0x7ffbd00719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd590b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd00b35d0 0x7ffbd00719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41034/0 (socket says 172.21.15.94:41034) 2024-04-26T02:28:29.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd590b640 1 -- 172.21.15.94:0/1247381283 learned_addr learned my addr 172.21.15.94:0/1247381283 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:29.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd510a640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbd00d2b60 0x7ffbd0158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd610c640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbd00d2f70 0x7ffbd00cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd590b640 1 -- 172.21.15.94:0/1247381283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbd00d2f70 msgr2=0x7ffbd00cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd590b640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbd00d2f70 0x7ffbd00cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd590b640 1 -- 172.21.15.94:0/1247381283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbd00d2b60 msgr2=0x7ffbd0158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd590b640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbd00d2b60 0x7ffbd0158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd590b640 1 -- 172.21.15.94:0/1247381283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd0070bf0 con 0x7ffbd00b35d0 2024-04-26T02:28:29.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd510a640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbd00d2b60 0x7ffbd0158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:29.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd610c640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbd00d2f70 0x7ffbd00cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:29.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd590b640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd00b35d0 0x7ffbd00719b0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00c390 tx=0x7ffbcc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbc67fc640 1 -- 172.21.15.94:0/1247381283 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc017070 con 0x7ffbd00b35d0 2024-04-26T02:28:29.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbc67fc640 1 -- 172.21.15.94:0/1247381283 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbcc0052d0 con 0x7ffbd00b35d0 2024-04-26T02:28:29.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/1247381283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd0070ed0 con 0x7ffbd00b35d0 2024-04-26T02:28:29.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbc67fc640 1 -- 172.21.15.94:0/1247381283 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc003820 con 0x7ffbd00b35d0 2024-04-26T02:28:29.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/1247381283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd0071ef0 con 0x7ffbd00b35d0 2024-04-26T02:28:29.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/1247381283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffbd00d1730 con 0x7ffbd00b35d0 2024-04-26T02:28:29.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.182+0000 7ffbc67fc640 1 -- 172.21.15.94:0/1247381283 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffbcc01a070 con 0x7ffbd00b35d0 2024-04-26T02:28:29.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.186+0000 7ffbc67fc640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ffbb406f430 0x7ffbb4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.186+0000 7ffbc67fc640 1 -- 172.21.15.94:0/1247381283 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7ffbcc08e830 con 0x7ffbd00b35d0 2024-04-26T02:28:29.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.186+0000 7ffbd510a640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ffbb406f430 0x7ffbb4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.186+0000 7ffbc67fc640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] conn(0x7ffbb4075920 0x7ffbb4077db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.186+0000 7ffbc67fc640 1 -- 172.21.15.94:0/1247381283 --> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffbb4078480 con 0x7ffbb4075920 2024-04-26T02:28:29.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.186+0000 7ffbc67fc640 1 -- 172.21.15.94:0/1247381283 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ffbcc08ecc0 con 0x7ffbd00b35d0 2024-04-26T02:28:29.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.186+0000 7ffbd510a640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ffbb406f430 0x7ffbb4071940 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00cdd90 tx=0x7ffbc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.186+0000 7ffbd610c640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] conn(0x7ffbb4075920 0x7ffbb4077db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.186+0000 7ffbd610c640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] conn(0x7ffbb4075920 0x7ffbb4077db0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.194+0000 7ffbc67fc640 1 -- 172.21.15.94:0/1247381283 <== osd.6 v2:172.21.15.138:6802/3744891971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffbb4078480 con 0x7ffbb4075920 2024-04-26T02:28:29.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.210+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/1247381283 --> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffbd006aa10 con 0x7ffbb4075920 2024-04-26T02:28:29.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.210+0000 7ffbc67fc640 1 -- 172.21.15.94:0/1247381283 <== osd.6 v2:172.21.15.138:6802/3744891971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffbd006aa10 con 0x7ffbb4075920 2024-04-26T02:28:29.215 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T02:28:29.215 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T02:28:29.215 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T02:28:29.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.210+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/1247381283 >> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] conn(0x7ffbb4075920 msgr2=0x7ffbb4077db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.210+0000 7ffbd7b6f640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.138:6802/3744891971,v1:172.21.15.138:6804/3744891971] conn(0x7ffbb4075920 0x7ffbb4077db0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.210+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/1247381283 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ffbb406f430 msgr2=0x7ffbb4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.210+0000 7ffbd7b6f640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7ffbb406f430 0x7ffbb4071940 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00cdd90 tx=0x7ffbc0008040 comp rx=0 tx=0).stop 2024-04-26T02:28:29.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.210+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/1247381283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd00b35d0 msgr2=0x7ffbd00719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.210+0000 7ffbd7b6f640 1 --2- 172.21.15.94:0/1247381283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd00b35d0 0x7ffbd00719b0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00c390 tx=0x7ffbcc00c850 comp rx=0 tx=0).stop 2024-04-26T02:28:29.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.210+0000 7ffbd610c640 1 -- 172.21.15.94:0/1247381283 reap_dead start 2024-04-26T02:28:29.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.210+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/1247381283 shutdown_connections 2024-04-26T02:28:29.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.210+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/1247381283 >> 172.21.15.94:0/1247381283 conn(0x7ffbd00d3610 msgr2=0x7ffbd0159770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:29.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.210+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/1247381283 shutdown_connections 2024-04-26T02:28:29.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.210+0000 7ffbd7b6f640 1 -- 172.21.15.94:0/1247381283 wait complete. 2024-04-26T02:28:29.234 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T02:28:29.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.362+0000 7f2d89478640 1 Processor -- start 2024-04-26T02:28:29.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.362+0000 7f2d89478640 1 -- start start 2024-04-26T02:28:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.362+0000 7f2d89478640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2d8406f470 0x7f2d8415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.362+0000 7f2d89478640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d84070630 con 0x7f2d840b0420 2024-04-26T02:28:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.362+0000 7f2d89478640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d840ae170 con 0x7f2d8415e160 2024-04-26T02:28:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.362+0000 7f2d89478640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f2d840aeaa0 con 0x7f2d8406f470 2024-04-26T02:28:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.362+0000 7f2d827fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2d8406f470 0x7f2d8415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.362+0000 7f2d827fc640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2d8406f470 0x7f2d8415dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41044/0 (socket says 172.21.15.94:41044) 2024-04-26T02:28:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.362+0000 7f2d827fc640 1 -- 172.21.15.94:0/218696116 learned_addr learned my addr 172.21.15.94:0/218696116 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.362+0000 7f2d82ffd640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f2d840b0420 0x7f2d8406ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51292/0 (socket says 172.21.15.94:51292) 2024-04-26T02:28:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.362+0000 7f2d827fc640 1 -- 172.21.15.94:0/218696116 >> v1:172.21.15.138:6789/0 conn(0x7f2d8415e160 legacy=0x7f2d8415e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.362+0000 7f2d827fc640 1 -- 172.21.15.94:0/218696116 >> v1:172.21.15.94:6789/0 conn(0x7f2d840b0420 legacy=0x7f2d8406ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.362+0000 7f2d827fc640 1 -- 172.21.15.94:0/218696116 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d840ae450 con 0x7f2d8406f470 2024-04-26T02:28:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.366+0000 7f2d827fc640 1 --2- 172.21.15.94:0/218696116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8406f470 0x7f2d8415dc20 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009870 tx=0x7f2d6c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b4d3839c832a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.366+0000 7f2d81ffb640 1 -- 172.21.15.94:0/218696116 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0b0070 con 0x7f2d8406f470 2024-04-26T02:28:29.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.366+0000 7f2d81ffb640 1 -- 172.21.15.94:0/218696116 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6c0ab600 con 0x7f2d8406f470 2024-04-26T02:28:29.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.366+0000 7f2d81ffb640 1 -- 172.21.15.94:0/218696116 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0ab9d0 con 0x7f2d8406f470 2024-04-26T02:28:29.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.366+0000 7f2d89478640 1 -- 172.21.15.94:0/218696116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8406f470 msgr2=0x7f2d8415dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.366+0000 7f2d89478640 1 --2- 172.21.15.94:0/218696116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8406f470 0x7f2d8415dc20 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009870 tx=0x7f2d6c01c4c0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.366+0000 7f2d89478640 1 -- 172.21.15.94:0/218696116 shutdown_connections 2024-04-26T02:28:29.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.366+0000 7f2d89478640 1 --2- 172.21.15.94:0/218696116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8406f470 0x7f2d8415dc20 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.366+0000 7f2d89478640 1 -- 172.21.15.94:0/218696116 >> 172.21.15.94:0/218696116 conn(0x7f2d840b47f0 msgr2=0x7f2d840b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:29.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.366+0000 7f2d89478640 1 -- 172.21.15.94:0/218696116 shutdown_connections 2024-04-26T02:28:29.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.366+0000 7f2d89478640 1 -- 172.21.15.94:0/218696116 wait complete. 2024-04-26T02:28:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d89478640 1 Processor -- start 2024-04-26T02:28:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d89478640 1 -- start start 2024-04-26T02:28:29.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d89478640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d840b0420 0x7f2d8409a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d89478640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d8415e160 0x7f2d84141be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d89478640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d84142120 0x7f2d84141030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d89478640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8416b880 con 0x7f2d840b0420 2024-04-26T02:28:29.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d89478640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8416b700 con 0x7f2d84142120 2024-04-26T02:28:29.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d89478640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8416ba00 con 0x7f2d8415e160 2024-04-26T02:28:29.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d82ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d840b0420 0x7f2d8409a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d82ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d840b0420 0x7f2d8409a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41048/0 (socket says 172.21.15.94:41048) 2024-04-26T02:28:29.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d82ffd640 1 -- 172.21.15.94:0/1464093620 learned_addr learned my addr 172.21.15.94:0/1464093620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:29.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d827fc640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d8415e160 0x7f2d84141be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d837fe640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d84142120 0x7f2d84141030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d82ffd640 1 -- 172.21.15.94:0/1464093620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d8415e160 msgr2=0x7f2d84141be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d82ffd640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d8415e160 0x7f2d84141be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d82ffd640 1 -- 172.21.15.94:0/1464093620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d84142120 msgr2=0x7f2d84141030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d82ffd640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d84142120 0x7f2d84141030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d82ffd640 1 -- 172.21.15.94:0/1464093620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8409aed0 con 0x7f2d840b0420 2024-04-26T02:28:29.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d82ffd640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d840b0420 0x7f2d8409a990 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400c760 tx=0x7f2d7400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d7b7fe640 1 -- 172.21.15.94:0/1464093620 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74017070 con 0x7f2d840b0420 2024-04-26T02:28:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d7b7fe640 1 -- 172.21.15.94:0/1464093620 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d7400ce10 con 0x7f2d840b0420 2024-04-26T02:28:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d89478640 1 -- 172.21.15.94:0/1464093620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d84154ce0 con 0x7f2d840b0420 2024-04-26T02:28:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d7b7fe640 1 -- 172.21.15.94:0/1464093620 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400a7e0 con 0x7f2d840b0420 2024-04-26T02:28:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d89478640 1 -- 172.21.15.94:0/1464093620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d841552a0 con 0x7f2d840b0420 2024-04-26T02:28:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.370+0000 7f2d89478640 1 -- 172.21.15.94:0/1464093620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d84076830 con 0x7f2d840b0420 2024-04-26T02:28:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.374+0000 7f2d7b7fe640 1 -- 172.21.15.94:0/1464093620 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2d7401a070 con 0x7f2d840b0420 2024-04-26T02:28:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.374+0000 7f2d7b7fe640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2d6406f430 0x7f2d64071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.374+0000 7f2d7b7fe640 1 -- 172.21.15.94:0/1464093620 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f2d7408e5e0 con 0x7f2d840b0420 2024-04-26T02:28:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.374+0000 7f2d827fc640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2d6406f430 0x7f2d64071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.374+0000 7f2d827fc640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2d6406f430 0x7f2d64071940 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0ad000 tx=0x7f2d6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.378+0000 7f2d7b7fe640 1 -- 172.21.15.94:0/1464093620 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2d7405aff0 con 0x7f2d840b0420 2024-04-26T02:28:29.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.526+0000 7f2d89478640 1 -- 172.21.15.94:0/1464093620 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f2d84144ba0 con 0x7f2d6406f430 2024-04-26T02:28:29.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.526+0000 7f2d7b7fe640 1 -- 172.21.15.94:0/1464093620 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2d84144ba0 con 0x7f2d6406f430 2024-04-26T02:28:29.532 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T02:28:29.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.530+0000 7f2d89478640 1 -- 172.21.15.94:0/1464093620 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2d6406f430 msgr2=0x7f2d64071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.530+0000 7f2d89478640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2d6406f430 0x7f2d64071940 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0ad000 tx=0x7f2d6c008040 comp rx=0 tx=0).stop 2024-04-26T02:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.530+0000 7f2d89478640 1 -- 172.21.15.94:0/1464093620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d840b0420 msgr2=0x7f2d8409a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.530+0000 7f2d89478640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d840b0420 0x7f2d8409a990 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400c760 tx=0x7f2d7400cc20 comp rx=0 tx=0).stop 2024-04-26T02:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.530+0000 7f2d89478640 1 -- 172.21.15.94:0/1464093620 shutdown_connections 2024-04-26T02:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.530+0000 7f2d89478640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d84142120 0x7f2d84141030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.530+0000 7f2d89478640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f2d6406f430 0x7f2d64071940 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.530+0000 7f2d89478640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d8415e160 0x7f2d84141be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.530+0000 7f2d89478640 1 --2- 172.21.15.94:0/1464093620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d840b0420 0x7f2d8409a990 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.530+0000 7f2d89478640 1 -- 172.21.15.94:0/1464093620 >> 172.21.15.94:0/1464093620 conn(0x7f2d840b47f0 msgr2=0x7f2d841562f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.530+0000 7f2d89478640 1 -- 172.21.15.94:0/1464093620 shutdown_connections 2024-04-26T02:28:29.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.530+0000 7f2d89478640 1 -- 172.21.15.94:0/1464093620 wait complete. 2024-04-26T02:28:29.553 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T02:28:29.554 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T02:28:29.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.690+0000 7f4c8b6e4640 1 Processor -- start 2024-04-26T02:28:29.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.690+0000 7f4c8b6e4640 1 -- start start 2024-04-26T02:28:29.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c8b6e4640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4c840cf3c0 0x7f4c840ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c8b6e4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8406d710 0x7f4c8406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c8b6e4640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c84073350 con 0x7f4c840ae5a0 2024-04-26T02:28:29.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c8b6e4640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f4c840d1210 con 0x7f4c840cf3c0 2024-04-26T02:28:29.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c8b6e4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4c8406e060 con 0x7f4c8406d710 2024-04-26T02:28:29.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c89480640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4c840cf3c0 0x7f4c840ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c89480640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4c840cf3c0 0x7f4c840ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41060/0 (socket says 172.21.15.94:41060) 2024-04-26T02:28:29.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c89480640 1 -- 172.21.15.94:0/2033751567 learned_addr learned my addr 172.21.15.94:0/2033751567 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:29.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c89c81640 1 --2- 172.21.15.94:0/2033751567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8406d710 0x7f4c8406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c89480640 1 -- 172.21.15.94:0/2033751567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8406d710 msgr2=0x7f4c8406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:29.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c89480640 1 --2- 172.21.15.94:0/2033751567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8406d710 0x7f4c8406db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c89480640 1 -- 172.21.15.94:0/2033751567 >> v1:172.21.15.94:6789/0 conn(0x7f4c840ae5a0 legacy=0x7f4c84073dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c83fff640 1 -- 172.21.15.94:0/2033751567 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 387148756 0 0) 0x7f4c84073350 con 0x7f4c840ae5a0 2024-04-26T02:28:29.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c89480640 1 -- 172.21.15.94:0/2033751567 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c840cfbf0 con 0x7f4c840cf3c0 2024-04-26T02:28:29.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c89480640 1 --2- 172.21.15.94:0/2033751567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c840cf3c0 0x7f4c840ae060 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d150 tx=0x7f4c7c01ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee19d7560cd12fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c83fff640 1 -- 172.21.15.94:0/2033751567 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c00b2d0 con 0x7f4c840cf3c0 2024-04-26T02:28:29.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c83fff640 1 -- 172.21.15.94:0/2033751567 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c7c00b470 con 0x7f4c840cf3c0 2024-04-26T02:28:29.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c83fff640 1 -- 172.21.15.94:0/2033751567 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0a7bf0 con 0x7f4c840cf3c0 2024-04-26T02:28:29.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/2033751567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c840cf3c0 msgr2=0x7f4c840ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.694+0000 7f4c8b6e4640 1 --2- 172.21.15.94:0/2033751567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c840cf3c0 0x7f4c840ae060 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d150 tx=0x7f4c7c01ec00 comp rx=0 tx=0).stop 2024-04-26T02:28:29.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/2033751567 shutdown_connections 2024-04-26T02:28:29.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 --2- 172.21.15.94:0/2033751567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8406d710 0x7f4c8406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 --2- 172.21.15.94:0/2033751567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c840cf3c0 0x7f4c840ae060 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/2033751567 >> 172.21.15.94:0/2033751567 conn(0x7f4c840d13c0 msgr2=0x7f4c840d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:29.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/2033751567 shutdown_connections 2024-04-26T02:28:29.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/2033751567 wait complete. 2024-04-26T02:28:29.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 Processor -- start 2024-04-26T02:28:29.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 -- start start 2024-04-26T02:28:29.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8406d710 0x7f4c8409a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c840ae5a0 0x7f4c840c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c840cf3c0 0x7f4c840c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c840d0050 con 0x7f4c840ae5a0 2024-04-26T02:28:29.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c840cfed0 con 0x7f4c840cf3c0 2024-04-26T02:28:29.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c8b6e4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4c840cf7d0 con 0x7f4c8406d710 2024-04-26T02:28:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c88c7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c840ae5a0 0x7f4c840c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c88c7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c840ae5a0 0x7f4c840c3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41066/0 (socket says 172.21.15.94:41066) 2024-04-26T02:28:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c88c7f640 1 -- 172.21.15.94:0/1103928526 learned_addr learned my addr 172.21.15.94:0/1103928526 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c89c81640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c840cf3c0 0x7f4c840c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c89480640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8406d710 0x7f4c8409a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c88c7f640 1 -- 172.21.15.94:0/1103928526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8406d710 msgr2=0x7f4c8409a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c88c7f640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c8406d710 0x7f4c8409a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c88c7f640 1 -- 172.21.15.94:0/1103928526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c840cf3c0 msgr2=0x7f4c840c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c88c7f640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c840cf3c0 0x7f4c840c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c88c7f640 1 -- 172.21.15.94:0/1103928526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8409aed0 con 0x7f4c840ae5a0 2024-04-26T02:28:29.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c89c81640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c840cf3c0 0x7f4c840c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:29.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c88c7f640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c840ae5a0 0x7f4c840c3a70 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800c3f0 tx=0x7f4c7800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.698+0000 7f4c81ffb640 1 -- 172.21.15.94:0/1103928526 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78017070 con 0x7f4c840ae5a0 2024-04-26T02:28:29.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c81ffb640 1 -- 172.21.15.94:0/1103928526 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c780052f0 con 0x7f4c840ae5a0 2024-04-26T02:28:29.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/1103928526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c840c7370 con 0x7f4c840ae5a0 2024-04-26T02:28:29.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c81ffb640 1 -- 172.21.15.94:0/1103928526 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78003860 con 0x7f4c840ae5a0 2024-04-26T02:28:29.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/1103928526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c840c7930 con 0x7f4c840ae5a0 2024-04-26T02:28:29.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/1103928526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4c840cef90 con 0x7f4c840ae5a0 2024-04-26T02:28:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c81ffb640 1 -- 172.21.15.94:0/1103928526 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4c7801a070 con 0x7f4c840ae5a0 2024-04-26T02:28:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c81ffb640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4c7006f400 0x7f4c70071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c81ffb640 1 -- 172.21.15.94:0/1103928526 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f4c7808e8e0 con 0x7f4c840ae5a0 2024-04-26T02:28:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c89480640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4c7006f400 0x7f4c70071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c81ffb640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] conn(0x7f4c700758f0 0x7f4c70077d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c81ffb640 1 -- 172.21.15.94:0/1103928526 --> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c70078450 con 0x7f4c700758f0 2024-04-26T02:28:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c81ffb640 1 -- 172.21.15.94:0/1103928526 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4c7808ed70 con 0x7f4c840ae5a0 2024-04-26T02:28:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c89480640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4c7006f400 0x7f4c70071910 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0b0000 tx=0x7f4c7c0a9070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c89c81640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] conn(0x7f4c700758f0 0x7f4c70077d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.702+0000 7f4c89c81640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] conn(0x7f4c700758f0 0x7f4c70077d80 crc :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.714+0000 7f4c81ffb640 1 -- 172.21.15.94:0/1103928526 <== osd.7 v2:172.21.15.138:6800/3634138036 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c70078450 con 0x7f4c700758f0 2024-04-26T02:28:29.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.726+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/1103928526 --> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4c840c7120 con 0x7f4c700758f0 2024-04-26T02:28:29.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.726+0000 7f4c81ffb640 1 -- 172.21.15.94:0/1103928526 <== osd.7 v2:172.21.15.138:6800/3634138036 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4c840c7120 con 0x7f4c700758f0 2024-04-26T02:28:29.731 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-26T02:28:29.731 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T02:28:29.732 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-26T02:28:29.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.726+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/1103928526 >> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] conn(0x7f4c700758f0 msgr2=0x7f4c70077d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.726+0000 7f4c8b6e4640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.138:6800/3634138036,v1:172.21.15.138:6801/3634138036] conn(0x7f4c700758f0 0x7f4c70077d80 crc :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.726+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/1103928526 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4c7006f400 msgr2=0x7f4c70071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.726+0000 7f4c8b6e4640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f4c7006f400 0x7f4c70071910 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0b0000 tx=0x7f4c7c0a9070 comp rx=0 tx=0).stop 2024-04-26T02:28:29.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.726+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/1103928526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c840ae5a0 msgr2=0x7f4c840c3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.726+0000 7f4c8b6e4640 1 --2- 172.21.15.94:0/1103928526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c840ae5a0 0x7f4c840c3a70 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800c3f0 tx=0x7f4c7800c8b0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.726+0000 7f4c89c81640 1 -- 172.21.15.94:0/1103928526 reap_dead start 2024-04-26T02:28:29.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.726+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/1103928526 shutdown_connections 2024-04-26T02:28:29.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.726+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/1103928526 >> 172.21.15.94:0/1103928526 conn(0x7f4c840d13c0 msgr2=0x7f4c840b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:29.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.726+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/1103928526 shutdown_connections 2024-04-26T02:28:29.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.726+0000 7f4c8b6e4640 1 -- 172.21.15.94:0/1103928526 wait complete. 2024-04-26T02:28:29.750 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T02:28:29.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.878+0000 7fba5f627640 1 Processor -- start 2024-04-26T02:28:29.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.878+0000 7fba5f627640 1 -- start start 2024-04-26T02:28:29.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.882+0000 7fba5f627640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fba580ce990 0x7fba5815da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.882+0000 7fba5f627640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba580b64c0 con 0x7fba580d2000 2024-04-26T02:28:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.882+0000 7fba5f627640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba580b67a0 con 0x7fba58165570 2024-04-26T02:28:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.882+0000 7fba5f627640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fba580b6270 con 0x7fba580ce990 2024-04-26T02:28:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.882+0000 7fba5cbc2640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fba580d2000 0x7fba580d2410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51302/0 (socket says 172.21.15.94:51302) 2024-04-26T02:28:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.882+0000 7fba5d3c3640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fba580ce990 0x7fba5815da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.882+0000 7fba5cbc2640 1 -- 172.21.15.94:0/2288496948 learned_addr learned my addr 172.21.15.94:0/2288496948 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.882+0000 7fba4ffff640 1 -- 172.21.15.94:0/2288496948 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 557311408 0 0) 0x7fba580b64c0 con 0x7fba580d2000 2024-04-26T02:28:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.882+0000 7fba4ffff640 1 -- 172.21.15.94:0/2288496948 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba44003530 con 0x7fba580d2000 2024-04-26T02:28:29.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.882+0000 7fba5d3c3640 1 -- 172.21.15.94:0/2288496948 >> v1:172.21.15.138:6789/0 conn(0x7fba58165570 legacy=0x7fba58167990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:29.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.882+0000 7fba5d3c3640 1 -- 172.21.15.94:0/2288496948 >> v1:172.21.15.94:6789/0 conn(0x7fba580d2000 legacy=0x7fba580d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.882+0000 7fba5d3c3640 1 -- 172.21.15.94:0/2288496948 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba580b6a80 con 0x7fba580ce990 2024-04-26T02:28:29.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.886+0000 7fba5d3c3640 1 --2- 172.21.15.94:0/2288496948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba580ce990 0x7fba5815da50 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fba580b63f0 tx=0x7fba5401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a328cdd30b2cf86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.886+0000 7fba4ffff640 1 -- 172.21.15.94:0/2288496948 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba540b5070 con 0x7fba580ce990 2024-04-26T02:28:29.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.886+0000 7fba4ffff640 1 -- 172.21.15.94:0/2288496948 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba540a8950 con 0x7fba580ce990 2024-04-26T02:28:29.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.886+0000 7fba4ffff640 1 -- 172.21.15.94:0/2288496948 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba540a8d20 con 0x7fba580ce990 2024-04-26T02:28:29.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.886+0000 7fba5f627640 1 -- 172.21.15.94:0/2288496948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba580ce990 msgr2=0x7fba5815da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.886+0000 7fba5f627640 1 --2- 172.21.15.94:0/2288496948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba580ce990 0x7fba5815da50 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fba580b63f0 tx=0x7fba5401ec50 comp rx=0 tx=0).stop 2024-04-26T02:28:29.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.886+0000 7fba5f627640 1 -- 172.21.15.94:0/2288496948 shutdown_connections 2024-04-26T02:28:29.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.886+0000 7fba5f627640 1 --2- 172.21.15.94:0/2288496948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba580ce990 0x7fba5815da50 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.886+0000 7fba5f627640 1 -- 172.21.15.94:0/2288496948 >> 172.21.15.94:0/2288496948 conn(0x7fba580cfea0 msgr2=0x7fba580d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:29.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.886+0000 7fba5f627640 1 -- 172.21.15.94:0/2288496948 shutdown_connections 2024-04-26T02:28:29.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.886+0000 7fba5f627640 1 -- 172.21.15.94:0/2288496948 wait complete. 2024-04-26T02:28:29.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5f627640 1 Processor -- start 2024-04-26T02:28:29.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5f627640 1 -- start start 2024-04-26T02:28:29.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5f627640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba580ce990 0x7fba5815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5f627640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba580d2000 0x7fba581604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5f627640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba58165570 0x7fba5815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5f627640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba580cf2e0 con 0x7fba58165570 2024-04-26T02:28:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5f627640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba580cf160 con 0x7fba580d2000 2024-04-26T02:28:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5f627640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fba580cda70 con 0x7fba580ce990 2024-04-26T02:28:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5dbc4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba58165570 0x7fba5815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5dbc4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba58165570 0x7fba5815dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41090/0 (socket says 172.21.15.94:41090) 2024-04-26T02:28:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5dbc4640 1 -- 172.21.15.94:0/3679646484 learned_addr learned my addr 172.21.15.94:0/3679646484 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5d3c3640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba580ce990 0x7fba5815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5cbc2640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba580d2000 0x7fba581604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5dbc4640 1 -- 172.21.15.94:0/3679646484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba580ce990 msgr2=0x7fba5815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5dbc4640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba580ce990 0x7fba5815ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5dbc4640 1 -- 172.21.15.94:0/3679646484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba580d2000 msgr2=0x7fba581604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5dbc4640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba580d2000 0x7fba581604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5dbc4640 1 -- 172.21.15.94:0/3679646484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba581513e0 con 0x7fba58165570 2024-04-26T02:28:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5cbc2640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba580d2000 0x7fba581604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5dbc4640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba58165570 0x7fba5815dfd0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7fba5000ba10 tx=0x7fba5000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba4dffb640 1 -- 172.21.15.94:0/3679646484 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50019070 con 0x7fba58165570 2024-04-26T02:28:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba4dffb640 1 -- 172.21.15.94:0/3679646484 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba5000cb40 con 0x7fba58165570 2024-04-26T02:28:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba4dffb640 1 -- 172.21.15.94:0/3679646484 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50014440 con 0x7fba58165570 2024-04-26T02:28:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5f627640 1 -- 172.21.15.94:0/3679646484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba581516c0 con 0x7fba58165570 2024-04-26T02:28:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5f627640 1 -- 172.21.15.94:0/3679646484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba58151c80 con 0x7fba58165570 2024-04-26T02:28:29.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.890+0000 7fba5f627640 1 -- 172.21.15.94:0/3679646484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba5806a730 con 0x7fba58165570 2024-04-26T02:28:29.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.894+0000 7fba4dffb640 1 -- 172.21.15.94:0/3679646484 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fba5001c070 con 0x7fba58165570 2024-04-26T02:28:29.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.894+0000 7fba4dffb640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fba3c06f400 0x7fba3c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:29.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.894+0000 7fba5d3c3640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fba3c06f400 0x7fba3c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:29.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.894+0000 7fba4dffb640 1 -- 172.21.15.94:0/3679646484 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fba5008ee40 con 0x7fba58165570 2024-04-26T02:28:29.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.894+0000 7fba5d3c3640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fba3c06f400 0x7fba3c071910 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fba540b2000 tx=0x7fba54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:29.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:29.898+0000 7fba4dffb640 1 -- 172.21.15.94:0/3679646484 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba5005c010 con 0x7fba58165570 2024-04-26T02:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba5f627640 1 -- 172.21.15.94:0/3679646484 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fba580ce370 con 0x7fba3c06f400 2024-04-26T02:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba4dffb640 1 -- 172.21.15.94:0/3679646484 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fba580ce370 con 0x7fba3c06f400 2024-04-26T02:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T02:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba5f627640 1 -- 172.21.15.94:0/3679646484 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fba3c06f400 msgr2=0x7fba3c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba5f627640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fba3c06f400 0x7fba3c071910 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fba540b2000 tx=0x7fba54008040 comp rx=0 tx=0).stop 2024-04-26T02:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba5f627640 1 -- 172.21.15.94:0/3679646484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba58165570 msgr2=0x7fba5815dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba5f627640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba58165570 0x7fba5815dfd0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7fba5000ba10 tx=0x7fba5000bed0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba5f627640 1 -- 172.21.15.94:0/3679646484 shutdown_connections 2024-04-26T02:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba5f627640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba58165570 0x7fba5815dfd0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba5f627640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba580d2000 0x7fba581604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba5f627640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7fba3c06f400 0x7fba3c071910 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba5f627640 1 --2- 172.21.15.94:0/3679646484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba580ce990 0x7fba5815ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba5f627640 1 -- 172.21.15.94:0/3679646484 >> 172.21.15.94:0/3679646484 conn(0x7fba580cfea0 msgr2=0x7fba580b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba5f627640 1 -- 172.21.15.94:0/3679646484 shutdown_connections 2024-04-26T02:28:30.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.054+0000 7fba5f627640 1 -- 172.21.15.94:0/3679646484 wait complete. 2024-04-26T02:28:30.081 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T02:28:30.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c4be4f640 1 Processor -- start 2024-04-26T02:28:30.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c4be4f640 1 -- start start 2024-04-26T02:28:30.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c4be4f640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3c44070180 0x7f3c4415cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:30.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c4be4f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3c4406de20 0x7f3c440af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:30.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c4be4f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c4415d620 con 0x7f3c44160bf0 2024-04-26T02:28:30.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c4be4f640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f3c4415dec0 con 0x7f3c44070180 2024-04-26T02:28:30.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c4be4f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3c44160760 con 0x7f3c4406de20 2024-04-26T02:28:30.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c49beb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3c44070180 0x7f3c4415cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:30.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c49beb640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3c44070180 0x7f3c4415cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41106/0 (socket says 172.21.15.94:41106) 2024-04-26T02:28:30.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c49beb640 1 -- 172.21.15.94:0/666406481 learned_addr learned my addr 172.21.15.94:0/666406481 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:30.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c49beb640 1 -- 172.21.15.94:0/666406481 >> v2:172.21.15.138:3300/0 conn(0x7f3c4406de20 msgr2=0x7f3c440af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:30.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c49beb640 1 --2- 172.21.15.94:0/666406481 >> v2:172.21.15.138:3300/0 conn(0x7f3c4406de20 0x7f3c440af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c49beb640 1 -- 172.21.15.94:0/666406481 >> v1:172.21.15.138:6789/0 conn(0x7f3c44160bf0 legacy=0x7f3c4406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:30.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.214+0000 7f3c49beb640 1 -- 172.21.15.94:0/666406481 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4415d900 con 0x7f3c44070180 2024-04-26T02:28:30.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c49beb640 1 --2- 172.21.15.94:0/666406481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c44070180 0x7f3c4415cf40 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f3c440b0d10 tx=0x7f3c4001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246bde75ba05171f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:30.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c48be9640 1 -- 172.21.15.94:0/666406481 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400b1070 con 0x7f3c44070180 2024-04-26T02:28:30.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c48be9640 1 -- 172.21.15.94:0/666406481 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c400ac620 con 0x7f3c44070180 2024-04-26T02:28:30.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c48be9640 1 -- 172.21.15.94:0/666406481 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400ac9f0 con 0x7f3c44070180 2024-04-26T02:28:30.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c4be4f640 1 -- 172.21.15.94:0/666406481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c44070180 msgr2=0x7f3c4415cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:30.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c4be4f640 1 --2- 172.21.15.94:0/666406481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c44070180 0x7f3c4415cf40 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f3c440b0d10 tx=0x7f3c4001c4b0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c4be4f640 1 -- 172.21.15.94:0/666406481 shutdown_connections 2024-04-26T02:28:30.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c4be4f640 1 --2- 172.21.15.94:0/666406481 >> v2:172.21.15.138:3300/0 conn(0x7f3c4406de20 0x7f3c440af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c4be4f640 1 --2- 172.21.15.94:0/666406481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c44070180 0x7f3c4415cf40 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c4be4f640 1 -- 172.21.15.94:0/666406481 >> 172.21.15.94:0/666406481 conn(0x7f3c440b47f0 msgr2=0x7f3c440b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:30.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c4be4f640 1 -- 172.21.15.94:0/666406481 shutdown_connections 2024-04-26T02:28:30.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c4be4f640 1 -- 172.21.15.94:0/666406481 wait complete. 2024-04-26T02:28:30.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c4be4f640 1 Processor -- start 2024-04-26T02:28:30.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.218+0000 7f3c4be4f640 1 -- start start 2024-04-26T02:28:30.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c4be4f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c44070180 0x7f3c440b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:30.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c4be4f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c44160bf0 0x7f3c441f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:30.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c4be4f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4414ff90 0x7f3c44152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:30.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c4be4f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4416a260 con 0x7f3c44070180 2024-04-26T02:28:30.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c4be4f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4416a0e0 con 0x7f3c44160bf0 2024-04-26T02:28:30.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c4be4f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3c4416a3e0 con 0x7f3c4414ff90 2024-04-26T02:28:30.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c49beb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c44070180 0x7f3c440b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:30.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c49beb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c44070180 0x7f3c440b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41122/0 (socket says 172.21.15.94:41122) 2024-04-26T02:28:30.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c49beb640 1 -- 172.21.15.94:0/2647439466 learned_addr learned my addr 172.21.15.94:0/2647439466 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:30.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c49beb640 1 -- 172.21.15.94:0/2647439466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4414ff90 msgr2=0x7f3c44152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:30.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c49beb640 1 --2- 172.21.15.94:0/2647439466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4414ff90 0x7f3c44152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c493ea640 1 --2- 172.21.15.94:0/2647439466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c44160bf0 0x7f3c441f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:30.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c49beb640 1 -- 172.21.15.94:0/2647439466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c44160bf0 msgr2=0x7f3c441f74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:30.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c49beb640 1 --2- 172.21.15.94:0/2647439466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c44160bf0 0x7f3c441f74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c49beb640 1 -- 172.21.15.94:0/2647439466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c440b1e10 con 0x7f3c44070180 2024-04-26T02:28:30.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c49beb640 1 --2- 172.21.15.94:0/2647439466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c44070180 0x7f3c440b18d0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f3c400af040 tx=0x7f3c4001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:30.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c3a7fc640 1 -- 172.21.15.94:0/2647439466 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400b1040 con 0x7f3c44070180 2024-04-26T02:28:30.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c3a7fc640 1 -- 172.21.15.94:0/2647439466 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c400bd070 con 0x7f3c44070180 2024-04-26T02:28:30.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c4be4f640 1 -- 172.21.15.94:0/2647439466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4409c5d0 con 0x7f3c44070180 2024-04-26T02:28:30.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c3a7fc640 1 -- 172.21.15.94:0/2647439466 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400b7880 con 0x7f3c44070180 2024-04-26T02:28:30.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c4be4f640 1 -- 172.21.15.94:0/2647439466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4409cb90 con 0x7f3c44070180 2024-04-26T02:28:30.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c4be4f640 1 -- 172.21.15.94:0/2647439466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c44076830 con 0x7f3c44070180 2024-04-26T02:28:30.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c3a7fc640 1 -- 172.21.15.94:0/2647439466 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3c400b7a20 con 0x7f3c44070180 2024-04-26T02:28:30.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c3a7fc640 1 --2- 172.21.15.94:0/2647439466 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c2806f400 0x7f3c28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:30.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c493ea640 1 --2- 172.21.15.94:0/2647439466 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c2806f400 0x7f3c28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:30.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.222+0000 7f3c3a7fc640 1 -- 172.21.15.94:0/2647439466 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f3c40129510 con 0x7f3c44070180 2024-04-26T02:28:30.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.226+0000 7f3c493ea640 1 --2- 172.21.15.94:0/2647439466 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c2806f400 0x7f3c28071910 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3c4406cc00 tx=0x7f3c34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:30.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.230+0000 7f3c3a7fc640 1 -- 172.21.15.94:0/2647439466 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3c400c7020 con 0x7f3c44070180 2024-04-26T02:28:30.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.382+0000 7f3c4be4f640 1 -- 172.21.15.94:0/2647439466 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c44150a50 con 0x7f3c2806f400 2024-04-26T02:28:30.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.382+0000 7f3c3a7fc640 1 -- 172.21.15.94:0/2647439466 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13653 (secure 0 0 0) 0x7f3c44150a50 con 0x7f3c2806f400 2024-04-26T02:28:30.388 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-26T02:28:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.386+0000 7f3c4be4f640 1 -- 172.21.15.94:0/2647439466 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c2806f400 msgr2=0x7f3c28071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.386+0000 7f3c4be4f640 1 --2- 172.21.15.94:0/2647439466 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c2806f400 0x7f3c28071910 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3c4406cc00 tx=0x7f3c34008040 comp rx=0 tx=0).stop 2024-04-26T02:28:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.386+0000 7f3c4be4f640 1 -- 172.21.15.94:0/2647439466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c44070180 msgr2=0x7f3c440b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.386+0000 7f3c4be4f640 1 --2- 172.21.15.94:0/2647439466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c44070180 0x7f3c440b18d0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f3c400af040 tx=0x7f3c4001acf0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.386+0000 7f3c4be4f640 1 -- 172.21.15.94:0/2647439466 shutdown_connections 2024-04-26T02:28:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.386+0000 7f3c4be4f640 1 --2- 172.21.15.94:0/2647439466 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f3c2806f400 0x7f3c28071910 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.386+0000 7f3c4be4f640 1 --2- 172.21.15.94:0/2647439466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4414ff90 0x7f3c44152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.386+0000 7f3c4be4f640 1 --2- 172.21.15.94:0/2647439466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c44160bf0 0x7f3c441f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.386+0000 7f3c4be4f640 1 --2- 172.21.15.94:0/2647439466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c44070180 0x7f3c440b18d0 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.386+0000 7f3c4be4f640 1 -- 172.21.15.94:0/2647439466 >> 172.21.15.94:0/2647439466 conn(0x7f3c440b47f0 msgr2=0x7f3c4406d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.386+0000 7f3c4be4f640 1 -- 172.21.15.94:0/2647439466 shutdown_connections 2024-04-26T02:28:30.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:30.386+0000 7f3c4be4f640 1 -- 172.21.15.94:0/2647439466 wait complete. 2024-04-26T02:28:30.410 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":17364,"stamp":"2024-04-26T02:28:29.636092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116,"ondisk_log_size":116,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":12674000,"kb_used_data":5128,"kb_used_omap":3913,"kb_used_meta":12664886,"kb_avail":742300720,"statfs":{"total":773094113280,"available":760115937280,"internally_reserved":0,"allocated":5251072,"data_stored":2953736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4007562,"internal_metadata":12968843638},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1339,"apply_latency_ms":1339,"commit_latency_ns":1339000000,"apply_latency_ns":1339000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26359320116,"num_objects":-14721,"num_object_clones":0,"num_object_copies":-29442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-14721,"num_whiteouts":0,"num_read":-1853234,"num_read_kb":-130412818,"num_write":-6601244,"num_write_kb":-199826865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.415877"},"pg_stats":[{"pgid":"1.0","version":"40'116","reported_seq":493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T02:28:19.453534+0000","last_change":"2024-04-25T16:50:42.994752+0000","last_active":"2024-04-26T02:28:19.453534+0000","last_peered":"2024-04-26T02:28:19.453534+0000","last_clean":"2024-04-26T02:28:19.453534+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-26T02:28:19.453534+0000","last_undegraded":"2024-04-26T02:28:19.453534+0000","last_fullsized":"2024-04-26T02:28:19.453534+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:50:41.933048+0000","last_clean_scrub_stamp":"2024-04-25T16:50:41.933048+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:33:04.660319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116,"ondisk_log_size":116,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834581785,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1649468,"kb_used_data":1076,"kb_used_omap":249,"kb_used_meta":1648134,"kb_avail":92722372,"statfs":{"total":96636764160,"available":94947708928,"internally_reserved":0,"allocated":1101824,"data_stored":811993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":254984,"internal_metadata":1687690232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":155,"apply_latency_ms":155,"commit_latency_ns":155000000,"apply_latency_ns":155000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834581783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1733752,"kb_used_data":496,"kb_used_omap":137,"kb_used_meta":1733110,"kb_avail":92638088,"statfs":{"total":96636764160,"available":94861402112,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":140568,"internal_metadata":1774705384},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":398,"apply_latency_ms":398,"commit_latency_ns":398000000,"apply_latency_ns":398000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834581784,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1144056,"kb_used_data":496,"kb_used_omap":829,"kb_used_meta":1142722,"kb_avail":93227784,"statfs":{"total":96636764160,"available":95465250816,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":849456,"internal_metadata":1170147792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":57,"apply_latency_ms":57,"commit_latency_ns":57000000,"apply_latency_ns":57000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834581784,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1398396,"kb_used_data":496,"kb_used_omap":820,"kb_used_meta":1397067,"kb_avail":92973444,"statfs":{"total":96636764160,"available":95204806656,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":840213,"internal_metadata":1430597099},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":93,"apply_latency_ms":93,"commit_latency_ns":93000000,"apply_latency_ns":93000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834581766,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1557240,"kb_used_data":496,"kb_used_omap":308,"kb_used_meta":1556427,"kb_avail":92814600,"statfs":{"total":96636764160,"available":95042150400,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":315597,"internal_metadata":1593782067},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834581771,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1743032,"kb_used_data":496,"kb_used_omap":703,"kb_used_meta":1741824,"kb_avail":92628808,"statfs":{"total":96636764160,"available":94851899392,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":720539,"internal_metadata":1783628133},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":415,"apply_latency_ms":415,"commit_latency_ns":415000000,"apply_latency_ns":415000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834581774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1710076,"kb_used_data":496,"kb_used_omap":239,"kb_used_meta":1709328,"kb_avail":92661764,"statfs":{"total":96636764160,"available":94885646336,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":244932,"internal_metadata":1750352700},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":102,"apply_latency_ms":102,"commit_latency_ns":102000000,"apply_latency_ns":102000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834581775,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1737980,"kb_used_data":1076,"kb_used_omap":626,"kb_used_meta":1736269,"kb_avail":92633860,"statfs":{"total":96636764160,"available":94857072640,"internally_reserved":0,"allocated":1101824,"data_stored":811993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":641273,"internal_metadata":1777940231},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":119,"apply_latency_ms":119,"commit_latency_ns":119000000,"apply_latency_ns":119000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T02:28:30.412 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T16:50:41.933048+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=16, tm_min=50, tm_sec=41, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=2, tm_min=28, tm_sec=24, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T02:28:30.412 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T02:28:50.414 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T02:28:50.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.569+0000 7f69b3340640 1 Processor -- start 2024-04-26T02:28:50.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.569+0000 7f69b3340640 1 -- start start 2024-04-26T02:28:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b3340640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f69ac15cf40 0x7f69ac06d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b3340640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69ac06dc50 0x7f69ac0af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b3340640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69ac06e060 con 0x7f69ac070180 2024-04-26T02:28:50.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b3340640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f69ac15dec0 con 0x7f69ac15cf40 2024-04-26T02:28:50.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b3340640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f69ac160760 con 0x7f69ac06dc50 2024-04-26T02:28:50.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b08db640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f69ac15cf40 0x7f69ac06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b08db640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f69ac15cf40 0x7f69ac06d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35714/0 (socket says 172.21.15.94:35714) 2024-04-26T02:28:50.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b08db640 1 -- 172.21.15.94:0/3534534750 learned_addr learned my addr 172.21.15.94:0/3534534750 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b18dd640 1 --2- 172.21.15.94:0/3534534750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69ac06dc50 0x7f69ac0af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b08db640 1 -- 172.21.15.94:0/3534534750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69ac06dc50 msgr2=0x7f69ac0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b08db640 1 --2- 172.21.15.94:0/3534534750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69ac06dc50 0x7f69ac0af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b08db640 1 -- 172.21.15.94:0/3534534750 >> v1:172.21.15.138:6789/0 conn(0x7f69ac070180 legacy=0x7f69ac160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b08db640 1 -- 172.21.15.94:0/3534534750 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ac161300 con 0x7f69ac15cf40 2024-04-26T02:28:50.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.573+0000 7f69b18dd640 1 --2- 172.21.15.94:0/3534534750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69ac06dc50 0x7f69ac0af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:50.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.577+0000 7f69b08db640 1 --2- 172.21.15.94:0/3534534750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac15cf40 0x7f69ac06d710 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f699c00bb10 tx=0x7f699c01e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47dfe1d8574a20c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:50.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.577+0000 7f69a3fff640 1 -- 172.21.15.94:0/3534534750 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c0b4070 con 0x7f69ac15cf40 2024-04-26T02:28:50.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.577+0000 7f69a3fff640 1 -- 172.21.15.94:0/3534534750 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699c0a7980 con 0x7f69ac15cf40 2024-04-26T02:28:50.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.577+0000 7f69a3fff640 1 -- 172.21.15.94:0/3534534750 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c0a7d50 con 0x7f69ac15cf40 2024-04-26T02:28:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.577+0000 7f69b3340640 1 -- 172.21.15.94:0/3534534750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac15cf40 msgr2=0x7f69ac06d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.577+0000 7f69b3340640 1 --2- 172.21.15.94:0/3534534750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac15cf40 0x7f69ac06d710 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f699c00bb10 tx=0x7f699c01e9f0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.577+0000 7f69b3340640 1 -- 172.21.15.94:0/3534534750 shutdown_connections 2024-04-26T02:28:50.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.577+0000 7f69b3340640 1 --2- 172.21.15.94:0/3534534750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69ac06dc50 0x7f69ac0af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.577+0000 7f69b3340640 1 --2- 172.21.15.94:0/3534534750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac15cf40 0x7f69ac06d710 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.577+0000 7f69b3340640 1 -- 172.21.15.94:0/3534534750 >> 172.21.15.94:0/3534534750 conn(0x7f69ac0b47f0 msgr2=0x7f69ac0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:50.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.577+0000 7f69b3340640 1 -- 172.21.15.94:0/3534534750 shutdown_connections 2024-04-26T02:28:50.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.577+0000 7f69b3340640 1 -- 172.21.15.94:0/3534534750 wait complete. 2024-04-26T02:28:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b3340640 1 Processor -- start 2024-04-26T02:28:50.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b3340640 1 -- start start 2024-04-26T02:28:50.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b3340640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69ac06dc50 0x7f69ac1589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b3340640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac070180 0x7f69ac154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b3340640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69ac15cf40 0x7f69ac09a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b3340640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac06fea0 con 0x7f69ac070180 2024-04-26T02:28:50.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b3340640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac15dbe0 con 0x7f69ac06dc50 2024-04-26T02:28:50.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b3340640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f69ac0afc10 con 0x7f69ac15cf40 2024-04-26T02:28:50.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b08db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac070180 0x7f69ac154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b08db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac070180 0x7f69ac154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35728/0 (socket says 172.21.15.94:35728) 2024-04-26T02:28:50.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b08db640 1 -- 172.21.15.94:0/154791866 learned_addr learned my addr 172.21.15.94:0/154791866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:50.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b18dd640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69ac15cf40 0x7f69ac09a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b10dc640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69ac06dc50 0x7f69ac1589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b08db640 1 -- 172.21.15.94:0/154791866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69ac15cf40 msgr2=0x7f69ac09a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:50.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b08db640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69ac15cf40 0x7f69ac09a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b08db640 1 -- 172.21.15.94:0/154791866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69ac06dc50 msgr2=0x7f69ac1589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:50.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b08db640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69ac06dc50 0x7f69ac1589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b08db640 1 -- 172.21.15.94:0/154791866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ac159190 con 0x7f69ac070180 2024-04-26T02:28:50.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b18dd640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69ac15cf40 0x7f69ac09a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:50.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b08db640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac070180 0x7f69ac154280 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f699c00b2c0 tx=0x7f699c01b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:50.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69a1ffb640 1 -- 172.21.15.94:0/154791866 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c0b4040 con 0x7f69ac070180 2024-04-26T02:28:50.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69a1ffb640 1 -- 172.21.15.94:0/154791866 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699c0c1070 con 0x7f69ac070180 2024-04-26T02:28:50.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b3340640 1 -- 172.21.15.94:0/154791866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69ac15f340 con 0x7f69ac070180 2024-04-26T02:28:50.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69a1ffb640 1 -- 172.21.15.94:0/154791866 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c0afa40 con 0x7f69ac070180 2024-04-26T02:28:50.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b3340640 1 -- 172.21.15.94:0/154791866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69ac1573a0 con 0x7f69ac070180 2024-04-26T02:28:50.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.581+0000 7f69b3340640 1 -- 172.21.15.94:0/154791866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69ac076830 con 0x7f69ac070180 2024-04-26T02:28:50.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.585+0000 7f69a1ffb640 1 -- 172.21.15.94:0/154791866 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f699c0b7070 con 0x7f69ac070180 2024-04-26T02:28:50.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.585+0000 7f69a1ffb640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f699006f400 0x7f6990071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.585+0000 7f69b10dc640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f699006f400 0x7f6990071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.585+0000 7f69a1ffb640 1 -- 172.21.15.94:0/154791866 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f699c12c8f0 con 0x7f69ac070180 2024-04-26T02:28:50.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.585+0000 7f69b10dc640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f699006f400 0x7f6990071910 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f69ac071c30 tx=0x7f69a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:50.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.589+0000 7f69a1ffb640 1 -- 172.21.15.94:0/154791866 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f69ac06e060 con 0x7f69ac070180 2024-04-26T02:28:50.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.745+0000 7f69b3340640 1 -- 172.21.15.94:0/154791866 --> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f69ac158f20 con 0x7f699006f400 2024-04-26T02:28:50.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.749+0000 7f69a1ffb640 1 -- 172.21.15.94:0/154791866 <== mgr.4107 v2:172.21.15.94:6832/1891059149 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13447 (secure 0 0 0) 0x7f69ac158f20 con 0x7f699006f400 2024-04-26T02:28:50.755 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:50.755 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-26T02:28:50.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.749+0000 7f69b3340640 1 -- 172.21.15.94:0/154791866 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f699006f400 msgr2=0x7f6990071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:50.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.749+0000 7f69b3340640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f699006f400 0x7f6990071910 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f69ac071c30 tx=0x7f69a8008040 comp rx=0 tx=0).stop 2024-04-26T02:28:50.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.749+0000 7f69b3340640 1 -- 172.21.15.94:0/154791866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac070180 msgr2=0x7f69ac154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:50.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.749+0000 7f69b3340640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac070180 0x7f69ac154280 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f699c00b2c0 tx=0x7f699c01b790 comp rx=0 tx=0).stop 2024-04-26T02:28:50.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.749+0000 7f69b3340640 1 -- 172.21.15.94:0/154791866 shutdown_connections 2024-04-26T02:28:50.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.749+0000 7f69b3340640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f699006f400 0x7f6990071910 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.749+0000 7f69b3340640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f69ac15cf40 0x7f69ac09a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.749+0000 7f69b3340640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac070180 0x7f69ac154280 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.749+0000 7f69b3340640 1 --2- 172.21.15.94:0/154791866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69ac06dc50 0x7f69ac1589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.749+0000 7f69b3340640 1 -- 172.21.15.94:0/154791866 >> 172.21.15.94:0/154791866 conn(0x7f69ac0b47f0 msgr2=0x7f69ac0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:50.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.749+0000 7f69b3340640 1 -- 172.21.15.94:0/154791866 shutdown_connections 2024-04-26T02:28:50.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.749+0000 7f69b3340640 1 -- 172.21.15.94:0/154791866 wait complete. 2024-04-26T02:28:50.779 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":17374,"stamp":"2024-04-26T02:28:49.641737+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":12674000,"kb_used_data":5128,"kb_used_omap":3913,"kb_used_meta":12664886,"kb_avail":742300720,"statfs":{"total":773094113280,"available":760115937280,"internally_reserved":0,"allocated":5251072,"data_stored":2953736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4007562,"internal_metadata":12968843638},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003512"},"pg_stats":[{"pgid":"1.0","version":"43'118","reported_seq":504,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T02:28:31.760188+0000","last_change":"2024-04-26T02:28:31.760296+0000","last_active":"2024-04-26T02:28:31.760188+0000","last_peered":"2024-04-26T02:28:31.760188+0000","last_clean":"2024-04-26T02:28:31.760188+0000","last_became_active":"2024-04-25T16:50:42.992451+0000","last_became_peered":"2024-04-25T16:50:42.992451+0000","last_unstale":"2024-04-26T02:28:31.760188+0000","last_undegraded":"2024-04-26T02:28:31.760188+0000","last_fullsized":"2024-04-26T02:28:31.760188+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'118","last_scrub_stamp":"2024-04-26T02:28:31.745350+0000","last_deep_scrub":"43'118","last_deep_scrub_stamp":"2024-04-26T02:28:31.745350+0000","last_clean_scrub_stamp":"2024-04-26T02:28:31.745350+0000","objects_scrubbed":2,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:17:03.752518+0000","scrub_duration":855,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":151,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834581789,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1649468,"kb_used_data":1076,"kb_used_omap":249,"kb_used_meta":1648134,"kb_avail":92722372,"statfs":{"total":96636764160,"available":94947708928,"internally_reserved":0,"allocated":1101824,"data_stored":811993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":254984,"internal_metadata":1687690232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834581787,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1733752,"kb_used_data":496,"kb_used_omap":137,"kb_used_meta":1733110,"kb_avail":92638088,"statfs":{"total":96636764160,"available":94861402112,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":140568,"internal_metadata":1774705384},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834581788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1144056,"kb_used_data":496,"kb_used_omap":829,"kb_used_meta":1142722,"kb_avail":93227784,"statfs":{"total":96636764160,"available":95465250816,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":849456,"internal_metadata":1170147792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834581788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1398396,"kb_used_data":496,"kb_used_omap":820,"kb_used_meta":1397067,"kb_avail":92973444,"statfs":{"total":96636764160,"available":95204806656,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":840213,"internal_metadata":1430597099},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834581770,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1557240,"kb_used_data":496,"kb_used_omap":308,"kb_used_meta":1556427,"kb_avail":92814600,"statfs":{"total":96636764160,"available":95042150400,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":315597,"internal_metadata":1593782067},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834581775,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1743032,"kb_used_data":496,"kb_used_omap":703,"kb_used_meta":1741824,"kb_avail":92628808,"statfs":{"total":96636764160,"available":94851899392,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":720539,"internal_metadata":1783628133},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834581778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1710076,"kb_used_data":496,"kb_used_omap":239,"kb_used_meta":1709328,"kb_avail":92661764,"statfs":{"total":96636764160,"available":94885646336,"internally_reserved":0,"allocated":507904,"data_stored":221625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":244932,"internal_metadata":1750352700},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834581779,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1737980,"kb_used_data":1076,"kb_used_omap":626,"kb_used_meta":1736269,"kb_avail":92633860,"statfs":{"total":96636764160,"available":94857072640,"internally_reserved":0,"allocated":1101824,"data_stored":811993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":641273,"internal_metadata":1777940231},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T02:28:50.780 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T02:28:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f697ee640 1 Processor -- start 2024-04-26T02:28:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f697ee640 1 -- start start 2024-04-26T02:28:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f697ee640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1f6406d710 0x7f1f6406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f697ee640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f64074380 con 0x7f1f6406e7a0 2024-04-26T02:28:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f697ee640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f6406e060 con 0x7f1f6415c7e0 2024-04-26T02:28:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f697ee640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1f640727e0 con 0x7f1f6406d710 2024-04-26T02:28:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f637fe640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f1f6406e7a0 0x7f1f6406ebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45652/0 (socket says 172.21.15.94:45652) 2024-04-26T02:28:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f637fe640 1 -- 172.21.15.94:0/892463742 learned_addr learned my addr 172.21.15.94:0/892463742 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-04-26T02:28:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f5bfff640 1 --2- 172.21.15.94:0/892463742 >> v2:172.21.15.138:3300/0 conn(0x7f1f6406d710 0x7f1f6406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3582860145 0 0) 0x7f1f5c0014a0 con 0x7f1f6406e7a0 2024-04-26T02:28:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f50003530 con 0x7f1f6406e7a0 2024-04-26T02:28:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4220672416 0 0) 0x7f1f50003530 con 0x7f1f6406e7a0 2024-04-26T02:28:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 >> v2:172.21.15.138:3300/0 conn(0x7f1f6406d710 msgr2=0x7f1f6406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 --2- 172.21.15.94:0/892463742 >> v2:172.21.15.138:3300/0 conn(0x7f1f6406d710 0x7f1f6406db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 >> v1:172.21.15.138:6789/0 conn(0x7f1f6415c7e0 legacy=0x7f1f640afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f64072ac0 con 0x7f1f6406e7a0 2024-04-26T02:28:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3495770997 0 0) 0x7f1f5c001bd0 con 0x7f1f6406e7a0 2024-04-26T02:28:50.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 >> v1:172.21.15.94:6789/0 conn(0x7f1f6406e7a0 legacy=0x7f1f6406ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:50.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 --2- 172.21.15.94:0/892463742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f500039c0 0x7f1f50003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 --2- 172.21.15.94:0/892463742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f50004810 0x7f1f50004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 --2- 172.21.15.94:0/892463742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f50005260 0x7f1f5000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f640727e0 con 0x7f1f500039c0 2024-04-26T02:28:50.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5000c1f0 con 0x7f1f50005260 2024-04-26T02:28:50.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1f5000c3f0 con 0x7f1f50004810 2024-04-26T02:28:50.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f62ffd640 1 --2- 172.21.15.94:0/892463742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f500039c0 0x7f1f50003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f5bfff640 1 --2- 172.21.15.94:0/892463742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f50004810 0x7f1f50004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f637fe640 1 --2- 172.21.15.94:0/892463742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f50005260 0x7f1f5000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f62ffd640 1 -- 172.21.15.94:0/892463742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f50004810 msgr2=0x7f1f50004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f62ffd640 1 --2- 172.21.15.94:0/892463742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f50004810 0x7f1f50004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f62ffd640 1 -- 172.21.15.94:0/892463742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f50005260 msgr2=0x7f1f5000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f62ffd640 1 --2- 172.21.15.94:0/892463742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f50005260 0x7f1f5000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f637fe640 1 -- 172.21.15.94:0/892463742 reap_dead start 2024-04-26T02:28:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.933+0000 7f1f62ffd640 1 -- 172.21.15.94:0/892463742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f50003530 con 0x7f1f500039c0 2024-04-26T02:28:50.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.937+0000 7f1f62ffd640 1 --2- 172.21.15.94:0/892463742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f500039c0 0x7f1f50003e30 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f1f54001780 tx=0x7f1f5401c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c4a78f409aae44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:50.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.937+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f540013a0 con 0x7f1f500039c0 2024-04-26T02:28:50.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.937+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f540052f0 con 0x7f1f500039c0 2024-04-26T02:28:50.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.937+0000 7f1f627fc640 1 -- 172.21.15.94:0/892463742 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54003840 con 0x7f1f500039c0 2024-04-26T02:28:50.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.937+0000 7f1f697ee640 1 -- 172.21.15.94:0/892463742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f500039c0 msgr2=0x7f1f50003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:50.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.937+0000 7f1f697ee640 1 --2- 172.21.15.94:0/892463742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f500039c0 0x7f1f50003e30 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f1f54001780 tx=0x7f1f5401c1f0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.937+0000 7f1f697ee640 1 -- 172.21.15.94:0/892463742 shutdown_connections 2024-04-26T02:28:50.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.937+0000 7f1f697ee640 1 --2- 172.21.15.94:0/892463742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f500039c0 0x7f1f50003e30 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.937+0000 7f1f697ee640 1 -- 172.21.15.94:0/892463742 >> 172.21.15.94:0/892463742 conn(0x7f1f640b47f0 msgr2=0x7f1f640b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:50.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.937+0000 7f1f697ee640 1 -- 172.21.15.94:0/892463742 shutdown_connections 2024-04-26T02:28:50.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.937+0000 7f1f697ee640 1 -- 172.21.15.94:0/892463742 wait complete. 2024-04-26T02:28:50.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f697ee640 1 Processor -- start 2024-04-26T02:28:50.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f697ee640 1 -- start start 2024-04-26T02:28:50.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f697ee640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f500039c0 0x7f1f6415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f697ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f64146970 0x7f1f64146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f697ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f64158cd0 0x7f1f641590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f697ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6415cbf0 con 0x7f1f64146970 2024-04-26T02:28:50.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f697ee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f64073ca0 con 0x7f1f64158cd0 2024-04-26T02:28:50.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f697ee640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1f640afdf0 con 0x7f1f500039c0 2024-04-26T02:28:50.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f5bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f64146970 0x7f1f64146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f5bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f64146970 0x7f1f64146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35746/0 (socket says 172.21.15.94:35746) 2024-04-26T02:28:50.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f5bfff640 1 -- 172.21.15.94:0/2667613477 learned_addr learned my addr 172.21.15.94:0/2667613477 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:50.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f637fe640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f64158cd0 0x7f1f641590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f62ffd640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f500039c0 0x7f1f6415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f5bfff640 1 -- 172.21.15.94:0/2667613477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f500039c0 msgr2=0x7f1f6415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f5bfff640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f500039c0 0x7f1f6415e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f5bfff640 1 -- 172.21.15.94:0/2667613477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f64158cd0 msgr2=0x7f1f641590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f5bfff640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f64158cd0 0x7f1f641590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f5bfff640 1 -- 172.21.15.94:0/2667613477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6415d200 con 0x7f1f64146970 2024-04-26T02:28:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f5bfff640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f64146970 0x7f1f64146d80 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c011600 tx=0x7f1f4c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:50.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f5b7fe640 1 -- 172.21.15.94:0/2667613477 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c004220 con 0x7f1f64146970 2024-04-26T02:28:50.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f5b7fe640 1 -- 172.21.15.94:0/2667613477 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4c0043c0 con 0x7f1f64146970 2024-04-26T02:28:50.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f697ee640 1 -- 172.21.15.94:0/2667613477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6415d4e0 con 0x7f1f64146970 2024-04-26T02:28:50.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f5b7fe640 1 -- 172.21.15.94:0/2667613477 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00f960 con 0x7f1f64146970 2024-04-26T02:28:50.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.941+0000 7f1f697ee640 1 -- 172.21.15.94:0/2667613477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6415daa0 con 0x7f1f64146970 2024-04-26T02:28:50.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.945+0000 7f1f697ee640 1 -- 172.21.15.94:0/2667613477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f64076830 con 0x7f1f64146970 2024-04-26T02:28:50.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.945+0000 7f1f5b7fe640 1 -- 172.21.15.94:0/2667613477 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1f4c00fb00 con 0x7f1f64146970 2024-04-26T02:28:50.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.945+0000 7f1f5b7fe640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1f4406f400 0x7f1f44071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:50.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.945+0000 7f1f62ffd640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1f4406f400 0x7f1f44071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:50.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.945+0000 7f1f5b7fe640 1 -- 172.21.15.94:0/2667613477 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f1f4c08f6f0 con 0x7f1f64146970 2024-04-26T02:28:50.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.945+0000 7f1f62ffd640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1f4406f400 0x7f1f44071910 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1f54001750 tx=0x7f1f540011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:50.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:50.949+0000 7f1f5b7fe640 1 -- 172.21.15.94:0/2667613477 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1f4c019030 con 0x7f1f64146970 2024-04-26T02:28:51.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.105+0000 7f1f697ee640 1 -- 172.21.15.94:0/2667613477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f1f641601d0 con 0x7f1f64146970 2024-04-26T02:28:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.113+0000 7f1f5b7fe640 1 -- 172.21.15.94:0/2667613477 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f1f4c08f0c0 con 0x7f1f64146970 2024-04-26T02:28:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.117+0000 7f1f697ee640 1 -- 172.21.15.94:0/2667613477 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1f4406f400 msgr2=0x7f1f44071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.117+0000 7f1f697ee640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1f4406f400 0x7f1f44071910 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1f54001750 tx=0x7f1f540011a0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.117+0000 7f1f697ee640 1 -- 172.21.15.94:0/2667613477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f64146970 msgr2=0x7f1f64146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:51.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.117+0000 7f1f697ee640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f64146970 0x7f1f64146d80 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c011600 tx=0x7f1f4c011ac0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.117+0000 7f1f697ee640 1 -- 172.21.15.94:0/2667613477 shutdown_connections 2024-04-26T02:28:51.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.117+0000 7f1f697ee640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f64158cd0 0x7f1f641590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.117+0000 7f1f697ee640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f64146970 0x7f1f64146d80 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.117+0000 7f1f697ee640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f1f4406f400 0x7f1f44071910 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.117+0000 7f1f697ee640 1 --2- 172.21.15.94:0/2667613477 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f500039c0 0x7f1f6415e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.117+0000 7f1f697ee640 1 -- 172.21.15.94:0/2667613477 >> 172.21.15.94:0/2667613477 conn(0x7f1f640b47f0 msgr2=0x7f1f6406f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:51.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.117+0000 7f1f697ee640 1 -- 172.21.15.94:0/2667613477 shutdown_connections 2024-04-26T02:28:51.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.117+0000 7f1f697ee640 1 -- 172.21.15.94:0/2667613477 wait complete. 2024-04-26T02:28:51.143 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T02:28:51.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T02:28:51.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f491089a640 1 Processor -- start 2024-04-26T02:28:51.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f491089a640 1 -- start start 2024-04-26T02:28:51.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f491089a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f490c06f6c0 0x7f490c0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:51.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f491089a640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f490c15ff50 0x7f490c0d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:51.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f491089a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f490c0b7050 0x7f490c0b7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:51.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f491089a640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f490c0d2fd0 con 0x7f490c15ff50 2024-04-26T02:28:51.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f491089a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f490c0d2070 con 0x7f490c06f6c0 2024-04-26T02:28:51.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f491089a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f490c0d2aa0 con 0x7f490c0b7050 2024-04-26T02:28:51.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f4909d9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f490c15ff50 0x7f490c0d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:51.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f4909d9b640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f490c15ff50 0x7f490c0d3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35748/0 (socket says 172.21.15.94:35748) 2024-04-26T02:28:51.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f4909d9b640 1 -- 172.21.15.94:0/1872816666 learned_addr learned my addr 172.21.15.94:0/1872816666 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:51.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f490a59c640 1 --2- 172.21.15.94:0/1872816666 >> v2:172.21.15.138:3300/0 conn(0x7f490c06f6c0 0x7f490c0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:51.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f4909d9b640 1 -- 172.21.15.94:0/1872816666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f490c0b7050 msgr2=0x7f490c0b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:51.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f490ad9d640 1 --2- 172.21.15.94:0/1872816666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f490c0b7050 0x7f490c0b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:51.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f4909d9b640 1 --2- 172.21.15.94:0/1872816666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f490c0b7050 0x7f490c0b7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f4909d9b640 1 -- 172.21.15.94:0/1872816666 >> v2:172.21.15.138:3300/0 conn(0x7f490c06f6c0 msgr2=0x7f490c0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:28:51.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f4909d9b640 1 --2- 172.21.15.94:0/1872816666 >> v2:172.21.15.138:3300/0 conn(0x7f490c06f6c0 0x7f490c0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f4909d9b640 1 -- 172.21.15.94:0/1872816666 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c0d1d40 con 0x7f490c15ff50 2024-04-26T02:28:51.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.273+0000 7f490a59c640 1 --2- 172.21.15.94:0/1872816666 >> v2:172.21.15.138:3300/0 conn(0x7f490c06f6c0 0x7f490c0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:51.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f4909d9b640 1 --2- 172.21.15.94:0/1872816666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f490c15ff50 0x7f490c0d3a10 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f490c0d1f10 tx=0x7f490001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9543f17aea6503b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f490959a640 1 -- 172.21.15.94:0/1872816666 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49000aa660 con 0x7f490c15ff50 2024-04-26T02:28:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f490959a640 1 -- 172.21.15.94:0/1872816666 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f49000af070 con 0x7f490c15ff50 2024-04-26T02:28:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f490959a640 1 -- 172.21.15.94:0/1872816666 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49000aaa40 con 0x7f490c15ff50 2024-04-26T02:28:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f491089a640 1 -- 172.21.15.94:0/1872816666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f490c15ff50 msgr2=0x7f490c0d3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f491089a640 1 --2- 172.21.15.94:0/1872816666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f490c15ff50 0x7f490c0d3a10 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f490c0d1f10 tx=0x7f490001c520 comp rx=0 tx=0).stop 2024-04-26T02:28:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f491089a640 1 -- 172.21.15.94:0/1872816666 shutdown_connections 2024-04-26T02:28:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f491089a640 1 --2- 172.21.15.94:0/1872816666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f490c0b7050 0x7f490c0b7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f491089a640 1 --2- 172.21.15.94:0/1872816666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f490c15ff50 0x7f490c0d3a10 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f491089a640 1 --2- 172.21.15.94:0/1872816666 >> v2:172.21.15.138:3300/0 conn(0x7f490c06f6c0 0x7f490c0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f491089a640 1 -- 172.21.15.94:0/1872816666 >> 172.21.15.94:0/1872816666 conn(0x7f490c09e950 msgr2=0x7f490c08d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:51.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f491089a640 1 -- 172.21.15.94:0/1872816666 shutdown_connections 2024-04-26T02:28:51.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f491089a640 1 -- 172.21.15.94:0/1872816666 wait complete. 2024-04-26T02:28:51.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f491089a640 1 Processor -- start 2024-04-26T02:28:51.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.277+0000 7f491089a640 1 -- start start 2024-04-26T02:28:51.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f491089a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f490c06f6c0 0x7f490c0ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:51.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f491089a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f490c0b7050 0x7f490c0cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:51.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f491089a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f490c15ff50 0x7f490c155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:51.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f491089a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f490c06fd90 con 0x7f490c15ff50 2024-04-26T02:28:51.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f491089a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f490c06fc10 con 0x7f490c06f6c0 2024-04-26T02:28:51.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f491089a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f490c0d5600 con 0x7f490c0b7050 2024-04-26T02:28:51.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f490ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f490c15ff50 0x7f490c155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:51.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f490ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f490c15ff50 0x7f490c155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35762/0 (socket says 172.21.15.94:35762) 2024-04-26T02:28:51.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f490ad9d640 1 -- 172.21.15.94:0/1651639799 learned_addr learned my addr 172.21.15.94:0/1651639799 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-26T02:28:51.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f4909d9b640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f490c0b7050 0x7f490c0cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:51.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f490a59c640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f490c06f6c0 0x7f490c0ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:51.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f490ad9d640 1 -- 172.21.15.94:0/1651639799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f490c0b7050 msgr2=0x7f490c0cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:51.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f490ad9d640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f490c0b7050 0x7f490c0cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f490ad9d640 1 -- 172.21.15.94:0/1651639799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f490c06f6c0 msgr2=0x7f490c0ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:51.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f490ad9d640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f490c06f6c0 0x7f490c0ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f490ad9d640 1 -- 172.21.15.94:0/1651639799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c0cf420 con 0x7f490c15ff50 2024-04-26T02:28:51.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f490a59c640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f490c06f6c0 0x7f490c0ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:51.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f4909d9b640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f490c0b7050 0x7f490c0cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:28:51.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f490ad9d640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f490c15ff50 0x7f490c155d70 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00c5f0 tx=0x7f48fc00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:51.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f48faffd640 1 -- 172.21.15.94:0/1651639799 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc015070 con 0x7f490c15ff50 2024-04-26T02:28:51.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f48faffd640 1 -- 172.21.15.94:0/1651639799 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f48fc004220 con 0x7f490c15ff50 2024-04-26T02:28:51.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f48faffd640 1 -- 172.21.15.94:0/1651639799 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc003620 con 0x7f490c15ff50 2024-04-26T02:28:51.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f491089a640 1 -- 172.21.15.94:0/1651639799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c0cee60 con 0x7f490c15ff50 2024-04-26T02:28:51.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f491089a640 1 -- 172.21.15.94:0/1651639799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c06c730 con 0x7f490c15ff50 2024-04-26T02:28:51.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f491089a640 1 -- 172.21.15.94:0/1651639799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f490c0cebd0 con 0x7f490c15ff50 2024-04-26T02:28:51.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.281+0000 7f48faffd640 1 -- 172.21.15.94:0/1651639799 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f48fc0037c0 con 0x7f490c15ff50 2024-04-26T02:28:51.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.285+0000 7f48faffd640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f48ec06f400 0x7f48ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:28:51.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.285+0000 7f48faffd640 1 -- 172.21.15.94:0/1651639799 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f48fc08eee0 con 0x7f490c15ff50 2024-04-26T02:28:51.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.285+0000 7f490a59c640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f48ec06f400 0x7f48ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:28:51.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.285+0000 7f490a59c640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f48ec06f400 0x7f48ec071910 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f490c0d0280 tx=0x7f49040084c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:28:51.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.289+0000 7f48faffd640 1 -- 172.21.15.94:0/1651639799 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f48fc05bf50 con 0x7f490c15ff50 2024-04-26T02:28:51.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.485+0000 7f491089a640 1 -- 172.21.15.94:0/1651639799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f490c076830 con 0x7f490c15ff50 2024-04-26T02:28:51.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.489+0000 7f48faffd640 1 -- 172.21.15.94:0/1651639799 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3870 (secure 0 0 0) 0x7f48fc05b850 con 0x7f490c15ff50 2024-04-26T02:28:51.494 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-26T02:28:51.494 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4757,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/478421533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":478421533},{"type":"v1","addr":"172.21.15.138:6833","nonce":478421533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4760,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2461866660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2461866660},{"type":"v1","addr":"172.21.15.138:6835","nonce":2461866660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5439,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2566493915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2566493915},{"type":"v1","addr":"172.21.15.94:6835","nonce":2566493915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":10518,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/759544782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":759544782},{"type":"v1","addr":"172.21.15.94:6837","nonce":759544782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-04-26T02:28:51.494 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 20 2024-04-26T02:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.489+0000 7f491089a640 1 -- 172.21.15.94:0/1651639799 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f48ec06f400 msgr2=0x7f48ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.489+0000 7f491089a640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f48ec06f400 0x7f48ec071910 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f490c0d0280 tx=0x7f49040084c0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.489+0000 7f491089a640 1 -- 172.21.15.94:0/1651639799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f490c15ff50 msgr2=0x7f490c155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.489+0000 7f491089a640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f490c15ff50 0x7f490c155d70 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00c5f0 tx=0x7f48fc00cab0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.489+0000 7f491089a640 1 -- 172.21.15.94:0/1651639799 shutdown_connections 2024-04-26T02:28:51.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.489+0000 7f491089a640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f490c15ff50 0x7f490c155d70 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.489+0000 7f491089a640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.94:6832/1891059149,v1:172.21.15.94:6833/1891059149] conn(0x7f48ec06f400 0x7f48ec071910 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.489+0000 7f491089a640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f490c0b7050 0x7f490c0cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.489+0000 7f491089a640 1 --2- 172.21.15.94:0/1651639799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f490c06f6c0 0x7f490c0ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:28:51.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.489+0000 7f491089a640 1 -- 172.21.15.94:0/1651639799 >> 172.21.15.94:0/1651639799 conn(0x7f490c09e950 msgr2=0x7f490c159e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:28:51.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.489+0000 7f491089a640 1 -- 172.21.15.94:0/1651639799 shutdown_connections 2024-04-26T02:28:51.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-26T02:28:51.489+0000 7f491089a640 1 -- 172.21.15.94:0/1651639799 wait complete. 2024-04-26T02:28:51.517 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-26T02:28:51.517 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T02:28:51.517 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T02:28:51.517 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:28:51.594 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T02:28:51.594 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T02:28:51.594 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:28:51.685 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T02:28:51.685 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T02:28:51.685 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:28:51.713 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T02:28:51.714 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T02:28:51.714 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:28:51.780 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T02:28:51.780 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T02:28:51.781 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T02:28:51.781 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:28:54.096 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-04-26T02:28:57.882 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T02:28:57.883 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T02:28:57.883 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:29:00.205 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-04-26T02:29:00.206 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-04-26T02:29:03.984 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T02:29:03.985 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T02:29:03.985 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:29:06.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-26T02:29:06.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-04-26T02:29:10.087 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T02:29:10.088 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T02:29:10.088 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:29:12.122 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-26T02:29:12.122 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-26T02:29:16.190 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T02:29:16.191 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T02:29:16.191 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:29:17.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-04-26T02:29:17.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-26T02:29:22.292 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T02:29:22.293 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T02:29:22.293 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:29:23.636 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-26T02:29:23.636 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-04-26T02:29:28.395 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T02:29:28.396 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T02:29:28.396 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:29:29.342 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-26T02:29:34.497 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T02:29:34.498 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T02:29:34.498 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:29:34.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-26T02:29:40.453 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-26T02:29:40.453 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-26T02:29:40.600 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T02:29:40.601 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T02:29:40.601 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T02:29:40.601 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:29:45.857 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-04-26T02:29:45.857 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-26T02:29:46.703 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T02:29:46.703 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T02:29:46.704 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:29:51.160 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-04-26T02:29:51.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-04-26T02:29:52.805 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T02:29:52.815 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T02:29:52.815 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T02:29:52.815 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:29:56.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-04-26T02:29:56.364 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-04-26T02:29:58.917 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T02:29:58.918 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T02:29:58.918 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:30:01.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-04-26T02:30:01.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-04-26T02:30:05.019 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T02:30:05.020 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T02:30:05.020 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T02:30:06.467 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-04-26T02:30:06.467 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-04-26T02:30:11.122 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T02:30:11.122 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T02:30:11.123 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-26T02:30:11.261 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-26T02:30:11.262 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T02:30:11.460 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-26T02:30:11.460 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T02:30:11.468 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-04-26T02:30:11.633 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-26T02:30:11.634 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T02:30:11.882 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi094.front.sepia.ceph.com 2024-04-26T02:30:11.882 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T02:30:12.110 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-26T02:30:12.110 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T02:30:12.423 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-26T02:30:12.423 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T02:30:12.678 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-26T02:30:12.678 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T02:30:12.911 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-26T02:30:12.911 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T02:30:13.144 INFO:tasks.ceph:Archiving mon data... 2024-04-26T02:30:13.146 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280/data/mon.a.tgz 2024-04-26T02:30:13.146 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-04-26T02:30:13.152 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.ppqn1JkJYF 2024-04-26T02:30:13.153 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ppqn1JkJYF 2024-04-26T02:30:13.331 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.ppqn1JkJYF 2024-04-26T02:30:13.422 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.ppqn1JkJYF is 600KB 2024-04-26T02:30:13.488 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.ppqn1JkJYF 2024-04-26T02:30:13.493 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280/data/mon.b.tgz 2024-04-26T02:30:13.494 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-04-26T02:30:13.500 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.a8KKGxjrho 2024-04-26T02:30:13.500 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.a8KKGxjrho 2024-04-26T02:30:13.690 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.a8KKGxjrho 2024-04-26T02:30:13.794 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.a8KKGxjrho is 614KB 2024-04-26T02:30:13.859 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.a8KKGxjrho 2024-04-26T02:30:13.867 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280/data/mon.c.tgz 2024-04-26T02:30:13.867 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-04-26T02:30:13.913 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.1VryqqmwWL 2024-04-26T02:30:13.914 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.1VryqqmwWL 2024-04-26T02:30:14.070 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.1VryqqmwWL 2024-04-26T02:30:14.133 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.1VryqqmwWL is 614KB 2024-04-26T02:30:14.200 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.1VryqqmwWL 2024-04-26T02:30:14.207 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T02:30:14.208 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T02:30:14.211 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T02:30:14.305 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T02:30:14.355 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T02:30:14.355 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T02:30:14.357 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280/remote/smithi094/crash 2024-04-26T02:30:14.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T02:30:14.371 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280/remote/smithi138/crash 2024-04-26T02:30:14.372 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T02:30:14.411 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T02:30:14.412 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph.py", line 257, in ceph_log logrotater.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph.py", line 222, in end self.thread.get() File "src/gevent/greenlet.py", line 805, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph.py", line 189, in invoke_logrotate run.wait(procs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 3: 'sudo logrotate /etc/logrotate.d/ceph-test.conf' 2024-04-26T02:30:14.480 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T02:30:14.493 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph.py", line 257, in ceph_log logrotater.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph.py", line 222, in end self.thread.get() File "src/gevent/greenlet.py", line 805, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph.py", line 189, in invoke_logrotate run.wait(procs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 3: 'sudo logrotate /etc/logrotate.d/ceph-test.conf' 2024-04-26T02:30:14.497 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T02:30:14.498 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T02:30:14.516 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T02:30:14.586 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T02:30:14.597 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T02:30:14.597 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T02:30:14.600 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T02:30:14.668 INFO:teuthology.orchestra.run.smithi094.stdout: remote refid st t when poll reach delay offset jitter 2024-04-26T02:30:14.668 INFO:teuthology.orchestra.run.smithi094.stdout:============================================================================== 2024-04-26T02:30:14.668 INFO:teuthology.orchestra.run.smithi094.stdout:+hv01.front.sepi 67.205.162.81 3 u 433 1024 377 0.187 -2.674 1.650 2024-04-26T02:30:14.668 INFO:teuthology.orchestra.run.smithi094.stdout:*hv02.front.sepi 63.231.80.2 3 u 332 1024 377 0.179 +1.163 0.228 2024-04-26T02:30:14.668 INFO:teuthology.orchestra.run.smithi094.stdout:+hv03.front.sepi 74.6.168.72 3 u 838 1024 377 0.142 -0.448 1.424 2024-04-26T02:30:14.669 INFO:teuthology.orchestra.run.smithi094.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-04-26T02:30:14.673 INFO:teuthology.orchestra.run.smithi138.stdout: remote refid st t when poll reach delay offset jitter 2024-04-26T02:30:14.673 INFO:teuthology.orchestra.run.smithi138.stdout:============================================================================== 2024-04-26T02:30:14.673 INFO:teuthology.orchestra.run.smithi138.stdout:+hv01.front.sepi 67.205.162.81 3 u 104 256 377 0.167 +2.010 1.423 2024-04-26T02:30:14.673 INFO:teuthology.orchestra.run.smithi138.stdout:+hv02.front.sepi 63.231.80.2 3 u 161 256 377 0.125 +6.015 1.104 2024-04-26T02:30:14.673 INFO:teuthology.orchestra.run.smithi138.stdout:*hv03.front.sepi 74.6.168.72 3 u 43 256 377 0.153 +3.663 1.488 2024-04-26T02:30:14.674 INFO:teuthology.orchestra.run.smithi138.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-04-26T02:30:14.675 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T02:30:14.688 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T02:30:14.689 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T02:30:14.704 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T02:30:14.740 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T02:30:14.772 INFO:teuthology.task.internal:Duration was 35453.002502 seconds 2024-04-26T02:30:14.772 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T02:30:14.805 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T02:30:14.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T02:30:14.808 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T02:30:14.990 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T02:30:14.990 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-04-26T02:30:14.990 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T02:30:15.011 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-04-26T02:30:15.011 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T02:30:15.022 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T02:30:15.022 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T02:30:15.054 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T02:30:15.190 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T02:30:15.191 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T02:30:15.194 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T02:30:15.937 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T02:30:15.948 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T02:30:15.948 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T02:30:15.975 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T02:30:16.035 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T02:30:16.046 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:30:16.048 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:30:16.074 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-04-26T02:30:16.082 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-04-26T02:30:16.105 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:30:16.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:30:16.150 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:30:16.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:30:16.155 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T02:30:16.166 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T02:30:16.166 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280/remote/smithi094 2024-04-26T02:30:16.167 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T02:30:16.219 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280/remote/smithi138 2024-04-26T02:30:16.219 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T02:30:16.240 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T02:30:16.240 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T02:30:16.259 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T02:30:16.281 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T02:30:16.292 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T02:30:16.293 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T02:30:16.313 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T02:30:16.314 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T02:30:16.320 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T02:30:16.323 INFO:teuthology.orchestra.run.smithi094.stdout: 658639 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 26 02:30 /home/ubuntu/cephtest 2024-04-26T02:30:16.324 INFO:teuthology.orchestra.run.smithi094.stdout: 788827 4 d--------- 2 ubuntu ubuntu 4096 Apr 25 16:52 /home/ubuntu/cephtest/mnt.0 2024-04-26T02:30:16.324 INFO:teuthology.orchestra.run.smithi094.stderr:find: '/home/ubuntu/cephtest/mnt.0': Permission denied 2024-04-26T02:30:16.325 INFO:teuthology.orchestra.run.smithi094.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-26T02:30:16.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:30:16.326 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph.py", line 257, in ceph_log logrotater.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph.py", line 222, in end self.thread.get() File "src/gevent/greenlet.py", line 805, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph.py", line 189, in invoke_logrotate run.wait(procs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 3: 'sudo logrotate /etc/logrotate.d/ceph-test.conf' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-26T02:30:16.342 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T02:30:16.354 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T02:30:16.382 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi094 with status 3: 'sudo logrotate /etc/logrotate.d/ceph-test.conf' 2024-04-26T02:30:16.382 INFO:teuthology.nuke:Checking targets against current locks 2024-04-26T02:30:16.391 INFO:teuthology.orchestra.run.smithi138.stdout: 658616 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 02:30 /home/ubuntu/cephtest 2024-04-26T02:30:16.415 DEBUG:teuthology.nuke:shortname: smithi094 2024-04-26T02:30:16.416 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T02:30:16.442 DEBUG:teuthology.nuke:shortname: smithi138 2024-04-26T02:30:16.442 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T02:30:16.446 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:30:25.219560', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGu8NGEuGQkb1jV9Xb27i1LpCP0Jj07s6pidfWmZr3LyeUMosqY0oO8/gJWyAfPqsll1QwH4HI+dPvOXZsNhRbE='} 2024-04-26T02:30:16.472 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673280', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:30:25.220807', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBFEDbrXlp5aEZYUnjRaYzIiE2ZC7qbzwLmxJvka+WOTYpvlaj5Y/OswrFC2ZJZx2t7z6yFZqE062kaVp4nguaU='} 2024-04-26T02:30:16.499 INFO:teuthology.orchestra.console.smithi094:Power off 2024-04-26T02:30:16.499 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T02:30:16.539 INFO:teuthology.orchestra.console.smithi138:Power off 2024-04-26T02:30:16.540 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T02:30:16.563 DEBUG:teuthology.orchestra.console.smithi094:power off output: Chassis Power Control: Down/Off 2024-04-26T02:30:16.563 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T02:30:16.573 DEBUG:teuthology.orchestra.console.smithi138:power off output: Chassis Power Control: Down/Off 2024-04-26T02:30:16.573 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T02:30:16.587 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-04-26T02:30:16.597 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-26T02:30:20.589 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T02:30:20.598 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T02:30:20.699 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-04-26T02:30:20.712 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-26T02:30:24.702 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T02:30:24.712 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T02:30:24.814 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2024-04-26T02:30:24.822 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-26T02:30:24.915 INFO:teuthology.orchestra.console.smithi094:Power off completed 2024-04-26T02:30:28.825 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T02:30:28.935 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2024-04-26T02:30:29.036 INFO:teuthology.orchestra.console.smithi138:Power off completed 2024-04-26T02:30:29.141 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} duration: 35453.00250196457 failure_reason: 'Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev)' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=a0837c9ee4e241a5b972501401419688 status: fail success: false 2024-04-26T02:30:29.141 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T02:30:29.224 INFO:teuthology.run:FAIL